instructables

VR Vision

by Pratyush Sharma

VR Vision bot was showcased in TechEvince 5.0 , IIT
Guwahati , as a simple image processing based
human-machine interface with a picam fitted in it
through which we can see live reality on oculus .Its
utility lies in the fact that no additional sensors or
wearable except a glove is required to control the bot
that runs on differential drive principle. In this
instructable, we will take you through the working
principle behind object tracking , gesture detection
and live video transfer and detection used in the
system. The source code of this project can be
downloaded from Github via link:
https://github.com/shwetank6044/gesture-hawk.git
Virtual reality is used to describe 3D, computer
generated environment which can be explored and
interacted by a person . In this project , we have used
the VR to show the reality which we can’t see by just
sitting in our room , through VR vision robot and hand
gestures . The person using the VR headset is able to
interact with the robot through hand gestures and
head movement and see the reality outside through
the pi cam fitted on it without actually going out .

VR Vision: Page 1
Step 1: Items Required

1)L298N Motor Driver

2)DC Motors

3)Robot car chassis

4)Arduino Uno

5)LiPo Batteries

6)Arduino USB Cable(long)

7)Servo (2) - one 180 degree and one 360 degree

8)Rasberry pi

9)oculus

10)Jumper wires

VR Vision: Page 2
VR Vision: Page 3
Step 2: Working Principle

VR Vision: Page 4
Step 3: Working

The PiCam feed from the robot is hosted on a local controlled accordingly.
Wi-Fi
Link for working of gesture car -
network using a Raspberry Pi. An MPU6050 is used
to track the head movement, which accordingly www.instructables.com/id/Gesture-Control-Car-
controls the PiCam placed on the servo mount so that Using... Mpu6050-and-Arduino/
the feed can be received from all directions. The feed
is then received and displayed on a VR Headset. Link for video transfer working -
Another MPU6050 module is used to read the angles.
An RF Transmitter sends the data to the Arduino on https://www.youtube.com/watch?v=JjPsW-7FUng
the bot. The data is processed and the bot is

Step 4: CODES

VR Vision: Page 5
All the respective connections are given in the Code . int val;

NRF TRANSMITTER MPU9250 mpu;

#include
// void setup() { radio.begin(); Serial.begin(115200);
#include radio.openWritingPipe(pipe);

#include //Sending the Data } void loop() {

while(Serial.available()) { Vector normaccel =
#include mpu.readNormalizeAccel();

#include int pitch = -(atan2(normAccel.XAxis,

sqrt(normAccel.YAxis*normAccel.YAxis +
"RF24.h" //SCK -> 13//MISO -> 12//MOSI -> 11//CSN normAccel.ZAxis*normAccel.ZAxis))*180.0)/M_PI;
-> 7//CE ->8
int roll = (atan2(normAccel.YAxis,
int msg[1]; normAccel.ZAxis)*180.0)/M_PI;

RF24 radio(8,7); Serial.print(" \nPitch = ");

const uint64_t pipe = 0xE8E8F0F0E1LL; Serial.print(pitch);

int potpin = 0; Serial.print(" Roll = ");

VR Vision: Page 6
Serial.print(roll); delay(1000); }

if(pitch > 20) { msg[0] = 1; }

Serial.println(msg[0]); }

radio.write(msg, 1); NRF RECIEVER

delay(1000); //connection for rf receiver

} else if(pitch < -20) { msg[0] = 2; //GND = GND

Serial.println(msg[0]); //DATA = 12

radio.write(msg, 1); //VCC = 5V

delay(1000); #include

} else { msg[0] = 0; //connection for motors

Serial.println(msg[0]); //motor one int enA = 10;

radio.write(msg, 1); int in1 = 9; int in2 = 8;

delay(1000); // motor two int enB = 11;

} int in3 = 7;

if(roll > 20) { msg[0] = 3; int in4 = 6;

Serial.println(msg[0]); void setup() { vw_set_ptt_inverted(true);

radio.write(msg, 1); // Required for DR3100 vw_set_rx_pin(12);

delay(1000); } vw_setup(4000);

else if(roll < -20) { msg[0] = 4; // Bits per sec

Serial.println(msg[0]); // set all the motor control pins to outputs

radio.write(msg, 1); pinMode(enA, OUTPUT);

delay(1000); } pinMode(enB, OUTPUT);

else { msg[0] = 0; pinMode(in1, OUTPUT);

Serial.println(msg[0]); pinMode(in2, OUTPUT);

radio.write(msg, 1); pinMode(in3, OUTPUT);

VR Vision: Page 7
pinMode(in4, OUTPUT); digitalWrite(in2, LOW);

Serial.begin(9600); vw_rx_start(); analogWrite(enA,255);

// Start the receiver PLL running digitalWrite(in3, LOW);

} digitalWrite(in4, HIGH);

void loop() analogWrite(enB,255);

{ }

uint8_t buf[VW_MAX_MESSAGE_LEN]; if(buf[0]=='s'){ Serial.print("stop");

uint8_t buflen = 2; Serial.println(buf[1]);

if (vw_get_message(buf, &buflen)) digitalWrite(in1, LOW);

// Non-blocking digitalWrite(in2, LOW);

{ analogWrite(enA,255);

Serial.print(buf[0]) ; digitalWrite(in3, LOW);

Serial.println(buf[1]); digitalWrite(in4, LOW);

if(buf[0]=='f'){ Serial.print("forward"); analogWrite(enB,255);

Serial.println(buf[1]); }

digitalWrite(in1, LOW); if(buf[0]=='l'){ Serial.print("left");

digitalWrite(in2, HIGH); Serial.println(buf[1]);

analogWrite(enA,255); digitalWrite(in1, LOW);

digitalWrite(in3, HIGH); digitalWrite(in2, HIGH);

digitalWrite(in4, LOW); analogWrite(enA,255);

analogWrite(enB,255); digitalWrite(in3, LOW);

} digitalWrite(in4, HIGH);

if(buf[0]=='b') { Serial.print("backward"); analogWrite(enB,255);

Serial.println(buf[1]); }

digitalWrite(in1, HIGH); if(buf[0]=='r'){ Serial.print("right");

VR Vision: Page 8
VR Vision: Page 9
Step 5: Procedure

STEP 1: ASSEMBLING THE BASE We used a Occulus in which MPU6050 Sensor and
NRF Transmitter was attached. We used MPU6050
A wooden base was taken in which tires were fitted. as Accelerometer and Gyrometer and was attached
The wooden base was designed as a car base. Some to the NRF Transmitter which was transferring the
slots were made for keeping the Arduino and Power data based on our head motions. The NRF
Supply in place. A wooden Rod was fitted vertically transmitter was connected to an Arduino UNO. The
over the centre of the base that would support the orientation of the head decides the pitch and the roll.
Camera and Servo Motors. A portion of the top of the The Gestures we used were:
Wooden Stick was hollowed out to support the
Servos. * Left – Left Movement

STEP 2: MOTION * Right – Right Movement

Two Motors were used for the rear wheel tires. Motor
Driver(L298N) was connected to the motors for
control in speed and turns. Arduino was connected to
the motor driver to control it.
STEP 3: GESTURE CONTROL
We used a Glove in which MPU6050 Sensor and RF
Transmitter was attached. We used MPU6050 as
Accelerometer and Gyrometer and was attached to
the RF Transmitter which was transferring the data
based on our hand motions. The RF transmitter was
connected to an Arduino Mini. The orientation of the
hand decides the pitch and the roll. The Gestures we
used were:
* Left – Left Movement
* Right – Right Movement
* Up – Backward Movement
* Down – Forward Movement
* Stable – No Movement
There was an RF reciever which was connected to an uptight, then four 3V batteries were connected to the
Arduino Mini attached to the bot. It was recieving the two respective Arduinos and One Li-PO was used to
data for specific movement.
* Up – Upward Movement
* Down – Downward Movement
* Stable – No Movement
NRF Reciever was recieving the data motions of our
heads. The reciever was connected to the Servo
motor which was present on the top part of the
wooden stick. The WebCam was attached to the
servo motor for respective motions. Servo and
Reciever were connected to Arduino UNO and
Webcam was connected to Raspberry Pi with the
respective connections.
STEP 5: Video Transfer
The video transfer was from the Raspberry-Pi to the
phone we were using. Further we created a local host
server so that anyone who has the IP address of that
server can see the video. The Phone was placed in
the Oculus as well for VR VISION.
STEP 6: Power Supply
When all the connections were done and were
run the motor driver.

STEP 4: HEAD MOVEMENT (GESTURE

CONTROLLED)

VR Vision: Page 10
That looks really cool. Could you share more about the physical assembly process?

VR Vision: Page 11