instructables

SPY-DER : a Speech & Web Controlled Surveillance Spider Robot

by Sparklers_The_Makers

SPY-DER is a speech and web-controlled surveillance spider robot. So basically, it is a robot, which can walk like a spider
and work as a spy through its camera, that’s why I have given it the name “SPY-DER”. One can control this robot in two
ways, rstly by giving it voice commands and secondly by a web-based control interface. So whenever I call it by its
nickname 'Bumblebee', it starts listening to me, and then based on my voice command it will act. Here, I am using mainly
two technologies, rst one is hot-word or wake-word detection and the second one is speech recognition. And with the
speech recognition, I am also using intent detection, so that I can give it the same command in di erent ways (i.e., “Wave
your hands” or “say hello” both the commands will make the robot wave its legs). Coming to the web control part, one can
simply open a particular URL in any browser and will be able to control the spider. The web-based interface contains all
the control options and one can control the spider using the buttons availabe there. Similarly, one can open another URL
and watch the live video feed from this robot’s camera. One can watch the demo of this project here:
https://youtu.be/3edXTxIZ_2U
Supplies:

I have used the following hardware components to build this:

a. Arduino Nano
b. Raspberry Pi Zero W
c. Raspberry Pi camera
d. 5v to 3.3v Logic level shifter
e. Nano 328P Expansion Adapter Breakout Board IO Shield
f. SG90 Mini Servo 12 pieces
g. Buck Converter Lm2596
h. Lithium-Ion Battery 2 pieces
i. LEDs
j. Jumper Wires
k. USB mic
l. OTG cable

https://youtu.be/3edXTxIZ_2U

SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 1

Step 1: Video Tutorial

If you prefer to watch a video, then here goes the full video tutorial for this project. Also, if you have any problem in
understanding any of the steps below, then also you can watch the video, where I have explained everything in details.
You can nd the rst part of the video at https://youtu.be/fnMmnd9k6q8 .

https://youtu.be/KkZiZggtvIU

Step 2: Overall Idea

Initially, I built a simple Bluetooth-controlled spider robot using an Arduino nano. But then I was able to control the robot
only using an android/IOS app, which was not cool obviously. Thus, I decided to implement the speech recognition, web
control, and surveillance part in it. Now implementing all these features using a single arduino was not possible, so
obviously, I needed a small computer here. That's why I decided to build this using Raspberry Pi zero. Now the whole
system can be implemented using a Raspberry Pi zero only, but in that case, I had to rewrite the whole spider movement
control code in Raspberry Pi, which would be really time taking. That's why I decided to connect the Raspberry Pi with
the Arduino nano through serial communication. Then the Arduino will be controlling all the movements of the spider
and the Raspberry Pi will simply control the Arduino, in this way using the same code with very few modi cations in
Arduino, I was able to implement all these new features in this project.
All the codes for this project are written in Python for Raspberry Pi. For the web-based control part, I have used Flask
framework. And the webpage was built using HTML, CSS and jQuery. For the live video streaming, I have used RPi-Cam-
Web-Interface (https://elinux.org/RPi-Cam-Web-Interface), due to the fact that the latency is very low here. And for the
speech recognition and hot word detection part I have used Picovoice (https://picovoice.ai/) and I have modi ed the
code in python. I have also tried to use local speech recognition systems, but as the ram and processing power is very
limited in Raspberry Pi zero, the accuracy was not that good and the latency was also very high.

Step 3: Designing and Printing Parts

While designing the robot, I have followed the the design of this existing robot
https://www.thingiverse.com/thing:2901132 . I have redesigned few parts in tinkercad (https://www.tinkercad.com/) and
all of those parts are available at https://www.thingiverse.com/thing:4815137 .Then I have 3d printed all those parts
using Ender 3 3d printer.

SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 2

SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 3
SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 4
Step 4: Attaching Servo Motors on the 3D Printed Parts

After printing the parts, you need to attach SG90 servo motors with the 3d printed parts.

SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 5

Step 5: Connecting All the Body Parts Together

Then, all the 3d printed body parts need to be connected together through the servo motors.

SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 6

SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 7
Step 6: Attaching Battery and Battery Management System

Then, the battery and battery management system need to be attached in the body of the robot. As I have connected
two lithium-ion batteries in series, I was getting around 7-8 volts. Thus I used LM2596 buck converter to reduce the
voltage to 5 volts, so that using this single power source Arduino, servo motors and Raspberry Pi can be powered.

SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 8

Step 7: Connecting Arduino in the Robot

At rst, Arduino nano need to be attached with the I/O shield and then all the servo motors need to attached with the I/O
shield. While attaching the servo motors, make sure you have attached them according to the numbering mentioned in
the servo numbering gure and circuit diagram gure.

SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 9

SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 10
Step 8: Uploading Code in Arduino

Next, you need to upload the Arduino sketch in the Arduino nano.

SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 11

At rst you need to calibrate the legs of the robot. The code is availabe at https://github.com/Arijit1080/Spidy-The-
Spider-Robot .The program is availabe in the Legs folder.
Next, you should check if the basic functions like forward, backward, left, right etc are working or not. For that you can
use the code availabe in the folder Program1.
To know more about last two steps, you can watch this video: https://youtu.be/fnMmnd9k6q8
Then, you can upload the nal code in Arduino which will work with Raspberry Pi. The code is availabe at
https://github.com/Arijit1080/SPY-DER-A-Speech-and-Web-Controlled-Surveillance-Spider-Robot-using-Raspberry-Pi-
and-Arduino .You need to upload the SPY-DER_Arduino.ino le in the Arduino.

Step 9: Raspberry Pi Setup With Mic & Camera

Next, you need to connect Raspberry Pi zero w with Raspberry Pi camera, USB mic and logic level shifter.
If you don't know about the connection for serial comunication, then watch this video : https://youtu.be/e04br5J4UpQ
To know, how to use Raspberry Pi camera with Raspberry Pi zero, watch this video: https://youtu.be/oo0A_yRrIxQ

Step 10: Connecting Raspberry Pi With Arduino

Then, you need to connect Arduino with the logic level shifter, so that the Arduino and Raspberry Pi will be connected
serially.
SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 12
Step 11: Setup Raspberry Pi

You need to Install all the required libraries in Raspberry Pi and upload the codes from my Github repository
(https://github.com/Arijit1080/SPY-DER-A-Speech-and-Web-Controlled-Surveillance-Spider-Robot-using-Raspberry-Pi-
and-Arduino).
In this step, you also need to check if the mic and camera connected with the Raspberry Pi are working properly or not.
Also, you need to clone the Picovoice ( https://picovoice.ai/) repository from Github
(https://github.com/Picovoice/picovoice) and required changes to that code are available in my Github repository.
For the web-control part Flask framework is required in python and all the python & HTML les are there in my
repository.
For the video streaming part, RPi-Cam-Web-Interface (https://elinux.org/RPi-Cam-Web-Interface) needed to be installed.
To know more about this step, you can watch this video: https://youtu.be/yzpqEw1kEGo

Step 12: Finishing the Build

Finally, close the lid after tting everything inside the body.

SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 13

SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 14
Step 13: Using the Robot

To run the web control interface:

cd web_control
python3 web_control.py

To run the speech control system:

cd picovoice
python3 demo/python/picovoice_demo_mic.py
--keyword_path resources/porcupine/resources/keyword_files/raspberry-pi/bumblebee_raspberry-pi.ppn
--context_path your_rhino_model

It is higly recommended that before running the codes, you should watch the build video here
https://youtu.be/KkZiZggtvIU , so that you will know which codes to modify and how to run them properly.
After running these codes, you can access the web control interface from any browser using the url
Raspberry_pi_ip:5010 and the live video stream using Raspberry_pi_ip:80

Step 14: Conclusion & Improvements

In this way, I have built this project. Though, one can build it in many di erent ways also. And also there is a space of
future improvements. One can try to use a local speech recognition system in this project. Also with the camera, many
image processing and AI based features can be added in this project. May be I will upgrade this project in near future, so
see you there :)

SPY-DER : a Speech & Web Controlled Surveillance Spider Robot: Page 15