2019 1st International Conference on Advances in Information Technology
Smart Mirror Using Raspberry Pi for Human
Monitoring and Intrusion Detection
Raju A. Nadaf Rubina M.
Department of Government
Polytechnic, Bagalkot,
Computer Science
India
(Autonomous) rmuddebihal@gmail.co
Basaveshwar m m
Engineering College
Bagalkot, India
raj.enggs@gmail.com
Abstract- The demand for latest technology is rising and there by
changing the way the world lives. The impact of technology is so
heavy on our lives that, we are surrounded by technology filled
equipments. Ranging from smart home to smart cities,
everything is turning to smart. The ease of life and comfort zone
also increasing with the advancement of technology. Hence, the
proposed model, The Smart Mirror is a system in which the
normal mirror will behave like a smart device. The Smart mirror
is designed using Raspberry Pi-3 model and a touch enabled
screen. The designed system is having two modes of operation
namely regular mode and the triggered mode. In regular mode, it
will act like a normal mirror and in triggered mode, the mirror
will act like a smart mirror, which will capable of accepting
commands and displays the results on the screen. There are three
ways in which commands can be issued to the Smart mirror
namely Voice, Touch and Mobile controlled commands. The
system displays weather, temperature and latest news on the
mirror. The system is primarily designed for the purpose of
Human Monitoring and also Intrusion Detection. The proposed
design is an interactive system and is made as a package bundled
with maximum possible features, which not only just displays
information over screen, but also can be used for providing
security. The system is built using hardware units like Raspberry
Pi-3 model, microphone, touch screen, mobile device, camera and
PIR (Passive Infrared Sensor) sensors and programming coded
in Python language. The Human Intrusion detection and Human
Monitoring is implemented using Yolo Machine learning
technique with OpenCV.
Keywords- Raspberry Pi, Human Monitoring, Yolo using
Raspberry, Intrusion Detection.
I. INTRODUCTION
The advancement in Technology has converted almost every
device as smart device in the present world scenario. Almost
all devices are becoming smart ranging from household things
to most advanced electronic gadgets.
The smart characteristics of present devices are due to the
incorporation of AI (Artificial Intelligence)
978-1-7281-3241-9/19/$31.00 © 2019 IEEE
Authorized licensed use limited to: Nitte Meenakshi Institute of Technology. Downloaded on April 08,2022 at 09:20:28 UTC from IEEE Xplore. Restrictions apply.
Sujata P. Vasudha M. Bonal
Government Department of
Polytechnic, Bagalkot, Computer Science
India
(Autonomous)
sujamoon20@gmail.co Basaveshwar
Engineering College
Bagalkot, India
vasudha_125@rediffmai
l.com
and Prediction Systems in those devices. So keeping such
thought in mind, a smart is proposed, which is capable of
displaying customized information on the display screen and
also act smartly by providing security when needed. The
system can be used to provide security in deployed
environment. It is also capable of Monitoring Humans. People
use mirrors usually for getting ready for the day’s work. Some
surveys say that, a person spends at least 28 minutes in front
of the mirror, per day. Hence, the basic is idea is to utilize this
time to keep a person updated with latest news, weather, date,
time, calendar and other updates. Due to present busy life, it
really difficult to take out time to check out for news and other
updates like weather. And also there is a lack of package that
combines and displays all such information in single package
and in a common screen. The proposed system is interactive in
nature, hence it is possible to give commands, such as voice
commands to the Smart Mirror even while grooming up, so as
to get required and related information on screen. There are
many systems proposed so far, but the main difference lies in
the way in which the product is being used. The available
products are mostly provide information and are passive in
nature with little interactivity implemented in them. The
primary task of such a system is to display information on the
mirror and they accept any one type of commands like voice
commands or touch commands or commands from mobile.
The screens of such Smart Mirrors are can be designed using a
LCD (Liquid Crystal Display) or LED (Light Emitting Diode)
monitor along with the two-way Acrylic sheet and a mirror.
The proposed system is can work with voice commands, touch
commands as well as mobile based commands. The proposed
work not only acts just as a means of providing information
but also provides security and can be used for human
monitoring. The system can be used for detection of Intrusion.
As soon as the activity/ motion/movement are being detected
by the PIR (Passive Infrared Sensor) sensor, the camera
connected to Raspberry Pi is activated. Image is being
captured by the camera connected to Raspberry Pi. Image
processing technique such as Background Subtraction and
116
 2019 1st International Conference on Advances in Information Technology
Simple Frame Difference Approach is used to detect the
intrusion. The intrusion occurred will be communicated to the
administrator of the mirror through alert message along with
the picture of an intruder.
The system is also being designed for Human Monitoring.
Nowadays the families are getting smaller in size. Hence it
becomes difficult to monitor children, elders and patients.
Because of busy schedule and working parents, it becomes
difficult task to monitor kids after they return from school.
The designed system is capable of monitoring human within a
coverage range of camera. If the human moves out of the sight
of the camera, then an alert message will be sent to the
administrator of the Smart Mirror. The Proposed model will
be helpful for Human Monitoring in home, monitoring of
prisoners in jail or hospital and employees in jewelry shop.
Voice commands can be used whenever the administrator of
Smart Mirror is in front of Smart Mirror, to push the Smart
Mirror either in Security mode or Human Monitoring mode
explicitly and Mobile commands can be used for the same
task, whenever the owner is not in front of mirror or away
from the mirror. The features of the proposed Smart Mirror
are shown in figure 1.
Fig. 1 Features of Smart Mirror
The system can be customized as per the needs of the user and
can be designed accordingly. The features can be added and
removed as per conveniences of the user. Figure 2 shows the
Smart mirror being designed with features shown in figure 1.
Fig. 2 Smart Mirror
978-1-7281-3241-9/19/$31.00 © 2019 IEEE
Authorized licensed use limited to: Nitte Meenakshi Institute of Technology. Downloaded on April 08,2022 at 09:20:28 UTC from IEEE Xplore. Restrictions apply.
II. RELATED WORK
Some related works have been already taken up in this field.
Such systems have been built with different hardware and
software. They also differ in design, capabilities, features,
applications and method or mode of operation.
Intelligent mirror which accepts voice command via the
microphone and been built with Raspberry Pi microcontroller,
LED monitor and acrylic mirror, displays the weather, time,
and location information on the screen [1]. Smart mirror built
with Raspberry Pi and MCU (Multi Control Unit) units
display weather and latest news updates on the screen.
Humidity and Temperature sensors are used and IoT is
implemented using cloud [2]. Futuristic multimedia based
Smart Mirrors are designed which accept voice commands.
Artificial Intelligence concepts are used. System alerts about
weather and suggests the user according to the weather. For
example, if it is cold day, it will display a message on the
mirror saying “Please wear jacket today” [3]
Some of the Smart mirror are having Webpage based
interface. These are operated using voice commands. These
make use of APIs of various website. The proposed mirror
makes use of Google Assistant and stores user details in the
database [4]. Some mirrors designed can be used as weight
and fitness trackers. The authentication is provided using Face
Recognition. GPS navigation, Bluetooth Connectivity and
wireless communication are added features. SONUS
technology is used for improved communication[5]. Some
mirrors designed use Hermoine 1.0, which is an extension of
Magic Mirror. The platform provides the user with easy
installation of a Smart Mirror for domestic use. System can be
used as a Home Assistant and is voice based [6]. Other
systems proposed work in two modes viz., Normal Mode and
Smart Mode. System is developed using Python and Javascript
programming tool such as Node.js. It is a voice command
based smart mirror [7]. Some Smart Mirrors implemented
with Raspberry and SMT32F030CT8T6 microcontroller as
core controlling chips. These are voice enabled and special
Speech Synthesis module is implemented using SYN6288
chip [8]
Smart mirror are designed for Theft Detection in a home
environment. PIR sensors are used for human motion
detection and the camera captures information and stores in
drop box. DHT22 chip is used for theft detection and VNC
viewer is used for mobile control [9]A comparative study of
Smart mirrors is given and a Voice based Smart mirror is
proposed. It is system which supports Human Gestures and
Face Detection. It is an AI based system. Machine Learning
Techniques are used for making system more responsive [10].
Health monitoring Smart Mirrors are designed to detect Health
Issues. System makes use of a special algorithm called PAA
(Posture Analysis Algorithm) to analyze postures of human to
find any changes in postures over a period of time
[11]Commercial and Home usage Smart Mirrors are designed
to capture real time data on the screen. System is voice based
and makes uses of AmI (Ambient Artificial Intelligence)
technique [12]. Multi-user Smart Mirrors are designed as
117
 2019 1st International Conference on Advances in Information Technology
commercial products which are based on RFID access of
employees. The device has a personalized user interface [13].
III. ISSUES AND CHALLENGES
The proposed model is designed for 3 modes of commands
namely Voice commands, Touch Commands and Mobile
based Commands. The device is primarily designed for
Human Intrusion Detection and Human Monitoring. All these
features implemented using Raspberry Pi as a heart of the
system. But, Raspberry is a microcontroller with limited
performance and storage capacity. The synchronization of all
these features into Smart Mirror is one of the challenges.
Power issues, delay in content delivery are some of the
observed issues. Sometimes SD card may be corrupted. These
are some of the common technical issues with the Raspberry
Pi device. PIR sensors are used for motion detection. Such
sensors have issues with the range and accuracy. The chance
of false alarm rate is also major issue.
The non-technical issues include cost and the durability of the
hardware devices. In order to reduce the cost of system one
can use the freeware and open-source software, but they lack
of user friendliness. Hence, it may make difficult for
programming and maintenance. The interfacing of the
hardware and software is actual challenge. Proper knowledge
of hardware devices and sensors is mandatory for providing
better and sophisticated solutions. Else it may lead to damage
of the sensors and other connected devices. Yolo techniques
needs fine tuning as far as Object detection is concerned.
IV. PROPOSED MODEL
Raspberry Pi is the heart of the proposed system. The block
diagram for the proposed model shows details. The Mirror
State is a synchronization unit. The main task of this software
component is to provide synchronization among all the
components connected to the Raspberry Pi. All the commands
that are issued to the Raspberry Pi are first passed to the
Mirror State. Mirror State interprets the commands issued to
the system. Figure 3 shows the detailed block diagram of the
proposed system.
Fig 3. Block Diagram of Smart Mirror
978-1-7281-3241-9/19/$31.00 © 2019 IEEE
Authorized licensed use limited to: Nitte Meenakshi Institute of Technology. Downloaded on April 08,2022 at 09:20:28 UTC from IEEE Xplore. Restrictions apply.
Fig 4.Block Diagram for Intrusion Detection and Human Monitoring
The Striking features of the system are Security against
Intrusion and Human Monitoring. The block
diagram for the same is shown in figure 4.The primary task of
Mirror State is to check whether the command can be
executed at that moment or not. If already one command is
under execution and the second command is being issued, then
the Mirror State decides whether the second issued command
can be executed or not.
As soon as an Intrusion is confirmed then an alert message
will be sent to the administrator of the mirror to his/her
registered mobile along with the picture of the Intruder. In
case of Human Monitoring, if the person under monitoring
moves out of the sight of the camera, the message will be
communicated to the administrator of the Smart Mirror
through an Alert Message. Later the administrator can view
the video streaming of the camera in the registered mobile and
take action accordingly.
There are 3 possible modes in which the system accepts inputs
namely voice commands, touch commands and can be
controlled through Mobile device. Motion detection is done
using the PIR sensors. The range of the PIR sensors may be
from 5 meters to 20 meters and it depends on the quality of the
sensor. When the movements are detected, raspberry Pi will
put on the camera attached to the Smart Mirror. Now the
camera will start detection of intrusion. The Intrusion is
actually detected using the YOLO technique. In case of human
monitoring also the Machine learning Techniques are used.
The Yolo with OpenCV technique for Human Detection is
used whenever the person under monitoring moves out of
sight of the camera. Such a human monitoring will be useful
to monitor children, elders, Patients, Prisoners admitted to
Hospital, Workers in Gold Shop and Prisoners in jail etc.
V. DESIGN AND IMPLEMENTATION
The System consists of 4 modules namely Login Module,
Input Module, Security Module and Human Monitoring
Module. The system is built using the hardware components
like Raspberry Pi, camera, microphone, Raspberry Pi
compatible touch screen, PIR sensors, mobile device and SD
card. The softwares required are Python, Raspbian OS,
OpenCV and VNC viewer.
118
 2019 1st International Conference on Advances in Information Technology
The Raspberry Pi is the main component of the system. All
other devices are connected to this component. The Raspberry
Pi 3 is having a CPU Quad-core with 64-bit ARM Cortex
having a speed of 1.2 GHz and has a GPU with 400MHz
speed. The camera taken here is 8 mega pixel and capable of
taking 3280 x 2464 pixel static images. The microphone
accepts voice commands which have working voltage of
microphone is 4.5V and has a wire length of 2 meters. A 10.1
inch touch screen is used as a mirror here which is a LCD
screen with 1280x800 resolutions. PIR is used for motion
detection and has a working range of 5 meters to 20 meters.
Android based mobile can be used for issuing commands to
the smart mirror. The mobile will be connected to the system
through the internet. The Raspberry is connected to the wi-fi.
32 GB SD card is used in Raspberry Pi for storage. The card
stores Operating System as well as the photo of intruders
along with the time stamp. The software component like
Python is needed for programming and for designing GUI Fig. 7 Input Module
(Graphical User Interface). Raspbian is a debian based
operating system for the Raspberry Pi. OpenCV library is used C. Security Module
for Yolo implementation for human detection. VNC viewer is The Security module is used for Intrusion detection using a
required to connect mobile device to the Smart mirror. VNC Camera fitted on the Smart mirror. The PIR sensors are used
server is installed in the Raspberry Pi and VNC client is for motion detection. When the motion is sensed, the Mirror
installed in android based mobile. This enables mobile device State activates Raspberry Pi which in turn triggers the camera.
to issue commands to the Smart mirror through internet Now Camera takes up the video. The video will be converted
connection. into frames. The Following figure 8 shows the details.
A. Login Module
The following figure 5 shows the Login Module. As soon as
the system boots up, the authentication process is taken up. It
is a username and Password driven authentication. A
successful authentication allows the user to access the Smart
mirror for command execution and navigation.

Fig. 8. Human Intrusion Detection using YOLO

D. Human Monitoring Module

Fig. 5 Login Module
B. Input Module
The following figure 6 shows the synchronizing unit called
Mirror State. The system accepts input through Input module.
The input can be voice, touch or mobile command.
The Human Monitoring can be done by using the Machine
Learning technique namely Yolo with the OpenCV. Python
coding is also being used. When the Human under observation
leaves the sight of Camera, an alert message will be sent to the
administrator of the Smart Mirror on his/ her registered mobile
number. The following figure 9 gives the details of the same.

Fig. 6 Mirror State

978-1-7281-3241-9/19/$31.00 © 2019 IEEE 119
Fig. 9. Human Monitoring using YOLO

Authorized licensed use limited to: Nitte Meenakshi Institute of Technology. Downloaded on April 08,2022 at 09:20:28 UTC from IEEE Xplore. Restrictions apply.
 2019 1st International Conference on Advances in Information Technology
VI. EXPERIMENTATION
A. Intrusion Detection
YOLO is a FCN (fully convolutional network) because
Convolutional Layers are used. Approximately 75
convolutional layers are used along with skip connections and
up sampling layers. No pooling is used, and to down sample
the feature of maps, the convolutional layer with a stride of 2
is used. Being a FCN, YOLO is not dependant on the size of
the input image. The network has to down sample the image
by a factor equal to the values called stride of the network. For
example, if the stride of the network is 32, and then an input
image of size 416 x 416 will yield an output of size 13 x 13.
B. Human Detection
Human detection is a part of human monitoring module. In
order to monitor any human, first the human need to be
detected. The Yolo technique with OpenCV which is based on
the Machine Learning technique is used for object detection.
The image is converted into SxS size grid. Image is converted
to bounding boxes and class probability map is drawn to
identify different object. Finally convolutions of size 1x1are
used to detect objects. The following figure 12 shows details.

The classifier/regressor accepts the features that are learned by

the convolutional layers. This process does the actual
detection prediction. The actual prediction in YOLO is done
by a convolutional layer which uses 1 x 1 convolutions.
The output produced itself is a feature map. Due to the use of
the 1 x 1 convolutions, the size of the prediction map will be
absolutely same as the size of the feature map. In prediction
map each cell will predict a fixed number of bounding boxes
in case of YOLO v3. Figure 10 shows the formulae to obtain
bounding box predictions. Fig. 12. Human Detection Process
The intrusion detection is a part of security and human
monitoring module.

Fig. 10. Bounding Box Equations

Bx and by are are the x,y center co-ordinates, bw and bh are

width and height of the prediction. The terms tx, ty, tw, th is
are the network outputs. The terms cx and cy are the top-left
co-ordinates of the grid. The terms pw and ph are anchors
dimensions for the box. The figure 11 shows the details of
actual detection and prediction process.
Fig. 13 Yolo Object Detection

The figure 14 shows how the human is identified in the input

image.

Fig.11. Actual Detection Prediction Process

978-1-7281-3241-9/19/$31.00 © 2019 IEEE 120

Fig. 14 Yolo Human Detection

Authorized licensed use limited to: Nitte Meenakshi Institute of Technology. Downloaded on April 08,2022 at 09:20:28 UTC from IEEE Xplore. Restrictions apply.
 2019 1st International Conference on Advances in Information Technology
[8]1Suryansh Chandel, 2Ashay Mandwarya, 3S.Ushasukhanya,
“Implementation of Magic Mirror Using Raspberry PI 3”, International
Hence, during human monitoring, the camera video is
Journal of Pure and Applied Mathematics Volume 118 No. 22 2018, 451-455
converted to frames. The frames are processed to detect -2018
human. If human isn’t present or missing in the immediate
frame being processed, then the human monitoring system [9] Sun Yong, Geng Liqing*, Dan ke, “Design of Smart Mirror based on
Raspberry Pi”, JETST- Vol.||05||Issue||04||Pages 6585- 6589||April||ISSN
will send an alert message indicating that human has moved
2348-9480-2018
out of camera range.
[10] Kun Jin, Xibo Deng, Zhi Huang, ShaoChang Chen, “Design of Smart
VII. CONCLUSION Mirror based on Raspberry Pi” ,2018 2nd IEEE Advanced Information
Management, Communicates, Electronic and Automation Control
Conference(IMCEC 2018)
The main theme of the proposed work is to design a product
bundled with maximum possible features. The system is not [11] R Akshaya, N.Niroshma Raj,S.Gowri ,“Smart Mirror- Digital Magazine
just devised as a means of information provider but also an for University Implemented Using Raspberry Pi”, International Conference on
Emerging Trends and Innovations in Engineering and Technological Research
interactive system which can actively be used for providing
(ICETIETR) -2018
security. The system can be used as security system. Human
motoring is of prime importance whenever we need to monitor [12] Lakshmi N M, Chandana M S, “IoT based Smart Mirror using Raspberry
a person. Smart mirror can be used for the same. Pi” ,International Journal of Engineering Research & Technology (IJERT),
Volume 6, Issue 13 -2018
The system can be proposed as a commercial product. There is
a scope for future work in this proposed system by adding [13] D.K. Mittal1*, V. Verma2, R. Rastogi3, “A Comparative Study and New
Artificial Intelligence. The same mirror can be extended to Model for Smart Mirror”International Journal of Scientific Research in
control the Home Appliances and lighting. Hence, we can be Research Paper, Computer Science and Engineering Vol.5, Issue.6, pp.58-61,
December -2017
able to control Home appliances and lighting, even when we
are getting ready for the day. In order to increase the level of [14] M.B.N. Siripala1, M. Nirosha2, P.A.D.A. Jayaweera4, N.D.A.S.
security, face detection can be used for authentication. The Dananjaya5, Ms. S.G.S. Fernando6, “Raspbian Magic Mirror-A Smart Mirror
system can also be tried with different intrusion and Human to Monitor Children by Using Raspberry Pi Technology”,International Journal
of Scientific and Research Publications, Volume 7, Issue 12, December 2017
monitoring techniques in order to increase the accuracy of the
335 ISSN 2250-3153-2017
system.
[15] Biljana Cvetkoska1, Ninoslav Marina1, Dijana Capeska Bogatinoska1,
REFERENCES Zhanko Mitreski1, “Smart Mirror E-health Assistant – Posture Analyze
Algorithm ”,IEEE EUROCON 2017, 6–8 JULY 2017, OHRID, R.
MACEDONIA -2017
[1]. Prof. P Y Kumbhar1, Allauddin Mulla2, Prasad Kanagi3, and Ritesh
Shah4, “Smart Mirror Using Raspberry PI”,International Journal For Research
[16] Prof B.P Kulkarni 1, Aniket V Joshi 2, Vaibhav V Jadhav3,
In Emerging Science And Technology ,Volume-5, Issue-4, Apr-2018
Akshaykumar T Dhamange4, “IoT Based Home Automation Using Raspberry
PI ”,International Journal of Innovative Studies in Sciences and Engineering
[2].Prof. Jagdish A.Pateljayshri T. Sadgir Sonal D. Sangaleharshada A.
Technology (IJISSET) ISSN 2455-4863 (Online) www.ijisset.org Volume: 3
Dokhale, “A Review Paper Design and Development of a Smart Mirror Using
Issue: 4 | April 2017
Raspberry Pi” International Journal of Engineering Science Invention (IJESI)
ISSN (Online): 2319 – 6734, ISSN (Print): 2319 – 6726 www.ijesi.org
[17] Vaibhav Khanna, Yash Vardhan, Dhruv Nair, Preeti Pannu,“Design And
||Volume 7 Issue 4 Ver. I || PP 40-43||April 2018
Development Of A Smart Mirror Using Raspberry PI” ,International Journal
Of Electrical, Electronics And Data Communication, ISSN: 2320-2084
[3] S C V S L S Ravi Kiran1, Naresh Babu Kakarla2, Banoth Praveen Naik1,
Volume-5, Issue-1, Jan.-2017
“Implementation of Home automation system using Smart Mirror”,
International Journal of Innovative Research in Computer and
[18] Jane Jose1, Raghav Chakravarthy2, Jait Jacob3, Mir Masood Ali4, Sonia
Communication Engineering Vol. 6, Issue 3, March 2018
Maria D’souza5, “Home Automated Smart Mirror as an Internet of Things
(IoT) Implementation - Survey Paper ”, International Journal of Advanced
[4] Dr.J. Ajayan1, P. Santhosh Kumar2, S. Saravanan3, S. Sivadharini4, R.
Research in Computer and Communication Engineering ISO 3297:2007
Sophia5, “Development of Smart Mirror using Raspberry-Pi 3 for Interactive
Certified Vol. 6, Issue 2, February 2017
Multimedia”, 12th International conference on Recent Innovations in Science
and Management ICRISEM’-2018
[19] Oihane Gomez-Carmona, Diego Casado-Mansilla ,“SmiWork: An
Interactive Smart Mirror Platform for Workplace Health Promotion”
[5] Divyashree K J1, Dr. P.A. Vijaya2, Nitin Awasthi 3, “Design And
,International Journal Of Electrical, Electronics And Data Communication,
Implementation Of Smart Mirror As A Personal Assistant Using Raspberry
ISSN: 2320-2084 Volume-5, Issue-1, Jan.-2017
PI ”, International Journal of Innovative Research in Computer and
Communication Engineering Vol. 6, Issue 3, March 2018
[20] Muhammad Mu’izzudeen Yusri1, Shahreen Kasim1, Rohayanti Hassan2,
Zubaile Abdullah1,Husni Ruslai3, Kamaruzzaman Jahidin4, Mohammad
[6] Prof. Manish Assudani, A. S. Kazi, P.O.Sherke, S. V. Dwivedi, Z. S.
Syafwan Arshad4, “Smart Mirror for Smart Life”,International Journal of
Shaikh ,“Hermione 1.0”- A voice Based Home Assistant System ”, National
Advanced Research in Computer and Communication Engineering ISO -2017
Conference on Advances in Engineering and Applied Science (NCAEAS)
29th January 2018

[7] Kamineni B.T. Sundari1, A. Prakash2, K. Suparna3, R. Krishna Nayak4,

“Using Raspberry Pi to Design Smart Mirror Applications”, IJETST-
Vol.||05||Issue||04||Pages 6585-6589||April||ISSN 2348-9480-2018

978-1-7281-3241-9/19/$31.00 © 2019 IEEE 121

Authorized licensed use limited to: Nitte Meenakshi Institute of Technology. Downloaded on April 08,2022 at 09:20:28 UTC from IEEE Xplore. Restrictions apply.