Augmented reality based iot switch using
unity and blynk
Name1, name2, name3, name4
SRM Institute of Science and Technology, Kattankulathur, Chennai, India.
Mailid1, Mailid2, Mailid3, Mailid4
Abstract
Augmented reality increases the interactive control
of the devices and appliances with the users
operating them. Now a days it is possible to
communicate and control the appliances remotely
with the emergence of smart devices. Many methods
are used to control the appliances remotely but it
becomes difficult for the users to use the smart
devices because of the Graphical User Interface
control. For this solution, we use augmented reality
(AR) technology for easier operation of home
appliances connected by home networks. AR
technology can provide virtual graphics and apply
additional information to specic areas through
camera display.
Keywords: Augmented reality, appliances, virtual
graphics.
Introduction
The aim of the project is to make a virtual button
using a target image.By using the blynk and
Unity.Using blynk we will give the data to ESP32
and using unity we will make the apk application by
giving the blynk data and target image.Downloading
the apk file in any android mobile we can see the
virtual buttons by the help of target image.As the
technology is growing up day by day now we have
iot based technology with which we can control
applicances by the help of mobile applications.In
future it way go to the augmented reality based
technology.Making use of the technology of
augmented reality and internet of things we are
doing a virtual button.
Augmented reality (AR) and the Internet of Things
(IoT) have received significant attention as key
enabling technologies for making spaces smarter
and more interactive.AR is a type of interactive
medium that provides a view of the real world
augmented by spatially registering useful computer-
generated information. It helps people to understand
the world and amplifies their intelligence for solving
problems and carrying out actual tasks.
IoT refers to a network of physical devices and
everyday objects embedded with minimal
computing elements for sensing, collecting,
communicating, and even interacting with the
objects themselves. Such an infrastructure will
provide the basis for smart environments through a
collective big data analysis and context- based
services (e.g., real-time analytics and automation).
Augmented reality (AR) is an approximation to the
reality in which physical objects are linked to a
virtual counterpart, with contextual computer-
generated information. AR has come a long way
from a science-fiction dream to a clearly established
scientific discipline. Until recently, the cost of
augmented reality made it somewhat prohibitive.
Fortunately, the situation has changed, and AR can
be implemented at a competitive cost in devices
ranging from smartphones, tablets, smart-glasses, or
even more complex (headsets) to become more
realistic. Nowadays, mobile augmented reality
(MAR) is in the technology hype cycle, which has
attracted the interest of both research and industry.
Thus, with the development of smart mobile
technology, many applications, services, and
content have proliferated. With the support of this
type of computer tool, focused on expanding the
perspective of the real world using mobile devices,
virtual images are presented, which transcend the
common perception and reveal a new universe to
discover.
Methodology
Make the connections as per the circuit
diagram(fig.1)using hardware components.
Using Blynk App we can on and off the bulb.
We need to make things controlling using blynk’s
api.
Using unity we can create a apk application by
giving the target image and api links while creating.
By install the apk application in the mobile.We can
experience AR by the camera by opening the apk
application installed and we are able to see the
virtual buttons on the screen by targeting the target
image.Through that we can control the bulb.
Fig.1
Experimental results

Uploading code in ESP 32

Controlling using API

API for ON : http://blynk-
cloud.com/MtS9SvQG6dccaRt8sIpAZ0PnjkvqNki
6/update/V1?value=0

Making the circuit.

API for OFF : http://blynk-

cloud.com/MtS9SvQG6dccaRt8sIpAZ0PnjkvqNki
6/update/V1?value=1

Controling the bulb by Blynk app

Targeting the image using camera to see the virtual

buttons.
Testing virtual buttons by switching ON and OFF
Switching on and off by using the apk application
Conclusion
Augmented reality is a technology that is
increasingly present in many areas of our life. It
helps us to create a more natural interface between
humans and the physical world, minimizing the
number of hardware devices we must carry with us.
First, computers were used as means to visualize
data generated by sensors to check the status of the
physical world; nowadays, smartphones are
everywhere and, in the very near future,
instrumented glasses will be used to monitor the
world around us, even with technology directly
integrated with our eyes. In this work, a monitoring
energy system based on augmented reality to
visualize in real-time the PQ parameters and the
energy consumption of home appliances in an easy
way is presented; so easy, that end users only have
to directly focus with their smartphones on the home
appliance they are interested in to know its energy
behavior. From an architectural point view, the
system presents an IoT energy device developed by
authors to obtain the energy information of the home
appliances, then this information is uploaded to the
cloud (ThingSpeak platform), and eventually the
integration of this information with the augmented
reality application when the frame captured by
smartphone’s camera corresponds with an image
target previously stored. The carried-out tests show
that the system easily detects the image targets and,
most importantly, the response time to detect and
overlap the energy information on the captured
frame is negligible.
The results show the use of Augmented reality to
control the bulb. With the help of smart phone
devices users will be able to switch on/off the bulb
through internet or home network . This system will
mainly help the disabled and elderly people to
control the appliances giving them a real world
experience with the help of Augmented reality.
References
1. H. Bauer, M. Patel, and J. Veira, “The internet of
things: Sizing up the opportu-
nity,” McKinsey, 2014.
2. I. Lee and K. Lee, “The internet of things (iot):
Applications, investments, and
challenges for enterprises,” Business Horizons, vol.
58, no. 4, pp. 431 – 440, 2015.
3. A. Al-Fuqaha, M. Guizani, M. Mohammadi, M.
Aledhari, and M. Ayyash, “Internet
of things: A survey on enabling technologies,
protocols, and applications,” IEEE
4. J. Ploennigs, A. Ba, and M. Barry, “Materializing
the promises of cognitive iot:
How cognitive buildings are shaping the way,” IEEE
Internet of Things Journal, vol. 5, no. 4, pp. 2367–
2374, Aug 2018.
5. A. F. Abate, F. Narducci, and S. Ricciardi,
“Mixed reality environment for mission
critical systems servicing and repair,” in Virtual,
Augmented and Mixed Reality.
Heidelberg, 2013, pp. 201–210.
6. G. White, V. Nallur, and S. Clarke, “Quality of
service approaches in iot: A systematic mapping,”
Journal of Systems and Software, vol. 132, pp. 186
–
203, 2017. [Online]. Available:
http://www.sciencedirect.com/science/article/pii/
S016412121730105X
7. G. White, A. Palade, and S. Clarke, “Forecasting
qos attributes using lstm net-
works,” in 2018 International Joint Conference on
Neural Networks (IJCNN), 2018.
8. “Apple inc. arkit.”
https://developer.apple.com/arkit/.
9. “Vuforia engine,” https://developer.vuforia.com/.
10. J. A. Garc ́ıa-Mac ́ıas, J. Alvarez-Lozano, P.
Estrada-Martinez, and E. Avil ́es-L ́opez,
“Browsing the internet of things with sentient
visors,” Computer, no. 5, pp. 46– 52,
17
2011.
11. D. Jo and G. J. Kim, “Ariot: scalable augmented
reality framework for interacting
with internet of things appliances everywhere,”
IEEE Transactions on Consumer
12. S. Kasahara, V. Heun, A. S. Lee, and H. Ishii,
“Second surface: multi-user spa-
tial collaboration system based on augmented
reality,” in SIGGRAPH Asia 2012
13. G. White and S. Clarke, “Smart cities with deep
edges,” in ECML International
14. J. Redmon, S. K. Divvala, R. B. Girshick, and A.
Farhadi, “You only look once:
Unified, real-time object detection,” CoRR, vol.
abs/1506.02640, 2015. [Online].
Available: http://arxiv.org/abs/1506.02640
15. G. White, A. Palade, and S. Clarke, “Qos
prediction for reliable service com-
position in iot,” in Service-Oriented Computing:
15th International Conference, 16. G. White, A.
Palade, C. Cabrera, and S. Clarke, “IoTPredict:
collaborative QoS
prediction in IoT,” in 2018 IEEE International
Conference on Pervasive Comput-
17. S. Henderson and S. Feiner, “Exploring the
benefits of augmented reality docu-
mentation for maintenance and repair,” IEEE
Transactions on Visualization and