International Conference on Communication and Information Processing

(ICCIP-2020)
Available on: SSRN

Smart Refrigerator
Utkarsh Kore1, Parth Akre2, Usama A. Mashayak3
123
Dr. Babasaheb Ambedkar Technological University, Lonere 402103, India

Abstract

Some portion of what makes getting in shape so troublesome is that checking calories is a vague science at best. Eating food without
knowledge of its composition and nutritional contents prompts poor processing leading to poor health. Each time you convey
nourishment from the device it indeed, even with nutritious data, one needs to screen serving sizes, people are continually searching
for approaches to enhance their health and wellbeing. The typical procedure of getting nutritional data is by utilizing google or
utilizing some applications. In this paper, we have proposed a system which is portable, handy, needs less space and power, fast,
is more proficient and simpler. The Smart Refrigerator system is able to remotely alert the user about the scarce contents, expiry
date and nutritional values. The Internet of Things (IoT) refers to the coordination of devices and systems that interconnect real
world sensors and actuators to the Internet. This paper presents the abstraction, architecture, interfacing and functionality of such
a device that incorporates the systems stated above: a system that interacts with objects of concern, gathers information about them,
process this information into relevant data with the help of a dataset that is later conveyed to the user via an IOT platform.

Keywords- Internet of Things (IOT), Raspberry Pi 3b, Load cell sensor, Pi Camera, Machine Learning.

© 2020 – Authors.

1. Introduction

The exemplar of the Internet of Things (IOT) requires extensive connectivity to billions of heterogeneous and
homogeneous devices. In recent times, rapid growth of IOT devices in the smart home sector envisioned a wide range
of state-of-the-art services and applications. Kitchen is one of the most important places for a Smart home setup as it
consists of many contrivances which provide better services to the household. The focus of our project is on the fridge.
Specifically, tweaking it in several ways to make it a smart fridge. Many efforts in the construction of the smart
refrigerator have been made, none of which has been energy efficient or cost effective. The 21st century fast-paced
environment doesn't allow the user to keep track of the food items inside the refrigerator. Although efforts have been
put by the industry to reinforce the smart refrigerator, the contemporary or the present technology is still not efficient
and effective enough to be made available to all households.
This automation is too complicated or difficult to comprehend for a simple household user who has limited information
of how all the functions behind the smart refrigerator works. The internet affinity at most of the places is still
substandard and there is limited network connectivity i.e. either slow internet speeds or low support of data transfer.
The smart home environment or the networked home doesn’t have enough security or anti-theft systems to protect the
outflow of data from the house. The privacy of the user and the house can be compromised by cyber criminals. There
is no special operating system for remote devices used to control the smart system. There is no standard for the area,
resulting in products obeying to different specifications by

Electronic copy available at: https://ssrn.com/abstract=3647969

2 Utkarsh Kore et.al./ICCIP-2020

different producers. The smart refrigerator or the internet refrigerator, is used to keep track of the items inside it and
notify about inadequate products, saving energy when possible. The proposal of connecting home appliances to the
internet or the smart home surroundings has been the future of IOT. It is highly regarded to be the futuristic standard
for automation and considered the next big thing by many.

2. Literature Survey

Nikhil Kakade, Prof. (Dr.) S. D.Lokhande, 2016, has proposed a system, ”IoT based Intelligent home using Smart
Devices” that senses the daily needs of particular items in the refrigerator and automatically places an order to the
grocery shop. Deepti Singh, Preet Jain,in 2016 has proposed the system, “Smart Refrigerator” which can sense the
weight of the food items placed in the refrigerator and sends a notification to the user’s mobile through an application
when the weight falls below the threshold value. Emily Moin, 2016, has proposed the system “Smart Refrigerator for
Grocery Management” which identifies the weight and expiry dates of the food items in the refrigerator and sends
customized notification to the user. The proposed system finds the weight and expiry dates of the food item and
notifies the users through application which helps to place the order when the contents fall below the threshold value.
Folasade Osisanwo et al, 2015 have proposed a system, “Internet Refrigerator-a typical IoT “. This system makes use
of RFID. The refrigerator is Wi-Fi enabled to transmit data. Shama Mubeena, N. Swati, 2017 has proposed a system,
“The Design and Implementation of a Wi-Fi Based User- Machine -Interacted Refrigerator”, which uses Wi-Fi for
wireless communication. It demonstrates monitoring of the fridge and notify users through mail id.
B. Ramesh, J. Lingaiah has proposed the system, 2016, “Raspberry Pi Based Interactive Home Automation System
through E-Mail”. This system uses Email rather than Bluetooth to send notification to the user as it is not
efficient. It also uses LEDs to indicate switching action. Jessica Tran et al, 2015 has proposed a system,” Automated
Demand Response Refrigerator Project” in which they gave a dashboard to view the sensor data and control the
refrigerator remotely. Soundhar Ganesh S et al, 2015, in their paper,” Raspberry Pi Based Interactive Home
Automation System through Internet of Things used Raspberry pi to interface the internet with embedded systems.
Carson Kai-Sang Leung et al, 2016, in “A Machine Learning Approach for Stock Price Prediction”, has proposed a
deep learning approach for structural support vector machines (SSVMs) to perform classification on complex inputs.
Prapulla S B et al, 2015, in “Smart Refrigerator Using Internet of Things” has used the sensors for sensing and
monitoring the contents. This smart fridge notifies the remote user using SMS or email services.
A. Alheraish, 2004, proposed a system “Design and implementation of home automation system”. Which represents
the use of a remote-control system using GSM. It tests PC-environment. Sarthak Jain et al, 2014 in “RaspberryPi
based Interactive Home Automation System through Email”. This paper designs the system in which communication
is through email. It uses the efficient algorithm for this in a python environment. LED switches are used here. Hasin-
Han Wu, 2017, has proposed a system, “Low-Cost Smart Refrigerator”, it describes the overall design of a low-cost
Smart Refrigerator built with Raspberry Pi. Next, it has two sensors controlling each camera, which are hooked up to
the Raspberry Pi board. Poonam B. Patil et al, 2016, has proposed a system “Home Automation System using Android
and Arduino Board” which implements the methodology of using mobile applications to control the home appliances
remotely. iTunes GSM modem to transmit the data to mobile. Surinder Kaur et al, 2016, proposed the system “HOME
AUTOMATION AND SECURITY SYSTEM”, this paper focused on reducing human effort by automating electrical
devices. It sends messages to users. It is also focused to save electricity.
Nishchol Mishra, Dr. Sanjay Silakarihas, 2012 proposed a model in “Predictive Analytics: A Survey, Trends,
Applications, Opportunities & Challenges” they have mentioned the classification and regression as the two main
objectives of predictive analytics. Where the classification, classifies the things according to the predefined class and
uses numeric variables for data attributes. And Regression is used to study the relationship between the variables.
M.P. Sathish et al, 2015 in “Home Automation through Email using Raspberry Pi” has used the email to instruct the
Raspberry pi, the operations to be performed. Where the pi reads the content and performs the operations accordingly
and acknowledges accordingly. Rajendra Banjade, Suraj Maharjan, 2011” Product recommendations using linear
predictive modeling” has proposed linear regression to improve the functioning of the system. Ravi Kishore Kodali,
Vishal Jain, 2016, has proposed "IoT based smart security and home automation system", conceptualizes the idea of
connecting to real world objects using the internet [19]. Sharon Panth, Mahesh Jivani has proposed a system called
“Home Automation System (HAS) using Android for Mobile Phone”. This paper helps to reduce human efforts by
using mobile phones. It uses Android platforms. The HAS is interfaced with a microcontroller. This device is accessed
through mobile application.

Electronic copy available at: https://ssrn.com/abstract=3647969

Smart Refrigerator 3

3. Proposed System Architecture

The proposed system can track food items in our refrigerator which helps us to collect data about them. This in turn
will help to track our food habits and control how much and when we eat. We can plan our balanced diets using this
data. Expiration date notifications will help reduce food wastage. We have used components like sensors and modules
which make it possible to monitor food items, the architecture is shown in the Fig. 1.

Fig. 1. Proposed System Architecture

3.1. Components

We have used sensors like a Load cell sensor which helps us to track the weight of the food item. A Load cell sensor
is a transducer and needs an analog to digital converter, here we have used HX711 (Load cell amplifier) to connect
the sensor to Raspberry Pi. To know what food item, it is we have used a Pi camera which helps us to take a photo
and save it. A push button is used to signal the camera to take a photo. These sensor inputs are then given to a
microcontroller, here we have used a Raspberry Pi 3b because it is an all-in-one module which has inbuilt Wi-Fi
module which helps us make the system wireless, it also has a user installable OS or GUI (Graphical User Interface).
Fig. 1 shows the basic architecture of our proposed system.
The data from the sensors is further processed by feeding it to a machine learning model which predicts the
food item form the photo and compares it with an offline database (.csv file) and retrieves the relevant data and sends
SMS messages to the users smartphone using an online SMS messaging API (Application Program Interface) known
as Twilio.

3.2. Whereabouts

The load cell sensor is fixed to a tray which can hold the food item inside the fridge. The Pi camera is fixed in such a way
that the frame for every photo is fixed and can cover the whole tray in an image. The Raspberry is powered with the help
of a portable battery.

4. Implementation

This section provides a deep dive inside the technologies used in the implementation of the smart refrigeration system.
The system consists of a few sub-modules which are sensing module, control module and transmission module.
Sensing module consists of load cell sensor and camera module while control module consists of microcontroller and
power supply unit at the sensors can sense the weight of a food item which is placed in the refrigerator and displays
the information and last but not least, transmission module consists of Wi-Fi module. These modules work together
to determine contents status inside the refrigerator and notify the user about the products via an SMS. Fig. 2 shows
the flowchart of the smart refrigerator.
The basic idea behind the Smart Refrigerator system is the like what is the food type, expiration date and nutrients
over a SMS to the user's smartphone. The steps followed for designing the system are:

Electronic copy available at: https://ssrn.com/abstract=3647969

4 Utkarsh Kore et.al./ICCIP-2020

● Check the initial weight value of the food item which is kept inside the refrigerator.
● Recognizing the food item with a camera and machine learning model.
● Compare the status of food items (present value with threshold value)
● If the load value is below the threshold value, then send the respective SMS notification to the user on the
mobile phone. The flowchart shown is Fig. 2 shows the basic logic flow of the architecture.
●

Fig. 2. Flow chart

4.1. Hardware essentials

System consists of Load Cell, HX711 & Raspberry Pi 3. The Load Cell measures the weight of the food items which
are placed inside the refrigerator in analog form. HX711 converts the value form analog into digital form which in
turn passes to the Raspberry Pi. When the value falls below the threshold value, the notification will be sent to the
user and will be notified through SMS.
The Pi Camera can be attached directly to the Raspberry Pi and is positioned such that the frame and the item have
good lighting conditions (artificial lighting can be used). The frame should get the photo of the whole item to gather
the most information. The camera sends the image to the Raspberry Pi and the machine learning model predicts the
item type. A push button is used to signal the Raspberry Pi to take a still using the camera.
The Load Cell Sensor is an analog device and sends analog signals, therefore an analog to digital converter HX711 is
used to convert and amplify the signal to further give it to the Raspberry Pi. The connections are shown in Fig. 3. The
load cell is mounted below the vegetable tray in the refrigerator that continuously measures the weight of vegetables
in the tray. Since the weight of the vegetable tray goes below threshold weight (set by the user approximately 1kg), it
senses the less presence of vegetables. Low signal will be generated corresponding to it which will be sent to the user
through SMS.

Electronic copy available at: https://ssrn.com/abstract=3647969

Smart Refrigerator 5

Fig. 3. Load Cell-Raspberry Pi interface

4.2. Software essentials

● Output from Load Cell sensor - The output from the Load Cell Sensor is taken through Raspberry Pi and
stored in a text file. A python program fetches the new values and compares it with the threshold value.
● Output form Pi camera - The camera captures a still when the user pushes the button. The Raspberry Pi saves
the still as ITEM.jpg. Fig. 4 shows the image captured.
● Machine learning model - The image is then given to the machine learning model. We have used a pre-
trained ImageNet model for classification of photos using an ImageNet image classifier which was trained
on thousands of images of food items and can predict the food item.
● Database - We have used a simple offline database stored in a .csv file. Which is updated weekly to widen
the range and update all the data. The .csv file which holds the important data like key, item name, expiration
date and nutrients. The output from the machine learning model is then compared with the database and the
corresponding information is fetched by the program.
● Twilio API - It is a free online SMS API which helps in sending customizable SMS messages to users'
smartphones. The program then uses the fetched data to send it to the user.

Table 1. Weight comparison

Weight detected Tray weight Food item already Total weight Weight Threshold Message sent
(kg) (kg) present (kg) (kg) (kg) to user

0.779 0.532 1.332 (1.332+0.779)- 2.0 Low on food item

0.532=1.579

0.729 0.532 1.856 (1.856+0.729)- 2.0 Food item added,

0.532=2.062 expiration date and
nutrition

Depending upon the weight it sends respective messages like if for e.g. If the food item recognized is a tomato and
the weight is less than the threshold weight, then the Raspberry will send the message to the user’s smartphone “You
are low on tomatoes.”. And when you stack the food item again the weight condition and food item recognition get
activated and it sends the message stating that the tomatoes are added and its expiration date, nutrients. Table 1 shows
how what message to send is calculated.

Electronic copy available at: https://ssrn.com/abstract=3647969

6 Utkarsh Kore et.al./ICCIP-2020

5. Results

The goal of the proposed architecture is to enhance the sustainability of IoT applications by exploiting smart and
reliable (networks of) things and by being able to utilize a big number of heterogeneous device platforms. The
proposed architecture enables the development of an environment for IoT applications through cross-platform
channels that incorporate technologies for Data, Information, Things and Decentralized Management. As briefly
described previously at a conceptual level, the proposed system's workflow is as follows. The Load cell sensor and
camera module work in tandem to help find out what is the status of quantity of an item. And use the database to
retrieve the data according to the output of the previous operation and send corresponding data to the user’s
smartphone using Twilio SMS API.
We have achieved the desirable result from the image detection from the classifier. The Fig. 4 shows the result of the
image detected and executes the command to send a message.

Fig. 4. Input image via Pi cam

The data output from the machine learning model is then matched with the database which is shown in the Fig. 5 and
then the relevant data is extracted by the sendText.py file.

Fig. 5. Database

Electronic copy available at: https://ssrn.com/abstract=3647969

Smart Refrigerator 7

Then the python program inputs the relative data onto the message which is eventually sent to the user's smartphone
using Twilio API. The message received for a sample experiment is shown in Fig 6.

Fig. 6. Output text message via Twilio

6. Conclusion

With the help of sensors like the Load cell sensor and Pi camera we were able to gather data from a food item such
as its weight and what it is. We were able to process this data with the Raspberry Pi and with the help of a pre-trained
Machine Learning model which compares the data with the preloaded database, further it can send SMS notifications
about item type, expiration and nutritional information of the food item. This unique method will lead to relatively
accurate results to track nutrition and keep a track of food items. Our results will show that the accuracy of our system
is acceptable, and it will greatly improve and facilitate current manual nutrition measurement techniques. Further
improvements to the project would be adding the option to place orders for food items from the reply itself, suggesting
recipes with the contents present in the fridge, add more sensors to gather more data about different parameters and
to use them to our benefit.
The existing systems use different kinds of sensors which collect information like temperature, humidity, color of
food item to recognize food items etc. but they miss out on recognizing the food item exactly and using this
information to our benefit. The system which we have proposed collects two types of data about a food item i.e. food
item name and weight. This system uses pre-existing resources to work with which helps in making the process fast
and efficient. Existing systems use Android application integration to get notifications from the fridge but in our
system, we have used an SMS API. This is an advantage as we can receive notification in places where there is a
network, but no internet and it doesn't need application integration as we have messaging applications pre-installed

Electronic copy available at: https://ssrn.com/abstract=3647969

8 Utkarsh Kore et.al./ICCIP-2020

in our smartphones. The hardware is cheap, portable and can be accessed remotely. As the system is connected to
Wi-Fi and connected to the internet, we can update databases and improve the machine learning model time to time.

References

[1] Nikhil Kakade, Prof. (Dr.) S.D. Lokhande,” IoT based Intelligent home using Smart Devices”, International Journal of
Innovative Research in Computer and Communication Engineering, vol. 4, Issue 6, June 2016.
[2] Deepti Singh, Preet Jain, “IoT Based Smart Refrigerator System”, International Journal of Advanced Research in
Electronics and Communication Engineering (IJARECE), Volume 5, Issue 7, July 2016.
[3] Emily Moin, “Smart Refrigerator for Grocery Management”, Technical Disclosure Commons, Defensive Publication
Series, May 05, 2015.
[4] Folasade Osisanwo, Shade Kuyoro, and Oludele Awodele,” Internet Refrigerator”,3rd International Conference on
Advances in Engineering Sciences & Applied Mathematics (ICAESM 2015) March 23-24, 2015.
[5] Shama Mubeena, N. Swati,” The Design and Implementation of a Wi-Fi Based User- Machine -Interacted Refrigerator”,
ISSN 2319-8885, Vol.06, Issue.14, April-2017.
[6] B. Ramesh, J. Lingaiah,” Raspberry Pi Based Interactive Home Automation System through E-Mail”, International
journal of Innovative Technologies ISSN 2321-8665, Vol.04, Issue.15, October-2016.
[7] Jessica Tran, Jordan Gilles, Ryan Mann, and Vishnu Murthy, “Automated Demand Response Refrigerator Project”, CE
186, OCTOBER 2015.
[8] Soundhar Ganesh S, Venkatesh S, Vidhyasagar P, Maragatha Raj S, “Raspberry Pi Based Interactive Home Automation
System through Internet of Things”, International Journal for Research in Applied Science & Engineering
Technology (IJRASET), Volume 3 Issue III, March 2015.
[9] Carson Kai-Sang Leung, Richard Kyle MacKinnon and Yang Wang, “A Machine Learning Approach for Stock Price
Prediction”, University of Manitoba, Winnipeg, MB, Canada. Vol 2, Issue II, July 2016
[10] Prapulla S B, Dr. Shobha G and Dr. Thanuja T C, “Smart Refrigerator Using Internet Of Things” Journal of
Multidisciplinary Engineering Science and Technology (JMEST) ISSN: 3159-0040 Vol.2 Issue 7, July – 2015.
[11] A. Alheraish, “Design and implementation of home automation system”, IEEE Trans. Consumer Electronics, vol. 50, no.
4. Pp.1087–1092, 2004.
[12] Sarthak Jain, Anant Vaibhav, Lovely Goyal Student member, IEEE, “Raspberry Pi based Interactive Home Automation
System through E-mail”, International Conference on Reliability, Optimization, and Information Technology -
ICROIT 2014.
[13] Iyer, B., Pathak, N.P. & Ghosh, D. RF sensor for smart home application. Int J Syst Assur Eng Manag 9, 52–57 (2018).
https://doi.org/10.1007/s13198-016-0468-5
[14] B. Iyer, N. P. Pathak and D. Ghosh, "Dual-Input Dual-Output RF Sensor for Indoor Human Occupancy and Position
Monitoring," in IEEE Sensors Journal, vol. 15, no. 7, pp. 3959-3966, July 2015, doi: 10.1109/JSEN.2015.2404437.
[15] Hsin-Han Wu, Yung-Ting Chuang, “Low-Cost Refrigerator”, Edge Computing (EDGE), 2017 IEEE International
Conference on.
[16] Poonam B. Patil, Roopali R. Patil, Swati V. Patil,” Home Automation System using Android and Arduino Board”,
International Journal of Innovative Research in Science and Engineering Technology, Vol.5, Issue 4, April 2016.
[17] Surinder Kaur, Rashmi Singh, NehaKhairwal and Pratyk Jain,” HOME AUTOMATION AND SECURITY SYSTEM”,
Advanced Computational Intelligence: An International Journal (ACII), Vol.3, No.3, July 2016.
[18] Nishchol Mishra, Dr. Sanjay Silakari “Predictive Analytics: A Survey, Trends, Applications, Opportunities &
Challenges”, (IJCSIT) International Journal of Computer Science and Information Technologies, Vol. 3 (3), 2012, 4434-
4438.
[19] M.P. Sathish, Dr. S.A.K. Jilani, Mr. D.Girish Kumar, “Home Automation through E-mail using Raspberry Pi”,
International Journal of Advanced Research in Electronics and Communication Engineering (IJARECE) Volume 4, Issue
9, September 2015.
[20] Rajendra Banjade, Suraj Maharjan,” Product recommendations using linear predictive modeling”, Internet (AH-
ICI),2011 Second Asian Himalayas International Conference, 4-6 Nov. 2011.
[21] Ravi Kishore Kodali, Vishal Jain, "IoT Based smart security and home automation system", Computing, Communication
and Automation (ICCCA), 2016 International Conference on.
[22] Sharon Panth, Mahesh Jivani “Home Automation System (HAS) using Android for Mobile Phone”, International Journal
of Electronics and Computer Science.

Electronic copy available at: https://ssrn.com/abstract=3647969

Smart Refrigerator 9

Utkarsh H. Kore was born in Tarapur, Maharashtra

in the year 1998. He is a student at Dr. Babasaheb
Ambedkar Technological University, Raigad,
Maharashtra. He is currently pursuing the bachelor’s
degree in technology in the Electronics &
Telecommunication discipline.

Parth M. Akre was born in Mumbai, Maharashtra in the

year 1999. He is a student at Dr. Babasaheb Ambedkar
Technological University, Lonere. He is currently
pursuing the bachelor’s degree in technology in the
Electronics & Telecommunication discipline.

Usama A Mashayak accomplished his B.E. in Electronics

and Telecommunication from MIT College of Engineering,
Pune, M.E. in Digital Communication from College of
Engineering, Ambajogai. His research interests include
Microstrip Antenna and Microwave Filters.

Electronic copy available at: https://ssrn.com/abstract=3647969