Proceedings of the Third International Conference on Inventive Research in Computing Applications (ICIRCA-2021)

IEEE Xplore Part Number: CFP21N67-ART; ISBN: 978-0-7381-4627-0

Markerless Augmented Reality based application

for E-Commerce to Visualise 3D Content
Yashvi Desai Naisha Shah Vrushali Shah
2021 Third International Conference on Inventive Research in Computing Applications (ICIRCA) | 978-1-6654-3877-3/21/$31.00 ©2021 IEEE | DOI: 10.1109/ICIRCA51532.2021.9545009

Department Of Information Technology Department Of Information Technology Department Of Information Technology

Sardar Patel Institute Of Technology Sardar Patel Institute Of Technology Sardar Patel Institute Of Technology
Mumbai,Maharashtra Mumbai, Maharashtra Mumbai, Maharashtra
yashvi.desai@spit.ac.in naisha.shah@spit.ac.in vrushali.sha h@ spit.a c.in

Dr. Prasenjit Bhavathankar Prof. Kaisar Katchi

Department of Information Technology Department of Applied Sciences and Humanities
Sardar Patel Institute of Technology Sardar Patel Institute of Technology
Mumbai, Maharashtra Mumbai, Maharashtra
p bhavathankar@spit.ac.in kaisar katchi@spit.ac.in

Abstract—Augmented reality has three principal features: different needs. Someone who wants to redecorate their house
combining the real world environment with the virtual world, would appreciate observing what the furniture looks like along
real-time interaction for users, and accurate representation of with the interior of their house before making a purchase.
3D objects. Augmented Reality in E-commerce allows customers
to view products or experience services in their physical space Because of these problems, a lot of times customers end up
before purchasing the required items. Current online shopping exchanging the items, or worse, returning them. This process
services only allow customers to see 2D images of the products of online shopping does not provide a smooth and personalized
they are buying. This type of experience is not personalized and user experience and therefore reduces user satisfaction.
sometimes leads to bad shopping choices choices; the customers
find it difficult to shop only with a static image view available. The architecture diagram in Fig[1] shows how the
Customers cannot accurately predict whether the product they camera captures the real world into a graphic system and
purchase will fit their home environment. This results in a lot of merges the video of real world into augmented video stream
people returning or exchanging the things their purchases. AR
thus giving the augmented reality version of the real world.
resolves these issues. Thus, a method has been proposed for
adding a virtual object in the real world by just using a real-time The proposed solution is to develop a marker less augmented
camera. The main aim of this paper is to provide user reality-based E-Commerce application. Markerless augmented
visualization of high resolution E-commerce products in a real reality is a technique where prior information about a user’s
environment. environment is not necessary. According to a survey, 63% of
Index Terms—Augmented Reality, E-commerce, virtual
the customers are confident that their shopping experience
will be improved if augmented reality is used. 3D models of
I. INT RODUCT ION every item will be generated to facilitate the process of
visualization as 35% of people say that if they get to virtually
Online shopping has recently gained a lot of popularity
try on stuff before purchasing it, they would like to shop
because of its feasibility and wide range of available options.
online more often. Further, rendered 3D models will give user
People find this way of purchasing things more practical be-
a feeling of in person shopping and will increase the user
because it does not require the user’s physical presence. There
satisfaction. The application presented in this study has been
is no doubt that E-commerce is being accepted by many
developed using Android studio and the models have been
people, but we cannot ignore the fact that online shopping
created using Unity 3D software. The application places 3D
lacks one of the most important aspects of shopping i.e. the in
objects on staged surfaces if required, lighting conditions are
person look and feel of the product. E-commerce can be
decided based on the actual lighting, and then, measures the
considered a rapidly growing domain as more and more
distance between 2 points and saves all the information
online shopping applications are getting launched in the
about detected planes.
market day by day. Usually, E-Commerce applications only
provide the images of the products, i.e. a static view of the
products which makes it difficult for a user to comprehend
how the product will look like in his/her physical space.
Information like height, breadth and material used can’t be
known precisely. In case of apparel shopping, it is difficult
for customers to imagine whether the piece of cloth will
fit/suit them as only images of clothes are available and
people with different body types have
978-0-7381-4627-0/21/$31.00 ©2021 IEEE 756

Authorized licensed use limited to: ULAKBIM UASL - ISTANBUL KULTUR UNIVERSITESI. Downloaded on March 12,2023 at 07:34:12 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the Third International Conference on Inventive Research in Computing Applications (ICIRCA-2021)
IEEE Xplore Part Number: CFP21N67-ART; ISBN: 978-0-7381-4627-0
Fig. 1. Augmented Reality Architecture Diagram
II. LIT ERAT URE SURVEY
Augmented Reality (AR) is one of the newest technologies
to emerge in recent years. R. Aggarwal et. al. [1] provides
us information about augmented reality and its various types.
An analysis of the pros and cons of all the types has been
provided. An overview of major applications of augmented
reality is given and potential threats have been listed. R.
Aggarwal et. al. is based on the research of various AR based
applications and their effects on day to day human lives. A.
Kanagaraj S et. al. [2] present techniques to search a image
using a technique which is content based and the details of the
same are received from the database. Details of the product are
displayed upon being scanned. Both Android and Windows
versions of the software are available. It allows the customers
to buy items with ease in a complete in- person experience.
The application has been built to keep up with the changes in
E-commerce trends and the growth in usage of new gadgets
like smartphones and tablets. The system proposed by K.
Srinivasan et et. al. [3] has implemented features like
foreground extraction, extraction of human silhouette, feature
points Extraction, warping of shirt, virtual fitting. Things
work without the help of any additional light adjuster or 3D
viewer. It provides a new and better tool for online marketers
for improving the market. 3D view of furniture is made
available by using depth camera in a study by J.
K. T. Tang et. al. [4] Optimum positioning of the specific
pieces of furniture is recommended by taking into account
the spatial and functional relationship. The software works
well for scanning and producing 3D models on floor. An
acceptable arrangement for the selected models of furniture
is suggested by the software. X. Li et. al. [5] proposed a new
approach of feature matching. Features are extracted by SIFT,
which are invariant to rotation and scaling of image, and are
partially affected by the view-point and the lighting factors.
By using this approach a new technology of AR E-commerce
applications has been implemented. Transient chaotic neural
network has been used to match the features and perform the
initial phase of tracking. S. Sharma, et. al. [6] proposed a new
technique which aids users in providing an improvised
perspective using markerless augmented reality. The main
benefit of this approach is that there is no need of placing
markers in the surface area beforehand. The drawback of
this paper is that the 3D model of the object moves along with
the camera as it is aligned according to the camera. C. Lv et.
al. [7] have presented body motion and touchscreen interaction
between the users and a smartphone-based AR system.
Interaction based on touch screen was performed through
Raycast Method and the Gyro sensor in a smartphone was
used to perform body motion based interaction. However
experimental result shows that touch screen-based interaction
was not efficient in this paper. The purpose of S. Iuliana et.
al’s. [8] study is to introduce the concept of Augmented
Reality (AR) and a summary of the methods used in AR. In
order to provide the user with excellent AR experience of
his surroundings
978-0-7381-4627-0/21/$31.00 ©2021 IEEE
Authorized licensed use limited to: ULAKBIM UASL - ISTANBUL KULTUR UNIVERSITESI. Downloaded on March 12,2023 at 07:34:12 UTC from IEEE Xplore. Restrictions apply.
they have surveyed technical requirements. Various domains
were taken into account in order to observe how AR system
interacts with them. Desti et. al. [9] focuses on growth and
impact of E-commerce, and users’ attitude towards
Augmented Reality technology in e-commerce. A survey was
made to test the potential of augmented reality as a tool in
market.They concludes with positive results showing the
acceptance of AR in E-commerce. S. Chakraborty et. al. [10]
present a survey that has analysed how augmented reality will
impact the shopping industry. The survey results show that
people who love using new gadgets and are technology savvy
will adapt to this technology quickly. Linear regression has
been used to analyse the data of 127 people who were
surveyed. It has been found out from their study that income
levels do not a play a role into people’s decision of using this
new technology. O. A. Egaji et. al. [11] propose an application
which allows users to visualize the 3D models of diamond
rings using their smartphones. They used the exploratory factor
analysis method to comprehend the variables of the dataset.
They conducted a survey to explore the compatibility and
likeability of augmented reality in the shopping world and
results were positive. S.Chakraborty et. al. [12] have used
SLAM algorithm to explain the 3D environments and its
visualization in real time. J. Pooja et.al. [13] have the
technology to help students learn subjects such as Engineering
Mathematics, Electromagnetism, and others by superimposing
3D models of the concepts into the real environment (online
largely while staying away).The paper is divided into two
sections: The marker-based AR approach is used in the first
step, which involves bringing an image target in front of
the camera and superimposing it with a 3D model. Feature
extraction, feature description, and feature matching,
homography, projection, and bringing a 3D object into the
scene are all part of it. The second section is the creation of an
Android application that explains an engineering idea utilizing
a markerless AR technique. There has been a comparison of
both portions of the work.
III. PROPOSED SOLUT ION
The proposed system consists of an application that allows
users to place the 3D models of the products and rotate, scale-
up, scale-down, change colours, and perform other object
manipulation to help them visualize the products better. This
is done by using markerless augmented reality that will aim
to enhance user experience. The system uses mobile phones
having ARcore to support the features provided by th e ap-
plication. Sceneform is a framework added by Google to the
AR-core SDK. It facilitates the rendering of 3D objects. It
has an API which handles the high-level graphics. It comes
with a plugin for Android Studio which can be used to
import libraries and build 3D assets. The camera scans the
surface or the body depending upon the product and generates
virtual objects similar to the actual product. Unity 3D is used
for the set up of these models. Google poly API is used
for simulation and modelling purposes.The selected model is
rendered in the user’s physical space using Google ARCore.
757
 Proceedings of the Third International Conference on Inventive Research in Computing Applications (ICIRCA-2021)
IEEE Xplore Part Number: CFP21N67-ART; ISBN: 978-0-7381-4627-0

ARSceneView is a component of scene form which is used to

render these models onto the surface. It detects the surface
and anchor points are highlighted. The models are mapped
to the user’s phone’s screens to decide the dimensions of
the model. These dimensions can be changed along with the
appearance of the model as mentioned above. A transformable
node is used to facilitate the features such as rotation and
scaling. Android provides a library called PickColourDialog
which can be used to change the colours of the models.
ModelRenderable is a library that essentially works towards
the rendering of the model. Similarly, ViewRenderable is used
for the rendering of text. A variety of tools are provided by
ARCore for understanding objects in the real world.
Horizontal and vertical surfaces and planes are detected by
ARCore. Motion Tracking is also included, which makes the
phones understand and track their positions relative to the
world. As it is improving and expanding each day, it will soon
add semantics-based understanding about the environment.
Figure 1 shows the architecture of the whole system. It
includes all the main modules that will be used in the
application. The camera module of the mobile phone, when
instantiated, opens the scanning module which then scans the
surface for markerless AR and returns this to the 3D
construction module which creates the 3D scene and renders it.
Fig. 2. Flow Chart
Optical flow method tracks interest points through camera
motion Markerless AR systems incorporating some or all of
the following components:- Interest Point Detection and
Matching Interest and Matching Interest Point Detection are
the backbone of computer vision and recognition methods and
processes . Points of interest are, in a general sense, 2D image
coordinates that differ differently in relation to their
surroundings.A positive interest point is always different in
appearance by changes such as rotation, scale, shear. Corner
Detectors: This operation is basically a rate-of-change in
gradients. It analyzes the partial derivatives. A good corner is
the one where horizontal and vertical gradients meet. Optic
Flow: Optical flow method tracks interest points through
camera motion instead of matching two interest points. Optic
flow is the process of observing the change in the image data
with the goal of recovering the true image flow. Tracked
points drift from their locations due to errors as each point is
tracked relative to the previous location. Whenever an input is
in the form of a continuous video stream, or images with very
small spatial separation optical flow method is used. Feature
Descriptors and Descriptor Matching: There are two feature
descriptor algorithms (SIFT and SURF). The Euclidean
distance is used in float point descriptors(SIFT,SURF). The
smaller the distance between two descriptors, the more similar
they are. In the sets of descriptors, lowest distance makes the
first set a best match to the second set of descriptors. Pose Fig. 3. Block Diagram
Estimation: Creating an appropriate projection matrix is very
important in augmented reality as the 2D scenes are projected The block diagram in Figure 2 shows all the different modules
into 3D objects using it. Best algorithm to detect pose is that work in our application. The flow of the application is
RANSAC. shown from the beginning at user’s side and how the internal
modules react to the users’ inputs and how it scans the users’
space and retrieves all the data from the stored data in the.

978-0-7381-4627-0/21/$31.00 ©2021 IEEE 758

Authorized licensed use limited to: ULAKBIM UASL - ISTANBUL KULTUR UNIVERSITESI. Downloaded on March 12,2023 at 07:34:12 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the Third International Conference on Inventive Research in Computing Applications (ICIRCA-2021)
IEEE Xplore Part Number: CFP21N67-ART; ISBN: 978-0-7381-4627-0

I V. RESULT S

Our application lets the user choose a product from a variety

of available products. And by clicking on the launch button,the
user can see the AR view. The user can change the colour of
the rendered model and also resize the model according to the
requirement. Proposed system has features such as rotating and
transforming the image. This will provide customers with an
interactive experience. The description of the rendered product
will also be displayed in the AR view. The above mentioned
features can be seen in Figures 3,4,5 below.

Fig. 4 Mannequin model with scale co-ordinates

Fig. 3 AR model of Sofa with description

Fig. 5 Color changing feature

978-0-7381-4627-0/21/$31.00 ©2021 IEEE 759

Authorized licensed use limited to: ULAKBIM UASL - ISTANBUL KULTUR UNIVERSITESI. Downloaded on March 12,2023 at 07:34:12 UTC from IEEE Xplore. Restrictions apply.
 Proceedings of the Third International Conference on Inventive Research in Computing Applications (ICIRCA-2021)
IEEE Xplore Part Number: CFP21N67-ART; ISBN: 978-0-7381-4627-0

V . CONCL USION AND FUT URE W ORK [12 ] S. Saeedi, B. Bodin, H. Wagstaff, A. Nisbet, L. Nardi, J. Mawer,
N. Melot, O. Palomar, E. Vespa, T. Spink, C. Gorgovan, A. Webb,
Augmented Reality is changing life of the people in many J. Clarkson, E. T omusk, T . Debrunner, K. Kaszyk, P. Gonzalez-De-
Aledo, A. Rodchenko, G. Riley, C. Kotselidis, B. Franke, M. F. P.
ways including health, education, shopping experience, O’Boyle, A. J. Davison, P. H. J. Kelly, M. Luján, and S. Furber,
gaming. Augmented reality in E-commerce provides new “Navigating the landscape for real-time localization and mapping for
opportunities for research in the field of online shopping as robotics and virtual and augmented reality,” Proceedings of the IEEE,
vol. 106, no. 11, pp. 2020–2039, 2018.
the customer will get benefit using this application and this [13 ] P. J, V. M, V. G. Pai, and A. M, “Comparative analysis of marker and
application will also provide the user a better understanding, marker-less augmented reality in education,” in 2020 IEEE International
enabling them to make a more proper decision before Conference for Innovation in Technology (INOCON), 2020, pp. 1–4.
purchasing the product. The proposed system helps the user to
view any 2D image of a product in AR view. This proposed
system uses Kudan SDK to provide markerless augmented
reality. The user using this application will be able to view a
3D model in various orthographic different views i.e. front
view, back view, side view. Moreover, the user will also be
able to change the colour of the product and rotate the product
with the fingers. The user can also view the product in the
same scene multiple times just by clicking on the product
again. In the future the system can include a suggestion
module. This module will give suggestions to the user about
other products that can be purchased along with the product
that is being purchased. Multiple objects of different types can
also be rendered simultaneously to make different
comparisons. This application can also be made intelligent by
generating 3D models automatically using generative query
algorithms.

REFERENCES

[1] R. Aggarwal and A. Singhal, “Augmented reality and its effect on our
life,” in 2019 9th International Conference on Cloud Computing, Data
Science Engineering (Confluence), 2019, pp. 510–515.
[2] Ananda Kanagaraj S, Arjun G, and Sree Sharmila T , “Pseudo eye:
T he next-generation shopping application using augmented reality,”
in 8th International Conference for Internet Technology and Secured
Transactions (ICITST-2013), 2013, pp. 549–553.
[3] K. Srinivasan and S. Vivek, “Implementation of virtual fitting room
using image processing,” in 2017 International Conference on Computer,
Communication and Signal Processing (ICCCSP), 2017, pp. 1–3.
[4] J. K. T . T ang, W. Lau, K. Chan, and K. T o, “Ar interior designer:
Automatic furniture arrangement using spatial and functional relation -
ships,” in 2014 International Conference on Virtual Systems Multimedia
(VSMM), 2014, pp. 345–352.
[5] X. Li and D. Chen, “Augmented reality in e-commerce with markerless
tracking,” in 2010 2nd IEEE International Conference on Information
Management and Engineering, 2010, pp. 609–613.
[6] S. Sharma, Y. Kaikini, P. Bhodia, and S. Vaidya, “Markerless augmented
reality based interior designing system,” in 2018 International Confer-
ence on Smart City and Emerging Technology (ICSCET), 2018, pp. 1–5.
[7] C. Lv, X. Yang, and J. Yu, “Interaction design in augmented reality
on the smartphone,” in 2016 9th International Congress on Image
and Signal Processing, BioMedical Engineering and Informatics (CISP-
BMEI), 2016, pp. 1317–1321.
[8] S. Iuliana, C. Ciprian, and B. Costin-Anton, “ A survey on augmented
reality,” 2017.
[9] Desti, Kannaiah, and B. Shanthi, “Journal of marketing and consumer
research,” 2015.
[10 ] S. Chakraborty and D. Gupta, “A study of the factors impacting the
adoption of augmented reality in online purchases in india,” in 2017
2nd IEEE International Conference on Recent Trends in Electronics,
Information Communication Technology (RTEICT), 2017, pp. 1526–
1529.
[11 ] O. A. Egaji, I. Asghar, W. Warren, M. Griffiths, and S. Evans, “An
augmented reality application for personalised diamond shopping,” in
2019 25th International Conference on Automation and Computing
(ICAC), 2019, pp. 1–7.

978-0-7381-4627-0/21/$31.00 ©2021 IEEE 760

Authorized licensed use limited to: ULAKBIM UASL - ISTANBUL KULTUR UNIVERSITESI. Downloaded on March 12,2023 at 07:34:12 UTC from IEEE Xplore. Restrictions apply.