Virtual Trial Room for Online Shopping
K. Durga Prasad
Department of Computer Applications
Velagapudi Ramakrishna Siddhartha Engineering
College, Vijayawada, India
durgaprasadkorikana88165@gmail.com
Abstract—Human Beings have the natural ability to
recognize body and sign language easily. This is possible
because of vision and synaptic interactions formed during
brain development. Humans also can pick up contextual
information to understand body and sign language. If we
want to replicate this ability in computers, we need to solve
several issues—the ability to differentiate between objects
of interest and the background. Choosing image capture
technology and image classification algorithms etc. So,
Hand gesture recognition is quickly becoming an
important and relevant technology. Given the recent
growth of AR and VR technologies, Hand gesture
recognition has become an exciting new technology field.
But currently, hand gestures are very specialized and
haven’t acquired mainstream adoption. We aim to show
that gesture recognition can enhance many aspects of
computer work and teaching. In our predicted model, the
system works by the user starting the program, which
would start providing the live webcam feed. The user then
makes the hand gesture inside the frame of detection
shown on the screen. When the hand gesture is made, the
gesture is segmented and isolated..Some possible
applications are to reduce the use of a physical keyboard
and mouse by using hand gestures recognition to control
your Computer like Pause/Play, close and open windows,
and manipulate media controls. Another application is
sign language. Hand gesture recognition here can be used
for communication and teaching purposes.
Index Terms - Convolution Neural Networks, Pattern
Recognition, Hand Gesture Recognition, Segmentation of Hand
Region, Data Collection, and Deep Learning.
I. INTRODUCTION
Purchasing clothing or accessories online poses inherent
risks, mainly because it's difficult to gauge how the items
will appear on oneself. Similarly, shopping for clothes or
jewellery in physical stores demands significant time
investment, involving the search for suitable shops and
trying on multiple items in fitting rooms. Our proposed
solution aims to streamline this process by digitizing the try-
on experience, thereby saving users time. We've opted to
utilize OpenCV for its efficiency and pre- trained
capabilities in detecting the user's body, enabling us to
overlay clothing swiftly. This approach not only expedites
the process but also enhances the user experience. Users
V Esther Jyothi
Asst. Professor, Department of Computer Applications
Velagapudi Ramakrishna Siddhartha Engineering
College, Vijayawada, India
vejyothi@vrsiddhartha.ac.in
receive real-time results, with the selected attire
superimposed on their bodies as they interact with the
application. This real-time feedback is achieved by
capturing and processing every frame of the video input,
seamlessly integrating the attire onto the user's body and
returning the modified frame for immediate viewing
Importantly, our implementation is cost-effective, requiring
no additional hardware expenses, unlike some existing
solutions. Moreover, it's platform-independent, capable of
running on any operating system and device with a camera,
internet access, and a web browser. The primary concerns
driving our project are twofold: ensuring the accuracy of
attire superimposition according to the user's body and
delivering a realistic viewing experience.
A. Precision in Superimposition:
The accuracy of superimposing wearables in a virtual trial
room is paramount as it dictates the system's overall
precision. The effectiveness relies heavily on whether the
application's algorithm can successfully identify the user
within video frames. Two primary methods exist: utilizing
neural networks to train the algorithm to detect the human
body or employing an RGB color marker to locate the user
within the frame. The latter method, while less user-
friendly, is mitigated by OpenCV's pre- trained algorithm
designed to recognize body parts such as the face, upper
body, and lower body, making it the preferred approach.
B. Enhancing Realism in the Virtual Trial Room:
A Virtual Reality setup falls short if it fails to evoke a sense
of reality, even though achieving complete realism akin to
the sensation of wearing different fabric types, like cotton
versus wool, remains challenging. Nevertheless, we can
enhance the user's experience by simulating a more realistic
environment, akin to trying on clothes in front of a mirror
within an authentic Tryon room. Augmented Reality blends
real-world elements seamlessly with virtual ones, refining
the user's perception of their surroundings. By overlaying
projected images onto the user's view, augmented reality
merges synthetic and natural light, creating an immersive
experience. Augmented Reality devices operate
independently, without the need for cables or desktop
computers.
OpenCV, short for Open Source Computer Vision Library,
supports various programming interfaces like Python, C++,
and Java. It prioritizes computational efficiency and real-
time applications, boasting the advantage of multi-core
processing for C or C++ code. Leveraging augmented
reality technology streamlines the virtual try-on process,
saving customers time and reducing the confusion
associated with online wearables shopping.
II. RELATED WORK
The current methodology of online searching frequently
fails to ensure accurate apparel sizing, resulting in a plethora
of product returns and prolonged replacement processes.
This challenge presents a substantial hurdle for the e-
commerce sector.
Numerous strategies have been proposed to tackle this issue:
Srinivasan K. and Vivek S. [1] delve into the burgeoning
realm of online shopping, highlighting the imperative need
for algorithms capable of digitally fitting clothing onto
individual silhouettes. They underscore the intricacies of
achieving precise fitment, especially within static images
amid varying backgrounds and noise levels.
Pros: Enriched online shopping experiences through virtual
try-on functionalities.
Cons: Absence of 3D visualization capabilities and
susceptibility to lighting variations.
Ari Kusumaningsih and Eko Mulyanto Yuniarno [2] pioneer
a bespoke virtual dressing room designed specifically for
Madura batik attire. Their objective is to invigorate sales
and perpetuate cultural heritage by delivering a tailor-made
digital fitting encounter, necessitating streamlined
computational methodologies for efficacious handling of 3D
models.
Pros: Accurate spatial perception for seamless object
placement.
Cons: Impact of lighting conditions on depth map precision.
Ting Liu and Ling Zhi Li [3] harness Kinect technology for
user segmentation and skeletal tracking to align clothing
models with users within a prototype virtual dressing
software. They underscore the criticality of swift and
authentic clothing modeling techniques.
Pros: Real-time visualization of clothing ensembles.
Cons: Static alignment constraints overlooking dynamic
movements.
Stephen Karungaru and Kenji Terada [4] advocate for a
Kinect-centric approach to effortlessly acquire human body
measurements. Despite successful data acquisition,
challenges persist in enhancing accuracy and user-
friendliness.
Pros: Emphasis on human-centric data acquisition.
Cons: Shortfall in interactive features and accuracy
calibration.
Dr. Anthony L. Brooks and Dr. Eva Petersson Brooks [5]
collate extensive feedback through open surveys to refine
their wheelchair-adaptive product, aiming to elevate user
experience and accessibility, notwithstanding technical
intricacies in execution.
Pros:Cutting-edge camera technologies facilitating
personalized digital overlays.
Cons: Escalating system complexity.
Reizo NAKAMURA and Masaki IZUTSU [6] delineate a
methodology for estimating body dimensions leveraging
Kinect data. They advocate for the utilization of multiple
Kinect sensors to enhance accuracy, albeit at a premium
cost.
Pros: Augmented size estimation through amalgamated
sensor inputs.
Cons: Escalating implementation expenses.
Poonpong Boonbrahma and Charlee Kaewrat [7]
prognosticate fabric appearances based on physical
parameters, aiding in material differentiation for virtual
fitting rooms. However, they stress the necessity for
meticulous experimentation to validate their predictions.
Pros: Fabric simulation across diverse environments.
Cons:Imperative requirement for comprehensive
experimental validation.
Umut Gültepe and Uğur Güdükbay [8] introduce a virtual
fitting room harnessing depth sensor data for lifelike
garment fitting. While promising, there exists a pressing
need to enhance measurement accuracy and collision
detection capabilities.
Pros: Authentic fitting experience enabled by depth sensing
technology.
Cons: Subpar measurement precision and redundant data.
Ayushi Gahlot and Purvi Agarwal [9] focus on Kinect-
driven action recognition and pose estimation, showcasing
the versatility of this technology. However, they stress the
importance of leveraging pose estimation for practical
applications beyond mere action prediction.
Pros: Precision pose estimation facilitated by Kinect
interactions.
Cons: Limited applicability beyond action.
Furthermore, we have extensively explored several
scholarly works delving into the concept of superimposing
attire, predominantly clothing, onto human forms. This
functionality empowers users to visualize themselves
donning various clothing items without the need for
physical try-ons. Initially, the user faces a camera,
capturing their image, which is then overlaid with assorted
garments for display. This streamlined process aids users
in making informed decisions while augmenting their
satisfaction levels.
In the research conducted by Shreya Kamani, F.
Isikdogan, and Vipin Paul (cited as [1], [2], and [6]
respectively), the implementation of virtual trial room
applications utilizing hardware sensors like the Microsoft
Kinect sensor is proposed. These sensors primarily capture
the user's skeletal structure, providing crucial data for
determining the appropriate size for virtual clothing
augmentation. Presently, commercial products leverage
technologies such as Microsoft's Kinect and Asus Xtion
III. METHODOLOGY
A. Detecting and Sizing the Body:
The initial phase of the proposed Online Virtual Trial Room
method involves acquiring the body's shape, including the
head or neck depending on the attire, to establish reference
points. These points are subsequently utilized to determine
the appropriate placement of specific clothing or
accessories. To achieve this, we experimented with various
techniques:
Filtering with thresholding, Canny edge detection, K-means,
and ii) Motion detection or skeleton detection, where
multiple frames were scrutinized for any movement.
However, the outcomes were inconsistent and insufficient
for obtaining accurate reference points to display the attire.
Consequently, we developed a novel detection approach
centered on identifying the user's face, adjusting a reference
point at the neck, and positioning the attire based on that
reference point. Furthermore, an Augmented Reality (AR)
marker can serve as an additional reference point. While
this method proved adequate for small items like glasses or
ornaments, it fell short in accurately mapping clothing onto
the user's body.
Fig. 1. – Freeman’s Codification
To determine the user's size, we employ a similar
automated body feature extraction technique as
demonstrated in [7]. The process involves positioning the
user in front of the camera at a predetermined distance. The
devices, which detect users' body coordinates when
positioned in front of a scanner. This data serves as the
foundation for generating a comprehensive 3D model of
the user.
algorithm identifies key points on the shoulders and
abdomen. By measuring the distance between these points
and the user-to-camera distance, we derive the user's size.
Once the image (video frame) is captured, we apply a
Canny edge detection filter to isolate the body's silhouette.
Given that Canny edge detection is sensitive to noise
present in unprocessed data, we utilize a Gaussian filter to
smooth the raw image. Following convolution, four filters
are employed to detect horizontal, vertical, and diagonal
edges within the processed image. Morphological functions
are then utilized to generate a closed silhouette.
Subsequently, we utilize an 8-point Freeman chain code, as
illustrated in Figure 1, to assign a direction to each pixel.
The choice between using 8 or 4 chain codes depends on
the scenario, with the following formula applicable:
z = 4*(deltax+ 2)+(deltay+ 2) (1)
The formula (1) generates a sequence corresponding to rows
1- 8 in the preceding table: z = {11,7,6,5,9,13,14,15}. These
values serve as indices into the table, enhancing the speed of
computing the chain code. Each difference between
consecutive numbers represents a 45º variation. If the
difference in direction between consecutive points exceeds
two (90º), a feature point is identified and marked in the
image. Numerous commercially available technologies exist
for assessing the precision, accuracy, and completeness of
hand configuration data. These technologies encompass
exoskeletons and instrumented gloves, known as "Data
gloves," which are worn on the hand and body. Data gloves
offer direct measurement of hand and finger characteristics,
data provision, high sampling frequency, ease of use, line of
sight, cost-effectiveness, and translation-independent data
characteristics. However, using a data glove presents certain
challenges, including calibration difficulties, reduced comfort
and range of motion, noise in cheaper systems, and the cost
of acquiring an accurate device.
ek =| dj+1 – dj |= 2 (2)
The equation (2) implies that the absolute difference
between two points exceeds 2, as stated in Eq. (2).
Ultimately, the distance between these points is measured
in the image and correlated with the distance from the user
to the camera to determine size.
B. Face Detection:
Upon the user's approach to the screen, the discrete
structure targeted for identification is the face. To achieve
face detection, Haar feature-based cascade classifiers are
employed. Unlike utilizing pixel intensity values, Haar
classifiers leverage contrasts between adjacent groups of
pixels, using the variance difference to discern light and
dark areas in the image [8]. This method adopts a machine
learning approach, requiring the cascade function to be
trained with numerous negative and positive images. A
plethora of negative images (devoid of faces) and positive
images (containing faces) are presented to the classifier for
training, enabling it to extract features effectively. The
integration of OpenCV facilitates this process, providing
pre-trained classifiers for faces, eyes, smiles, and more.
Equipped with both a trainer and a detector, OpenCV
allows for straightforward training of custom classifiers for
object detection. Upon identifying a match, it returns Rect
(x, y, w, h) denoting the coordinates for the left, top,
bottom, and right boundaries.
C. Image Masking
Image masking involves setting certain pixel intensity
values of the masked image to zero. Wherever the pixel
intensity value in the original image is zero, the
corresponding pixel intensity in the resulting masked image
is typically set to the background value, which is commonly
zero. The regions of interest (ROIs) for each slice are
utilized to define the mask. If necessary, masking can be
managed on a slice-by-slice basis within the ROI toolkit.
Importantly, masking operations in the ROI toolkit do not
impact a slice that lacks an ROI.
F. Proposed Application:
Our application is built using the Python Flask Web
Application Interface. It enables users to browse through
various clothing items and accessories on the website,
giving them the option to either make a purchase or
virtually try on the attire. To initiate the virtual try-on
process, users need to click on the 'Quick View' button,
triggering the execution of the Tryon script. Using
OpenCV, the application captures video from the device
camera and seamlessly superimposes the selected attire
onto the user’s body in real-time. Should the user find the
attire satisfactory, they can proceed with the purchase or
continue exploring additional wearables available on the
website, mimicking the experience of shopping in a
physical store.

D .Edge Detection:
Numerous techniques exist for edge detection, among which
the Canny Edge detection technique [9] has been employed
for body detection, as previously discussed. Gaussian filters
are utilized in this edge detection method to eliminate noise
in digital images, thereby preventing false detections by the
processor. These filters smooth the image and diminish the
impact of noise, facilitating the proper functioning of the
processor. Through this process, the intensity gradients of
the image are determined. Since edges in an image can be
oriented horizontally, vertically, or diagonally, this
algorithm employs four filters to detect all types of edges in
the blurred image. Following this stage, non- maximum
suppression is applied to thin the edges, resulting in
accurate edge pixels relative to actual edges. Additionally, Fig. 2. Screenshots of the Web Application
some pixels may be artifacts of noise, for which a double
threshold is applied.

E. Scaling of Attire:
Scaling refers to resizing the image based on specific
conditions. When the user approaches the screen, the attire's A. Algorithm
size should adjust accordingly to fit the body. As the user
moves closer to the screen, the image size should increase
to accommodate the user's proximity, but the actual
measurements of the attire should remain constant. For
instance, if the person is trying clothes with a size
measurement of S, the size should not change to M or L as
the person moves closer to the screen. Instead, only the
overall view of the clothing should be enlarged or reduced
as needed. This adjustment is achieved through scaling
methods.
C.
Steps involved in the Algorithm:
• Step-1: Begin, import required libraries like os, numpy,
cv2.
• Step-2: Make directories for training and test data using
os, path. exists().
• Step-3: Next call cv2.videocapture() for capturing the
gestures and loading the train data.
• Step-4: Now, map the gestures with desired outputs
using cv2 modules.
 • Step-5: Then import required modules of keras like
sequential, convolutional2D, maxpooling2D, Flatten,
Dense for training the CNN model.
• Step-6: Later, do the image preprocessing using
Keras.preprocessing.image and import
imageDatagenerater and train model with loaded train
IV. RESULTS AND ANALYSIS
Hand gesture recognition and the working of the CNN
model follow standard steps, making it easier for the project
to split into modules.
• Segmentation of hand region: The Hand region of the
webcam feed is to be segmented to save space, remove
unwanted parts of the image, and reduce background
noise. This is achieved by: Background subtraction, Mo-
tion Detection, and Thresholding Contour Extraction.
Opencv is used to achieve this as it is a module that
deals with image processing and computer vision. It can
be used to identify objects and faces. Example functions
cv2.cvtcolor - used to convert an image from one color
space to another cv2. Threshold- used to assign pixel
values about the threshold value cv2.imshow-used to
display an image in a window.
• Dataset collection Data set is obtained by manually
providing the program with images of segmented hand
gestures. Separate folders are made that store the image
corresponding to the hand gesture made. The os module
is used within the program to make new directories. Ex:
os.makedirs - is used to make directories os.Listers -
to list all files in the directory. Opencv is also used here.
Ex: cv2.imwrite - used to save any image to a storage
location. Steps involved in dataset creation.
1) First, we will make new directories for each hand
segment in the ’train’ and the ’test’ modes.
2) We will then turn on video capture to capture the
images for the dataset.
3) Next, we will create a small frame with specific
dimensions within the webcam feed where the hand
gesture is to be made.
4) We will also convert the RGB feed to a black and
white feed to eliminate background noise and make
it easier for the webcam to distinguish the hand from
its surroundings easily.
5) We will only capture the image within the
mentioned frame to remove unwanted background
and reduce space.
6) We make the dataset by making the corresponding
hand gesture and then pressing the key to indicate
the hand gesture. For example, the user gestures
data.
• Step-7: Now execute the code; the model will start
capturing live video sequences and giving gestures.
• Step-8: Check the results predicted by the model and
train the model with different gestures.
for ’5’ by opening the palms of her hand and
outstretched fingers within the window frame and
pressing the key ’5’ on the numeric keypad. This
saves the image in the directory for the ’5’ hand
gesture.
• Training the CNN model: The CNN Model is trained by
classifying the data set collection to the appropriate hand
gesture and creating a 2D Convolutional Neural Network.
Keras is used to create a Convolutional model. There are
two ways to use Keras, sequential or functional. Here
sequential is used for layer-by-layer model creation.
Functions used:
1) Convolution2D - a layer that divides a picture into
many images.
2) MaxPooling2D - a layer that maximizes the value
from a size matrix.
3) Flatten - a layer that flattens the image’s dimensions
after it has been convolved.
4) Dense - a layer that turns this into a fully linked
model.
5) ImageDataGenerator - zooms in on the image, re-
sizes it, applies shear in some range, zooms the image
and does horizontal flipping with the image
train_datagen.flow_from_directory - is the function
that is used to prepare data from the train_dataset
directory Target_size specifies the target size of the
image.
6) Test datagen.flow from directory - is employed to
prepare test data for the model.
• Recognition of hand gestures: Get the thresholded image
from the live feed and use the model to recognize the hand
gesture in the image as shown in Fig. 2. Functions used:
cv2.cvtcolor - used to change an image’s color space from
one to another. imshow- used to show a picture in a cv2
window. To obtain a video capture object for the camera,
use the video capture method.

Fig. 2.
SEGMENTED HAND REGION
V. CONCLUSION
The model can recognize the hand gesture in the webcam
with 80-90% accuracy. The accuracy depends on factors like
room lighting, exposure, distance from the camera, back-
ground noise, etc. Improvement of these conditions can give
a much better hand gesture recognition accuracy.
Discrepancies can be ironed out by providing more data
set images to train the CNN model. The segmented images
need a plain background to pick up the hand gesture easily.
Too much background noise will cause the hand not to be
picked up by the camera. Likewise, overexposure or
underexposure of the camera will also cause issues picking
up hand gestures.
pp. 1–4.
[6] V. E. Jyothi, D. L. S. Kumar, B. Thati, Y. Tondepu, V. K.
Pratap, and S. P. Praveen, “Secure data access management for cyber
threats using artificial intelligence,” in 2022 6th International
Conference on Electronics, Communication and Aerospace
Technology. IEEE, 2022, pp. 693–697.
[7] S. B. Wang, A. Quattoni, L.-P. Morency, D. Demirdjian, and T.
Darrell, “Hidden conditional random fields for gesture recognition,”
in 2006 IEEE Computer Society Conference on Computer Vision and
Pattern Recognition (CVPR’06), vol. 2. IEEE, 2006, pp. 1521–
1527.
[8] D. C. Cires¸an, U. Meier, L. M. Gambardella, and J. Schmidhuber,
“Deep, big, simple neural nets for handwritten digit recognition,”
Neural computation, vol. 22, no. 12, pp. 3207–3220, 2010.
[9] A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet
classification with deep convolutional neural networks,”
Communications of the ACM, vol. 60, no. 6, pp. 84–90, 2017.
[10] P. Y. Simard, D. Steinkraus, J. C. Platt et al., “Best practices for
convolutional neural networks applied to visual document analysis.” in
Icdar, vol. 3, no. 2003. Edinburgh, 2003.
[11] Jyothi, V. Esther, B. D. C. N. Prasad, and Ramesh Kumar Mojjada.
"Analysis of Cryptography Encryption for Network Security." IOP
Conference Series: Materials Science and Engineering. Vol. 981. No.
2. IOP Publishing, 2020.
[12] A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and
L. Fei-Fei, “Large-scale video classification with convolutional neural
networks,” in Proceedings of the IEEE conference on Computer
Vision and Pattern Recognition, 2014, pp. 1725–1732.
[13] S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep
network training by reducing internal covariate shift,” in International
conference on machine learning. pmlr, 2015, pp. 448–456.

REFERENCES
[1] S. P. Praveen, T. B. Murali Krishna, C. Anuradha, S. R. Mandalapu,
P. Sarala, and S. Sindhura, “A robust framework for handling health
care information based on machine learning and big data engineering
techniques,” International Journal of Healthcare Management, pp. 1–
18, 2022.
[2] S. Ahlawat, V. Batra, S. Banerjee, J. Saha, and A. K. Garg, “Hand
gesture recognition using convolutional neural network,” in
International Conference on Innovative Computing and
Communications: Proceedings of ICICC 2018, Volume 2. Springer,
2019, pp. 179–186.
[3] S. Ahmed, K. D. Kallu, S. Ahmed, and S. H. Cho, “Hand gestures
recognition using radar sensors for human-computer-interaction: A
review,” Remote Sensing, vol. 13, no. 3, p. 527, 2021.
[4] C. J. L. Flores, A. G. Cutipa, and R. L. Enciso, “Application of
convolutional neural networks for static hand gestures recognition
under different invariant features,” in 2017 IEEE XXIV international
confer- ence on electronics, electrical engineering and computing
(INTERCON). IEEE, 2017, pp. 1–4.
[5] S. P. Praveen, S. Sindhura, A. Madhuri, and D. A. Karras, “A novel
effective framework for medical images secure storage using advanced
cipher text algorithm in cloud computing,” in 2021 IEEE International
Conference on Imaging Systems and Techniques (IST). IEEE, 2021,