K. J.

Somaiya College of Engineering, Mumbai-77

(A Constituent College of Somaiya Vidyavihar University)
Department of Electronics Engineering

Course Name: AR and MDS

Semester: VI

Experiment No: Four

Title: Inserting AR object in Webcam

Date of Performance: 23 / 02 / 2023

Faculty Name:
Dr. Ninad Mehendale

Name and Roll No.:

Satbir Singh Sohal, 16010320105
AR and MDS 2022-23
Contents
1 Aim and Objective of the Experiment: 3

2 Theory: 4

3 Code: 4
3.1 Tool Used/language: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.2 Programs: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.3 Output: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

4 Post Lab Question: 6

5 Conclusion 7
AR and MDS 2022-23

Experiment: Four
Inserting AR object in Webcam
Satbir Singh Sohal, 16010320105

April 28, 2023

1 Aim and Objective of the Experiment:

Aims:
1. To investigate the feasibility of inserting AR objects into a real-time webcam feed.
2. To determine the accuracy and precision of the AR object placement in the webcam feed.
3. To explore the potential applications of AR object insertion in various fields such as education,
entertainment, and advertising.
Objectives:
1. To develop a software program or application that can insert AR objects into a webcam feed.
2. To test the program or application on different devices and operating systems to ensure
compatibility.
3. To evaluate the quality of the AR object insertion, such as the clarity of the image, the stability
of the object, and the accuracy of the placement.
4. To assess the user experience of interacting with the AR objects in the webcam feed, including
ease of use and intuitiveness.
5. To identify potential applications for the AR object insertion technology in different fields and
industries.
6. To explore the technical challenges associated with the development and implementation of AR
object insertion in a webcam feed and propose solutions to overcome them.
AR and MDS 2022-23
2 Theory:
Augmented Reality (AR) is a technology that blends virtual and real-world environments, enabling
users to interact with virtual objects as if they exist in the real world. AR applications can be created
using various programming languages and software development tools. In this case, MATLAB,
a widely-used programming language for technical computing and visualization, has been used to
insert an AR object into a webcam feed.
To insert an AR object into a webcam feed using MATLAB, several steps need to be followed.
First, the webcam feed needs to be captured using MATLAB’s built-in video capture functions.
The captured video frames are then processed to identify markers or features that can be used to
anchor the virtual object in the real world. The markers or features can be detected using computer
vision algorithms such as feature detection, template matching, or machine learning-based object
recognition.
Once the markers or features are detected, the virtual object can be inserted into the webcam feed
using projection techniques. The projection techniques involve rendering the virtual object onto the
webcam feed based on the position and orientation of the markers or features detected in the video
frames. The rendered object can be textured and illuminated to match the lighting conditions in the
real world, providing a realistic appearance.
The accuracy and stability of the AR object insertion depend on the quality of the markers or features
detected in the video frames and the precision of the projection techniques used to render the virtual
object. To improve the accuracy and stability of the AR object insertion, advanced computer vision
techniques such as multiple view geometry, optical flow, and depth estimation can be used.
AR object insertion has various applications, including gaming, education, advertising, and industrial
design. For example, in gaming, AR can be used to overlay virtual characters or objects onto the
real-world environment, creating immersive gameplay experiences. In education, AR can be used to
provide interactive learning experiences that help students visualize complex concepts. In advertising,
AR can be used to create engaging and interactive marketing campaigns that capture the attention
of consumers.

3 Code:
3.1 Tool Used/language:
Using AR (Augmented Reality) with MATLAB provides a powerful platform for developing AR
applications. MATLAB provides various toolboxes and functions for computer vision, image processing,
and machine learning, which can be used to develop AR applications. These toolboxes and functions
enable MATLAB users to develop AR applications that can detect markers, track objects, and overlay
virtual content on real-world scenes.
AR and MDS 2022-23
In MATLAB, AR applications can be developed using the Image Processing Toolbox, Computer
Vision Toolbox, and the Robotics System Toolbox. These toolboxes provide functions for image
acquisition, feature extraction, feature detection, image registration, and projection, which are
essential for AR applications. MATLAB’s Simulink also provides blocks that can be used for AR
application development.
To develop an AR application in MATLAB, the following steps can be followed:
1. Capture video frames: Capture video frames from a camera or a video file using MATLAB’s
video capture functions.
2. Detect markers or features: Use MATLAB’s computer vision functions to detect markers or
features in the video frames that can be used to anchor virtual objects in the real world.
3. Track objects: Use MATLAB’s tracking algorithms to track the detected markers or features
across multiple video frames.
4. Render virtual objects: Use MATLAB’s 3D graphics capabilities to render virtual objects and
overlay them onto the real-world scene.
5. Project virtual objects: Use MATLAB’s projection functions to project the virtual objects onto
the real-world scene based on the position and orientation of the markers or features detected
in the video frames.
6. Display the augmented video: Display the augmented video on the screen or export it to a
video file.

3.2 Programs:
cam = webcam(1);
objectImg = imread(’download.jpg’);
objectImg = imresize(objectImg, [480, 640]);
videoPlayer = vision.VideoPlayer(’Position’, [100, 100, 640, 480]);
while true
frame = snapshot(cam);

overlayFrame = imadd(frame, objectImg);

step(videoPlayer, overlayFrame);
end
AR and MDS 2022-23
3.3 Output:

4 Post Lab Question:

1. What are some potential applications of AR object insertion in a webcam feed?
There are many potential applications of AR object insertion in a webcam feed. Here are some
examples:
(a) Gaming: AR can be used to create immersive gaming experiences by overlaying virtual
objects or characters onto the real-world environment.
(b) Education: AR can be used to provide interactive learning experiences that help students
visualize complex concepts, increase student engagement and motivation, and enhance the
retention of information.
(c) Advertising: AR can be used to create engaging and interactive marketing campaigns that
capture the attention of consumers.
(d) Industrial Design: AR can be used to visualize and simulate the design of products in real-
world environments, allowing designers to test and refine their designs before producing
physical prototypes.
(e) Entertainment: AR can be used to create interactive and engaging entertainment experiences,
such as virtual concerts or art installations.
(f) Healthcare: AR can be used to create simulations and training programs for healthcare
professionals, allowing them to practice and refine their skills in a safe and controlled
AR and MDS 2022-23
environment.
(g) Retail: AR can be used to create virtual try-on experiences for clothing and accessories,
allowing customers to see how products will look on them before making a purchase.
2. What computer vision techniques can be used to detect markers or features in a webcam feed?
There are several computer vision techniques that can be used to detect markers or features in
a webcam feed. Here are some examples:
(a) Feature Detection Algorithms: Feature detection algorithms, such as Harris corner detection,
SIFT (Scale-Invariant Feature Transform), and SURF (Speeded Up Robust Feature), can
be used to identify distinctive image features in the video frames. These algorithms work
by detecting and extracting key points or features that are unique to each frame.
(b) Template Matching: Template matching techniques can be used to detect specific objects
or patterns in the video frames. This technique involves comparing a template image to
the video frame and looking for areas of similarity.
(c) Machine Learning-Based Object Recognition: Machine learning-based object recognition
techniques, such as convolutional neural networks (CNNs), can be used to detect and
classify objects in the video frames. These techniques involve training a machine learning
model on a large dataset of images and using the model to classify objects in new images.
(d) Optical Flow: Optical flow techniques can be used to estimate the motion of objects in
the video frames. Optical flow works by analyzing the movement of pixels between frames
to determine the direction and speed of object motion.
(e) Multiple View Geometry: Multiple view geometry techniques can be used to estimate the
position and orientation of objects in the scene from multiple camera viewpoints. This
technique involves using geometric calculations to triangulate the position of objects based
on their appearance in multiple video frames.

5 Conclusion
1. We successfully inserted an AR object in webcam feed using MATLAB.
2. We explored the various applications of inserting AR objects in webcam feed as well as the
computer vision techniques to detect markers or features in the webcam feed.

References
[1] Anónimo, A. (2008). Blog del año 2002 con un formato horrible. http://www.blogfeo.
blogspot.com.
AR and MDS 2022-23
[2] Mendoza, S. and Duende, J. (2022). Libro apantallante que probablemente no leı́ste y solo lo
pusiste por poner algo. Ediciones Tepito, 1st edition.

[3] Pérez, J. (2008). Artı́culo de dudosa calidad de una página web patito. Recuperado de: https:
//XD.com. Fecha de consulta: 03/11/2021.