REAL-TIME VIDEO BACKGROUND REMOVER

A MINI PROJECT REPORT

submitted in partial fulfilment of the requirements
for the award of the degree of
BACHELOR OF TECHNOLOGY
In
ELECTRONICS AND COMMUNICATION ENGINEERING
Submitted by
R. Bhavana P. Ratna Rohith
(19B91A04J0) ( 19B91A04H3)

P. Sai Kiran P. Santhi Priyanka

(19B91A04G8) (19B91A04I1)

Under the esteemed guidance of

J. RAVI, M. Tech
ASSISSTANT PROFESSOR

DEPARTMENT
OF
ELECTRONICS AND COMMUNICATION ENGINEERING
S.R.K.R. ENGINEERING COLLEGE
(Affiliated to JNTU, KAKINADA)
(Recognized by A.I.C.T.E., Accredited by N.B.A.,
& Accredited by N.A.A.C. with ‘A’ grade, NEW DELHI)
CHINNA AMIRAM, BHIMAVARAM-534204
(2021-2022)
 S R K R ENGINEERING COLLEGE (Autonomous)
(Affiliated to JNTUK, KAKINADA)
(Recognized by A.I.C.T.E., Accredited by N.B.A,
& Accredited by N.A.A.C. with ‘A’ grade, NEW DELHI)
CHINNA AMIRAM, BHIMAVARAM-534204
DEPARTMENT OF ELECTRONICS AND COMMUNICATION
ENGINEERING

CERTIFICATE
This is to certify that the project work entitled
“REAL-TIME VIDEO BACKGROUND REMOVER”
Is the bona fide work of

Mr./Ms. R. BHAVANA(Reg.no:19B91A04J0), P. RATNA ROHITH (Reg.no: 19B91A04H3),

P.SAI KIRAN (Reg.no: 19B91A04G8), P.SANTHI PRIYANKA (Reg.no:19B91A04I1)
submitted in the partial fulfilment of the requirements for the MINI PROJECT in
BACHELOR OF TECHNOLOGY in ELECTRONICS AND COMMUNICATION
ENGINEERING during the academic year 2021-2022.

Guide: Head of the Department:

Ass. Prof. J. RAVI Dr. N. UDAYA KUMAR
M. TECH M. Tech, Ph.D., S.M.I.E.E.E., F. I.E.T.E.,F.I.E
Department of ECE. Department of ECE.
 ACKNOWLEDGEMENTS

Our most sincere and grateful acknowledgement to our alma mater SAGI
RAMA KRISHNAM RAJU ENGINEERING COLLEGE for giving us the opportunity to
fulfil our aspirations and for successful completion of the project.

We are highly indebted to Asst. Prof. J. RAVI, Department of Electronics and

Communication Engineering, our project guide for giving valuable and timely suggestions for
the project work

We convey sincere thanks to Dr. N. UDAYA KUMAR, Head of the department

of Electronics and Communication Engineering, for his kind cooperation in the successful
completion of project work.

We are grateful to our principal Dr. M. JAGAPATHI RAJU, providing us with

necessary facilities to carry out our project.

We extend our sense of gratitude to all our teaching and non-teaching staff and
all our friends, who indirectly helped us in this endeavor.

- Project Associates
 CONTENTS

ABSTRACT
1. INTRODUCTION
• Computer Vision
• OpenCV
• CvZone
• MediaPipe
2. METHODOLOGY
• Importing Packages
• Implementation of code
3. APPLICATIONS
4. CONCLUSION
5. REFERENCES
REAL-TIME VIDEO BACKGROUND REMOVER

ABSTRACT:

The idea of background removal is not a modern one; instead, it has existed in the field since the dawn of modern
technology. However, to overcome many issues i.e. interruptions in the background or the background color doesn’t
suit the requirement due to which the background needs to be modified. For this real time background replacement
techniques are used to substitute the backgrounds and replace them with desired content. Hence, there have been
numerous techniques for background removal in the domain of computer vision. Nevertheless, classical computer
vision techniques are constantly being overthrown by modern, artificial intelligence-based ones by lodging more
innovative ideas giving more precise and accurate results. A problem of background interruption is encountered
during video conferencing which results in inconvenient conversation. This often causes reputational damage to an
individual. For this background remover is a solution. Although background replacement is a quite aspect in the
present video conferencing applications. In this project Image Processing is done by using Python.
INTRODUCTION:

Background replacement, now enjoys wide-spread use in video conferencing tools like Zoom, Google Meet, and
Microsoft Teams. Communication through video calling is most common aspect these days. Sometimes background
interrupts worthwhile communication or an individual may not wish to share background for some reasons. Having a
feature to hide background space establishes a convenient and effective conversation. In addition to adding
entertainment value, background replacement can enhance privacy, particularly in situations where a user may not
want to share details of their location and environment to others on the call. To overcome this issue and establish a
better communication a program is developed. Through this background is removed and add new background
according to our convenience.

Before jumping into image processing, first understand what exactly constitutes an image. An image is represented
by its dimensions (height and width) based on the number of pixels. For example, if the dimensions of an image are
500 x 400 (width x height), the total number of pixels in the image is 200000.This pixel is a point on the image that
takes on a specific shade, opacity or color. It is usually represented in one of the following:

Grayscale - A pixel is an integer with a value between 0 to 255 (0 is completely black and 255 is completely white).
RGB - A pixel is made up of 3 integers between 0 to 255 (the integers represent the intensity of red, green, and blue).
RGBA - It is an extension of RGB with an added alpha field, which represents the opacity of the image.

Image processing is the process of transforming an image into a digital form and performing certain operations to get
some useful information from it. The image processing system usually treats all images as 2D signals when applying
certain predetermined signal processing methods. Image processing requires fixed sequences of operations that are
performed at each pixel of an image. The image processor performs the first sequence of operations on the image,
pixel by pixel. Once this is fully done, it will begin to perform the second operation, and so on. The output value of
these operations can be computed at any pixel of the image.

There are five main types of Image processing:

1.Visualization - Find objects that are not visible in the image

2.Recognition - Distinguish or detect objects in the image
3.Sharpening and restoration - Create an enhanced image from the original image
4.Pattern recognition - Measure the various patterns around the objects in the image
5.Retrieval - Browse and search images from a large database of digital images that are similar to the original image.

Image processing basically includes the following three steps:

1. Importing the image.

2.Analysing and manipulating the image.
3.Output in which result can be altered image or report that is based on image analysis.

Popular background removal techniques:

• Edge-based background removal- This technique detects edges in the image and then finds a continuous
edge path. All the elements that are outside the path are considered as background.

• Foreground detection- Foreground detection is a technique that detects changes in the image sequence. The
background subtraction method is used here to separate the foreground from an image.

• Machine-Learning Based Approach- In this technique various machine learning algorithms are used to
separate the foreground from the background.
 • Image clipping path – This technique is used if the subject of the image has sharp edges. All those elements
that fall outside the path will be eliminated.

• Image cut-out – Cut the required region or subject in a frame and remove the background
• Image masking – If the images have frills or fine edges use image masking techniques.

• Erasing the background – Erasing the background of an image using any different tools.

Computer vision is a process by which one can understand the images and videos how they are stored and how it can
be manipulated and retrieve data from them. Computer Vision is the base or mostly used for Artificial Intelligence.
Computer-Vision is playing a major role in self-driving cars, robotics as well as in photo correction apps.

OpenCV is the huge open-source library for the computer vision, machine learning, and image processing and now it plays a
major role in real-time operation which is very important in today’s systems. By using it, one can process images and videos to
identify objects, faces, or even handwriting of a human. When it integrated with various libraries, such as NumPy, python is
capable of processing the OpenCV array structure for analysis. To Identify image pattern and its various features vector space
is used and mathematical operations are performed on these features.

CV zone is a computer vision package that makes us easy to run like face detection, hand tracking, pose estimation, etc., and
also image processing and other AI functions. At the core, it uses OpenCV and Media Pipe libraries.

Media Pipe is Google’s open source framework, used for media processing. It is cross-platform or say it is platform
friendly. Media Pipe offers ready-to-use yet customizable Python solutions as a prebuilt Python Package. Media Pipe
is a framework for building machine learning Pipelines for processing time series data like video, audio, etc. It
performs optimized with end-to-end on device inference in mind.

METHODOLOGY:
Background removal or selfie segmentation basically returns a segmented mask of the foreground from the image.
Foreground means the subject, in this case, humans are the foreground in the image, and the rest of the things are the
background.
What is segmentation?
Segmentation is an image processing technique that returns a binary black and white image mask of a targeted image.
It is a process of labeling pixels that shares certain characteristics. Segmentation is one of the most difficult steps of
image processing. It involves partitioning an image into its constituent parts or objects.

What is selfie-segmentation?
Media pipe’s selfie segmentation API separates humans from the background within a scene. It creates a binary mask
of the scene and focuses only on humans. The value of the background pixels is 0 and the value of the interested area
contains greater than 0 but the maximum is 1 depending on certain criteria like lighting, the complexity of the scene,
etc.

Brief description of steps:

Below are the steps to develop remove image background project in python:

1.Import necessary packages.

2. Initialize selfie-segmentation object.
3. Read frames from a webcam.
4. Create the segmented mask.
5. Replace the background with an image.

Many famous applications use a background removal technique and replace it with a custom one. Here we are going
to implement something similar, but using OpenCV and CVzone.

Let’s start the implementation

Install the required modules.

-- pip install OpenCV-python

-- pip install CVzone

-- pip install Mediapipe

First, check if webcam is working fine.

The above code pops up a window if there is a webcam, Here the frame size is 640 X 480. So take a note here because
the background replacing images should be of the same size as the frame, that is 640 X 480.

Webcam output

Now create the folder inside the project directory here, let the name be ‘BackgroundImages’. Any images or any
number of images can be downloaded and place them in this directory.

The project structure will look like the image given below:

Project structure
Write a small piece of code in a separate python file to resize all the images in the
folder ‘Background Images’ to 640 X 480.

The above code will read the image (jpg) files in the specified folder and resize all the images to 640 X480 at once.

Output after resizing all the images

Now, all set to implement the background replacement technique.

Import the required modules

Above module, ‘SelfiSegmentation’ is used to remove the background of the frame and replace it with our images in
the directory.
In the above code, input from the webcam is taken and also set the frame width to 640 X 480. Then
call SelfiSegmentation() and assign it to a variable called segmentor, and in order to display the frames per second(fps)
in the output frames, use cvzone.FPS() function.

Then create a list of images present in the Background Images folder and loop through that list and read each and
every image and append it to an empty list. The initial index is set to zero.

Now the main part, inside a while loop reads the frames from the webcam, and then
use segmentor.removeBG() function to remove the background from the frames and replace it with our images in the

directory. In the above code, you can see three parameters are passed to segmentor.removeBG() function, that is image
frame from webcam (img), then the list of images present in the directory along with an index of
image (imgList[indexImg]) and finally the threshold. The threshold cuts everything if it’s set to 1, here set it to 0.8,
for better edges, play with different threshold values.

Then stack the images using cvzone.stackImages, here the output of the background replaced image or frames will be

achieved. Then using a simple if statement assign keys to change the background. For example, if there are 10
background images, as per the above code one can use key “a” or key “d” to change the background of the frames.

The entire code is given below.

Applications:

• Replacing the background during Online meetings

• Providing smooth and pleasant Video conferencing
• Helps to replace the inappropriate backgrounds for Virtual classes
• Gives additional affect to Video edits
• Live streaming

Conclusion :

Initially, The basic Image Processing is studied thereby Computer vision is studied, which is an interdisciplinary
scientific field that deals with how computers can gain high-level understanding from digital images or videos .From
the perspective of engineering, it seeks to understand and automatic tasks that the human visual system can do. On
getting a prior idea, moved to packages that are essential regarding Image processing to replace the background such
as cv2, Cvzone , media pipe. Using these packages, A program is achieved that can either replaces the background
with a color or any desired wallpaper. On summing up all these aspects a worthwhile communication is achieved.

Using real-time background remover real time video background replacement is achieved in an easy and effective
way using technologies such as OpenCV package and MediaPipe .This project is not just limiting to replace image
with a plain background , in addition to it one can use any plain color or other aesthetic background wallpapers.

Thereby Real-Time Background Remover project provides effective and worthwhile communication through video
conferencing, virtual meetings etc..
References:

• https://github.com/PeterL1n/BackgroundMattingV2

• https://data-flair.training/blogs/python-remove-image-background/