AI Based Personal Assistant

Dr. Babasaheb Ambedkar Technological University, Lonere.

Project Phase I Report

on
AI Based Personal Assistant
Submitted in the partial fulfillment of award of the degree of

BACHELOR OF TECHNOLOGY

Computer Science & Engineering

Submitted By
Mr. Mahesh Uttam Godage Roll No:06
Miss. Vaishnavi Amar Khatri Roll No:02
Mr. Kiran Prakash Kadam Roll No:07
Mr. Prashant Vaman Chavan Roll No:08

Under the guidance of

Prof. A. G. Mote

Department Of Computer Science & Engineering

Vidya Vikas Pratishthan Institute of Engineering & Technology,
Solapur
Academic Year 2022-23

VVPIET Solapur 1
 AI Based Personal Assistant

Vidya Vikas Pratishthan Institute of Engineering & Technology,

Solapur
(Affiliated Dr. Babasaheb Ambedkar Technological University, Lonere)

Certificate
This is to certify that, the project report entitled

“AI Based Personal Assistant”

Submitted By
Name PRN No.
1.Mahesh U Godage 1963211242039
2.Vaishnavi A Khatri 1963211242022
3.Kiran P Kadam 1963211242040
4.Prashnat V Chavan 1963211242041

As the partial fulfillment of award of

BACHELOR OF TECHNOLOGY
In
Computer Science & Engineering

Is a record of students’ own work, carried out by them during

academic year 2022-23 of B.TECH Semester VII

(Project Guide) (H O D) (Principal)

Prof. A. G. Mote Prof. V. V. Palmur Dr. Umesh S Mugale

VVPIET Solapur 2
 AI Based Personal Assistant

Certificate by Examiners

This is to certify that, the project report entitled

“AI Based Personal Assistant”

Submitted By

1.Mahesh U Godage 1963211242039

2.Vaishnavi A Khatri 1963211242022
3.Kiran P Kadam 1963211242040
4.Prashnat V Chavan 1963211242041

is examined on .

Internal Examiner External Examiner

VVPIET Solapur 3
 AI Based Personal Assistant

Declaration
We undersigned hereby declare that the project report entitled “AI Based Personal
Assistant” written and submitted under the guidance of Prof. A.G.Mote is our original
work. The empirical finding in this work is based in the data collected by ourselves. While
preparing this report, we have not copied from any another report.

We understand that any such copying is liable to be punished in a way that the
University Authority may deem fit.

Thank You,

Yours Faithfully,

Mahesh Godage
Vaishnavi Khatri
Kiran Kadam
Prashant Chavan

Date:-
Place:-Solapur.

VVPIET Solapur 4
 AI Based Personal Assistant

Acknowledgement
We have taken efforts for final report of this project. However, it would not have been
possible without the kind support and help of many individuals.

We deeply express our sincere gratitude to our Head of Department Prof. V. V. Palmur
for her valuable guidance & enlighten comments for this project report. It has been an altogether
different to work with her.

We gladly take this opportunity to thank the project coordinator Prof.A.G.Mote for
providing facilities during progress of project report. We would like to thank our Principal sir
Dr. Umesh S. Mugale who encouraged us and created a healthy environment for all of us to
learn in the possible way.

We are also thankful for all other faculty and staff members of our institute for their
cooperation.

VVPIET Solapur 5
 AI Based Personal Assistant

ABSTRACT
Abstract Requirements definition and management is recognized as a necessary step in the
delivery of successful systems and software projects, discipline is also required by standards,
regulations, and quality improvement initiatives. Creating and managing requirements is a
challenge of IT, systems and product development projects or indeed for any activity where
you have to manage a contractual relationship. Organization needs to effectively define and
manage requirements to ensure they are meeting needs of the customer, while proving
compliance and staying on the schedule and within budget. The impact of a poorly expressed
requirement can bring a business out of compliance or even cause injury or death.
Requirements definition and management is an activity that can deliver a high, fast return on
investment.
The “AI Based Personal Assistant” Computer vision-based application which provides
functionality like Voice Assistant, Virtual Mouse, Object Detection, Face Recognition. The
proposed system will allow the user to navigate the computer cursor using different hand
gestures. The proposed system uses nothing more than webcam, and microphone that acts as
a sensor and it can track the user’s hand and voice. The system will be implemented using the
python Technologies such as AI and NLP (Natural Language Processing). The hand gesture
is the most effortless and natural way of communication. The output of the camera will be
displayed on the monitor. And voice-based command will be captured by microphone and
processed using NLP. Also, this application gives functionalities like Face Detection for
Security Purposes.

VVPIET Solapur 6
 AI Based Personal Assistant

INDEX

Sr. No Title Page No.

1. Introduction 1

2. Literature Review 2

3. Methodology 3

4. Problem Statement 4

5. Scope 5

6. Modules

7. Technical Specification

8. System Design

8.1 Use Case Diagram

8.2 Data Flow Diagram

8.3 UML Diagram

9. Future Enhancement

10. Limitations

11. Conclusion

12. Reference

VVPIET Solapur 7
 AI Based Personal Assistant

1. INTRODUCTION

With the development technologies in the areas of augmented reality and the devices that we
use in daily life ,these devices are becoming compact in the form of wireless technologies.
Our project proposes an AI virtual mouse system that makes the use of hand gesture and hand
tip detection for for performing mouse functions and other functions like voice assistant, face
recognition along with it. In this with the use of AI virtual mouse system, we can track the
fingertip of the hand gesture by using built in camera or web camera and perform all this
operations.

The project is to develop a virtual mouse application that targets a few aspects like
voice assistant and face recognition along with it. It aims to eliminate the needs of having
physical mouse while able to interact with the computer system through webcam by using
various processing technique. Other than voice assistant and face recognition it aims to
develop a virtual mouse application that can be operational on all kind of surfaces and
operations.
Python programming language is used for developing virtual mouse, face recognition
system and voice assistant. OpenCV which is library for computer is used in the AI virtual
mouse system. We can employ several image processing algorithms to implement face
recognition.

It the virtual human mouse computer device that replaces physical mouse by using webcam
or any other image capturing devices that can be alternative to web cam or touch screen or
any other required physical devices. The AI assistant can understand the human natural
language and can complete the task for end user . this project also include automatic face
detection. We use the cam to recognize the faces in real time basically it is related to face
mapping of human faces unlike face verification.

VVPIET Solapur 8
 AI Based Personal Assistant

2. LITERATURE REVIEW

OpenCV (Open-Source Computer Vision Library) is an open-source computer vision and
machine learning software library. OpenCV was built to provide a common infrastructure for
computer vision applications and to accelerate the use of machine perception in the
commercial products. OpenCV was started at Intel in 1999 by Gary Bradsky, and the first
release came out in 2000. Vadim Pisarevsky joined Gary Bradsky to manage Intel's Russian
software OpenCV team. In 2005, OpenCV was used on Stanley, the vehicle that won the
2005 DARPA Grand Challenge.
Image processing is any form of signal processing for which the input is an image,
such as photographs or frames of video; the output of image processing can be either an
image or a set of characteristics or parameters related to the image. Most image-processing
techniques involve treating the image as a two-dimensional signal and applying standard
signal-
processing techniques to it. Image processing usually refers to digital image processing, but
optical and analog image processing are also possible. Here image processing is in the sense
that we are splitting each pixel of the image into RGB components.
Natural language processing (NLP) refers to the branch of computer science—and
more specifically, the branch of artificial intelligence (AI) concerned with giving computers
the ability to understand text and spoken words in much the same way human beings can.
NLP combines computational linguistics—rule-based modeling of human language—with
statistical, machine learning, and deep learning models. Together, these technologies enable
computers to process human language in the form of text or voice data and to ‘understand’ its
full meaning, complete with the speaker or writer’s intent and sentiment.

VVPIET Solapur 9
 AI Based Personal Assistant

3. METHODOLOGY

Voice assistant applications work based on Automatic Speech Recognition (ASR) system.
ASR systems record the speech and then break it down into phonemes, which are later get
processed into text. A phoneme (not words of syllables) is a basic unit of measurement for
human speech recognition. Phoneme recognition delivers better results than the process of
word decoding, as the last one tends to analyze word as a standalone unit ignoring the context
limits.
The methods used in face detection can be knowledge-based, feature-based, template
matching or appearance-based. Each has advantages and disadvantages: Knowledge-based, or
rule-based methods, describe a face based on rules. The challenge of this approach is the
difficulty of coming up with well-defined rules.
The AI virtual mouse system makes use of the transformational algorithm, and it
converts the co-ordinates of fingertip from the webcam screen to the computer window full
screen for controlling the mouse. When the hands are detected and when we find which
finger is up for performing the specific mouse function, a rectangular box is drawn with
respect to the computer window in the webcam region where we move throughout the
window using the mouse cursor

VVPIET Solapur 10
 AI Based Personal Assistant

4. Problem Statement

4.1 We are all aware about Cortana, Siri, Google Assistant, and many other virtual
assistants which are designed to aid the tasks of users in Windows, Android and
iOS platforms. But to our surprise, there is no such complete virtual assistant
available for Core Windows platform consisting of 70% of the users. So, this is a
major problem for users where there could be internet instability, server problems
and places where internet is not accessible.

4.2 Face recognition has many challenges due to illumination variations, large
dimensionality, uncontrolled environments, pose variations and aging. In the
recent years, Face recognition get remarkable improvement and accuracy to
overcome these challenges, but illumination change is still changing. an NIR
imaging system that gives satisfactory results for face recognition in illumination
variance conditions but it does not give good results when matching NIR image to
visible images. Unfortunately, all face images in the database are store in the
visible spectrum.

4.3 To design motion tracking mouse which detect finger movements gestures instead
of physical mouse. To design an application (.exe file) with user friendly user
interface which provides feature for accessing motion tracking mouse feature. The
camera should detect all the motions of hand and performs the operation of
mouse.

VVPIET Solapur 11
 AI Based Personal Assistant

5. SCOPE:

The future work will include implementation of additional gestures which can enable the
user to perform more functions with ease. The proposed system during this project uses only
the proper hand to perform gestures. Hence, improvement of the implemented technique in
future will be possible using both hands for performing different gesture movement.
 We can give more advanced virtual mouse including more facilities.
 Implement some extra features from time to time for enhancing the software quality
 For most laptop touchpad is not most comfortable and convenient.
 It is real time application known as virtual multitask mouse.
 User friendly application.
 This project removes the requirement of having physical devices.
 Voice assistants are expected to take more proactive role.
 More industries are are investing in facial recognition technology as way of
improving security ,creating a better user experience and reducing cost.

VVPIET Solapur 12
 AI Based Personal Assistant

6. MODULES

VVPIET Solapur 13
 AI Based Personal Assistant

7. TECHNICAL SPECIFICATION

Gesture Controlled Virtual Mouse makes human computer interaction simple by making use
of Hand Gestures. The computer requires almost no direct contact. All I/O operations can be
virtually controlled by using static and dynamic hand gestures.
 Machine Learning and Computer Vision algorithms to recognize hand gestures
 CNN implemented by MediaPipe running on top of pybind11
 Python 3.11. 0
 OpenCV – 4.5.5

SYSTEM REQUIREMENTS-
1. HARDWARE REQUIREMENTS:
 Quad core processor
 RAM: 8 GB
 System type: 64-bit OS, x64-based processor
 Webcam: 720p HD Webcam

2. SOFTWARE REQUIREMENTS
 Operating System: Windows 10
 Coding Language: Python 3.11.0
 Tool Kit: Image Acquisition and Image Processing
 IDE: PYCHARM

VVPIET Solapur 14
 AI Based Personal Assistant

8. SYATEM DESIGN
8.1 Use Case Diagram

8.2 Data Flow Diagram

VVPIET Solapur 15
 AI Based Personal Assistant

Fig. Data Flow Diagram for Voice Assistant

Fig. Data Flow Diagram for Virtual Mouse

VVPIET Solapur 16
 AI Based Personal Assistant

Fig. Data Flow Diagram for Face Recognition

VVPIET Solapur 17
 AI Based Personal Assistant

13. REFERENCES

[1] Abhilash S. S., Lisho Thomas, Naveen Wilson, Chaithanya C “Virtual Mouse Using
Virtual Mouse”, International Research Journal of Engineering and Technology
(IRJET) Volume: 05 Issue: 04 | Apr-2018
[2] Anadi Mishra, Sultan Faiji, Pragati Verma, Shyam Dwivedi, Rita Pal “Virtual Mouse
Using Hand Gesture”, Journal of Emerging Technologies and Innovative Research
(JETIR) 2022 JETIR April 2022, Volume 9, Issue 4
[3] Viral Doshi, Nishiket Bidawat, Ankit Sharma, Parth Sakariya “Gesture Controlled
Virtual Mouse”

VVPIET Solapur 18
 AI Based Personal Assistant

VVPIET Solapur 19