Voice Prescription

Developed By

Name of the Students

1 ARNAB CHAKRABORTY

2 ANGANA DAS

3 SOINDRILA DEB

4 AYANI BANERJEE

Under the Supervision of

Mr. TANMOY GHOSH

Assistant Professor, CSE

05.12.22
 Voice Prescription
A Dissertation Submitted in partial fulfillment for the
Degree of Bachelor of Technology (B.TECH), 7 th Semester in
Computer Science & Engineering
Submitted By
Name University University Roll
Registration Number
Number
ARNAB 011542-(2019-2020) 430119010064
CHAKRABORTY

ANGANA DAS 011696-(2019-2020) 430119020081

SOINDRILA DEB 003803-(2019-2020) 430119020091
AYANI BANERJEE 011408-(2019-2020) 430119020092

Under the Supervision of

Mr. TANMOY GHOSH

Assistant Professor, CSE

Narula Institute Of Technology

Maulana Abul Kalam Azad University of Technology

(05 December, 2022)
CERTIFICATE OF ORIGINALITY

The project entitled Voice Prescription has been carried out by ourselves in partial fulfillment
of the degree of Bachelor of Technology in Computer Science & Engineering of Narula
Institute of Technology, Agarpara, Kolkata under Maulana Abul Kalam Azad University of
Technology during the academic year 2022-23.

While developing this project no unfair means or illegal copies of software etc. have been
used and neither any part of this project nor any documentation have been submitted
elsewhere or copied as far in our knowledge.

Signature.:

Name.: ARNAB CHAKRABORTY

University Roll No.: 430119010064

University Registration No.: 011542-(2019-2020)

Signature.:

Name.: ANGANA DAS

University Roll No.: 430119020081

University Registration No.: 011696-(2019-2020)

Signature.:

Name.: SOINDRILA DEB

University Roll No.: 430119020091

University Registration No.:003803-(2019-2020)

Signature.:

Name.: AYANI BANERJEE

University Roll No.: 430119020092

University Registration No.: 011408-(2019-2020)

CERTIFICATE OF APPROVAL
This is to certify that the project entitled Voice Prescription has been carried out by ARNAB
CHAKRABORTY, ANGANA DAS, SOINDRILA DEB, AYANI BANERJEE, under my
supervision in partial fulfillment for the degree of Bachelor of Technology (B.TECH) in
Computer Science & Engineering of Narula Institute of Technology, Agarpara affiliated to
Maulana Abul Kalam Azad University of Technology during the academic year 2022-23.

It is understood that by this approval the undersigned do not necessarily endorse any of the
statements made or opinion expressed therein but approves it only for the purpose for which
it is submitted.
Submitted By:

Name: ARNAB CHAKRABORTY Name: ANGANA DAS

University Roll No.: 430119010064 University Roll No.: 430119020081

University Registration No: 011542-(2019-2020) University Registration No: 011696-(2019-2020)

Name: SOINDRILA DEB Name: AYANI BANERJEE

University Roll No.: 430119020091 University Roll No.: 430119020092

University Registration No:003803-(2019-2020) University Registration No: 011408-(2019-2020)

-------------------------- ------------------------------------

Mr. TANMOY GHOSH (External Examiner)

Designation,CSE

-----------------------------------------------------

(Dr. Subhram Das)

HOD, CSE Dept

 Acknowledgement

We would like to express our profound gratitude Dr. Subhram Das, HOD, CSE department,
for their contributions to the completion of my project titled Voice prescription.

We would like to express my special thanks to our mentor Mr. Tanmoy Ghosh for his time
and efforts he provided throughout the year. Your useful advice and suggestions were really
helpful to me during the project’s completion. In this aspect, we are eternally grateful to you.

We would like to acknowledge that this project was completed entirely by us and not by
someone else.
Contents Page No
1. Abstracts 1
2. Introduction 2
3. Survey / Broad observations 3
4. Objectives 4
5. Proposed approaches 5
6. Desirable Features 6
7. System Requirement Specification 7-8
8. 6.1 Identification of Need
9. 6.2 Technical Specification
10. 6.3 Cost Estimation
11. System Analysis 9-17
12. 7.1 Software Development Life Cycle (SDLC)
13. 7.2 Feasibility Study
14. 7.3 Data Flow Diagram (Level 0, Level 1, Level 2)
15. 7.4 Entity Relationship Diagram
16. 7.5 Data Dictionary
17. 7.6 Project Scheduling
18. 7.6.1 Milestones and Timelines
19. 7.6.2 Gantt Chart
20. 7.6.3 PERT Chart
21. Working Procedures /Implementation/System Design 18
22. Scope for future work 19
23. 22.Conclusion 20
24. 23.References 21
Abstracts

Doctors play a key role in the well-being of our society. And they continuously try to build a
better society and work toward patient security. But in this busy schedule doctors do lots of
struggles with the prescription. It may happen that for the same diseases they have to write
several copies with different data, which is tired-some work. Sometimes to upload a whole
prescription in an online portal after clicking pictures makes it more difficult. So, we are
come up with a solution where a doctor could able to dictate a prescription and software
going to record it with the help of the “voice recognition” technique. This software going to
work like a prescription manager for a doctor and can save all the documents for future
purposes. This also abstains the patient party from being misled. This software going to
generate a pdf once the doctor completed his work and then sends it for print. So, the
patient’s party could have two prescriptions at a time, one as a soft copy and another as a
hard copy. This software going to give a new edge to the medical field.

1
Chapter I: Introduction

“Health is wealth”- this is an evergreen line. Whenever we are talking about health and
human wellbeing doctors always played a key role in it. Whenever we fell sick, we used to
visit doctors and they prescribed us medicine and treat us further. But we all used to stuck
into our busy schedule and even we fell sick we don’t have enough time to visit a doctor. On
the other hand, sometimes, it also happens that we need to visit a doctor but we don’t have
much time in hand. In opposite now days doctor also do have a very busy schedule. And in
this schedule writing a wholesome prescription become tedious. As now a days online
consulting is also available and it is a very tiring work that to write a prescription and then to
upload it on the portal. It also happens that sometimes pictures are not clearly visible and
patients are easily miss leads. Sometime we also face problems like handwriting issues. So, to
deal with this problem we are introducing a new solution called “voice prescription”. In
general, we also can term it as a virtual prescription manager. By the virtue of voice
recognition technique one can deal with this problem. It may happen that doctor could dictate
a prescription and it stores automatically is a pre define or a desired format. And then this
format gets printed or uploaded in a portal.

2
Chapter II: Survey/Broad Observations

It is estimated that in 2021, 4.69 billion retail prescriptions will be filled throughout
India.According to the Institute of Medicine’s Preventing Medication Errors report, the
average hospitalized patient is subject to at least one medication error per day. This confirms
previous research findings that medication errors represent the most common patient safety
error.

More than 40 percent of medication errors are believed to result from inadequate
reconciliation in prescriptions during admission, transfer, and discharge of patients. Of these
errors, about 20 percent are believed to result in harm.

Many of these errors can be easily prevented if the handoffs in form of prescriptions are
available in an easy-to-understand, legible standard form. If such a format is achieved, the
patient will be able to easily understand exactly which medicine is required at which time.
Current prescription formats depend largely on guesswork on part of both the patient and the
chemists, as handwritten documents may vary greatly in legibility and layouts.

3
Chapter III: Objectives

A doctor will be able to dictate his prescription while talking to a patient virtually. To reduce
the effort of taking pictures of prescriptions and then uploading it on the portal and to reduce
inconveniences like blur pictures or unreadable formats or any other ambiguity we are
proposing a solution called voice prescription. Not only in virtual mode but a doctor can even
dictate his proposed prescription in physical mode also.

4
Chapter IV: Proposed Approaches

Most of the issues can be countered if a systematic and standard prescription is available. In
this paper, we try to provide an interface for the practitioners to prescribe orally and prepare
drug prescriptions in the form of a well-arranged table in compliance with the e-prescription
norms. Several approaches are possible for this: -

1. Usage of a software program to type out a prescription manually in a standard format

given by either the hospital chain or the clinic, or by the Medical Association of that
area. This would either be done by a doctor, or an assistant. While this is effective to
improve legibility, not only would it require additional resources on the part of the
doctor or the hospital chain, but it would also result in the familiar issue of lost hand-
offs and data.
2. Usage of a software program that takes the doctor’s utterances for data and has an au-
tonomous agent to create a standardized prescription in a quick and easy manner. This
is not only much faster but is also exponentially cheaper, secure, and convenient, es-
pecially if used with certain other techniques such as auto-mailing and digital signa-
ture for legitimacy.

In this paper, we will basically go on with the second approach due to its efficiency. This is
because there is a time-saving factor associated with voice-based data entry, eco-friendliness
due to the lack of requirement of paper, and doctor-patient confidentiality due to the
prescription being sent directly to the parties involved.

ADVANTAGES OF THE PROPOSED SYSTEM

 Saving time and energy by just speaking to the application rather than writing the pre-
scription on paper.
 Saving paper by introducing a digital prescription that can be viewed directly from the
document file.
 Security of the prescription and doctors’ notes by directly delivering the prescription
to the user’s phone and email.
 No problem with handwriting now and nhanced patient safety
 Decreased medication errors and Increased access to patient prescription records
 Improved pharmacy workflow multi-optional (i.e) either mail/SMS

5
Chapter V: Desirable Features

Throughout the project to try to keep it simple. As it is going to be a commercial software it

has to be easy to use. Key features of this project are

Flexibility: this software should work in every operating system like MAC,LINUX,
WINDOWS, etc.

Scalability: software should ably handle huge task at a time

Keyword: in this software there will be some pre define keyword that going to open some pre
built template which are been uploaded by the doctors.

Voice control able: once it is open doctors can control the whole system using his voice
instructions and while talking to a patient.

Dynamicity: this software also can be used without voice instructions. So that if a doctor
prefers to use it manually, they can.

Formatting: software will deliver all the prescription in a pdf format so that it does not lose its
authenticity. And could be print or upload easily.

Preservation: this software going to preserve all the prescriptions for references or security
purposes.

Security: once a prescription saved then it cannot be changed. So that it could escape any
disputation.

6
Chapter VI: System Requirement Specifications

6.1. Identification of Need

Drug side effects are the leading problem of death in the world, and drug or prescription
mistakes cause tens of thousands of mortalities worldwide each year. Most of these mistakes
are made by caregivers taking the incorrect medication or dosage due to obscure handwriting,
drug interactions, confusing medicine names, etc. Thus, the introduction of an alternate,
modern system is the need of the hour for the medical industry.

The introduction of speech perception applications can reduce some of these mistakes, as
prescription information can be obtained and heard through voice responses rather than
medical scripts. The objective is to avoid taking the incorrect medication for general diseases.
We need to bypass the wrong medication for common diseases viz. fever, cough, cold, body
pain, etc. by designing a virtual application on voice-based medicine prescription. By
implementing a voice-based medical alert system we need to reduce the problem of lost
prescriptions at the time of review. And we need to save paper by introducing digital
prescriptions that can be viewed directly from the document file.

6.2. Technical Specification

For the project, the following specification is required and recommended:

System requirements for Python Installation:

1. Operating system: Linux- Ubuntu 16.04 to 17.10, or Windows 7 to 10, with 2GB
RAM (4GB preferable)
2. You have to install Python 3.6 and related packages, please follow the installation in-
structions given below as per your operating system

System requirements for running the application:

1. Recording device or microphone with clear audio retention, preferably inline

Library and software dependencies for application:

7
1. Photoshop (template design and standard resolution)
2. Speech Recognition -> https://youtu.be/K_WbsFrPUCk
3. Python Packages:
 Tkinter GUI
 OpenCV
 Pillow
 Google text to speech

8
Chapter VII: System Analysis

7.1. Software Development Life Cycle (SDLC)

Although Python is easy to learn, building a successful voice recognizer application using
Google GTTS API requires not only software development skills, but also other skills like
understanding human factors for the telephone interface, linguistics, speech recognition, and
audio production.

The software development life cycle contains the steps for building voice applications. The
Unified Modelling Language (UML) was used to capture and model some of the
functionalities in the application. The UML is a visual language that provides a means to
visualize, construct, and document the artifacts of software systems.

9
7.2. Feasibility Study

The prime focus of the feasibility is evaluating the practicality of the proposed system
keeping in mind a number of factors. The following factors are taken into account before
deciding in favour of the new system:-

7.2.1. Economic Feasibility

Report generation in the proposed system is precise in that reports are generated as per user
requirements, which reduces the use of paper and manual labor. This means that the costs
while using this product are much lower when compared to using assistants for faster
prescription creation.

7.2.2. Technical feasibility

Keeping in view the above fact, nowadays all organizations are automating the repetitive and
monotonous work done by humans. The key process areas of the current system are nicely
amenable to automation and hence the technical feasibility is proved beyond doubt.

7.2.3. Operational Feasibility

The present system has automated most of the manual tasks. Therefore, the proposed system
will increase the operational efficiency of the doctors. Additionally, since the software is
easy-to-use and automatically sends handoffs to the patients, the efficiency increases
exponentially.

10
7.3. Data Flow Diagram

The flow of data of a system or a process is represented by DFD. The DFD belongs to
structured-analysis modelling tools It also gives insight into the inputs and outputs of each
entity and the process itself. DFD does not have control flow and no loops or decision rules
are present.

7.3.1. Level 0

Medicine Care Advice

Symptoms Problems
Doctor Diagnosis Voice Patient
Prescription Schedule
Tests System Feedback

It is also known as a context diagram. In the above Level 0 DFD the central process is the
Voice Prescription System which takes input from the entity called Doctor, which include the
various symptoms, possible diagnosis and effective medicine and gives and out to the entity
called Patient which receives information regarding appointment scheduling, care taking and
gives the information of signs and symptoms to the system.

11
7.3.2. Level 1

Start Command
Query Patient
Doctor Keyword Voice
Recognition
Investigative Tests Prescription
Keyword

Query Identifying
Medicine Data Store Data Prescription
Data Medicine Generation

In 1-level DFD, the context diagram is decomposed into multiple bubbles/processes. In this
level, we highlight the main functions of the system and breakdown the high-level process of
0-level DFD into subprocesses. In the above Level 1 Data Flow Diagram the Voice
Prescription System has the processes of Voice Recognition, Identify Medicine and
Prescription Generation with data store for medicine and various query requests.

12
7.3.3. Level 2

Feedback
Keyword Voice Patient
Doctor
Recognition Patient ID Patient ID
Patient Details
Data Medicine
Query
Instruction
Investigation
Report Patient Data

Query
Query
Medicine Data Identifying Data
Data medicine

Data

Report Prescription Data Pharmacist

Test
Generation

2-level DFD goes one step deeper into parts of 1-level DFD. It can be used to plan or record
the specific/necessary detail about the system’s functioning. The DFD is further broken into
components such as Pharmacist and Tests.

13
 7.4. Entity Relationship Diagram

Age pt_History chk_Date serial_No name_Pt

dr_id
Dr_id serial_No
serial_no

Dr_name
dr_Name Doctor’sRecord
Doctor’s record

Record of Patient
Record of Patient Treatment
Treats
dept_Id
dept_id Belongs
Belongs
Have
Have patient_Id
Patient_id

Date
Date serial_No
serial_no Department
Department

Prescription details Dept_name

dept_name Hospital_id
hospital_Id
Patient Details

prescription _Date
Prescription_id prescription_no
prescription _No

Entity Relationship Diagram is a diagram that displays the relationship of entity sets stored in
a database. In other words, ER diagrams help to explain the logical structure of databases.
The above diagram shows the database that stores the information regarding the doctor,
patient, prescription created and the appointment details.

14
7.5. Data Dictionary

A data dictionary contains metadata i.e data about the database. As per the Entity Relation
Diagram these are the data contained in the database of this project:

Age: This attribute contains the age of the patient in Record of Patient table.

pt_History: This attribute contains patient history i.e. the previous known symptoms of the
patient as known by the system.

chk_Date: This date is the scheduled date for the appointment with the doctor

serial_No: This is the serial number that is the primary key of the Record of Patient table.

name_Pt: This attribute contains the name of the patient.

dr_id: This is the id that that contains the unique identification number of the doctor
registered. This is also the primary key for the Doctor’s Record table.

Record of Patient: This is the table that contains all the information of the patients.

dr_Name: This contains the name of the doctor that are registered into the system.

Doctor’s Record: This table contains the information regarding of the doctor.

patient_Id: This is the id that that contains the unique identification number of the patient
registered. This is also the primary key for the Patient Details table.

dept_Id: This is the primary key of the Department table. This is the identification number of
the departments of the hospital.

Date: This contains the details of the scheduled appointment done previously.

Patient Details: This table contains the detail of the patient. The primary key of this table is
prescription_No

prescription _Date: This attribute contains the date of creation of the prescription.

prescription _No: This is the unique identification of the prescription registered every time
one is created.

dept_name: This attribute contains the department name of the hospitals.

15
 7.6. Project Scheduling

Scheduling in project management is the listing of activities, deliverables, and milestones

within a project. A schedule also usually includes a planned start and finish date, duration,
and resources assigned to each activity. This project has been scheduled using visual aids
such as gnatt chart and PERT chart which help view its progress very closely starting from
the very base to the top most layer of this application.

7.6.1. Milestones and Timelines

Planning Testing Security

complete. complete. assurance
Design Security
Analysis of complete.
complete. assurance
requirement Product is in
Coding starts from
s starts. beta phase
starts. here.
Milestone 1 now. Major
Milestone Milestone 4
reached. milestone 2
3 reached. reached. 21st
14th Jan 23 reached. 24th
1 Feb 23 March 23 March 23

Planning Analysis Coding All bugs

starts for complete. complete. expectedly
our Designing Major ironed out.
project starts milestone 1 Product is now
Milestone complete. released and
1st Jan 23 2 reached. First usable completely
24th Jan 23 deliverable usable. 9th Apr
made. 18th 23
March 23

16
7.6.2. Gantt Chart

January 23 February 23 March 23 April 23

Planning Task 1
Analysis Task 2
Design Task 3
Coding Task 4
Testing Task 5
Security Task 6
Maintenance Task 7

7.6.3. PERT Chart

TASKS ACTIVITIES DAYS DEPENDENCIES

1 Planning 14 -
2 Analysis 10 Task 1
3 Design 8 Task 2
4 Coding 45 Task 3
5 Testing 3 Task 4
6 Security 3 Task 5
7 Maintenance 22 Task 4

Start Analysis Testing End

Planning Coding Maintenance

Design Security

17
Chapter VIII: Working Procedure/System Design

The doctor initiates the application by a button prompt or a trigger and starts dictating the
prescription. The application records the speech with the help of Google Speech API as a
recognizer (using the GTTS engine and Python speech recognition library) and fills the form
containing various categories such as Patient details, diagnosis, medication, etc. A PDF
document of the EHR (Electronic Health Record) will be generated which is compliant with
HIPAA (Health Insurance Portability and Accountability Act) using OpenCV and Pillow.

The format will either be standard to the hospital or clinic chain in question, or it’ll be
customized by the doctor or a graphic designer using photo editing software such as Adobe
Photoshop. This image will then be automatically populated with the relevant medical
information using the OpenCV framework used in the program.

The doctor will utter certain designated “keywords” which will be used to classify the
information based on various internal parameters such as patient information, medicine name,
disease name, timings for medication, doctor names, etc.

This populated image is then passed through the Pillow framework for Python, in order to
convert it into the desired format, i.e., PDF. This format allows easy reading, and quicker
access and transfer.

This transfer is proposed to take place using the integration of SMTPlib. This will ensure that
a copy of the digitized prescription is duly sent to the patient’s email address, which will be
entered during the consultation. This ensures that the patient always retains the medication
information and loss of prescription is a negligible risk.

18
Chapter IX: Scope for future work

In the near future, the devs are planning to integrate and use the system in the real hospital
ecosystem to test and validate the implementation and to analyze the impact it will create in
the healthcare domain.

Additionally, newer features can be implemented in the application to improve its efficiency
and make it more of an all-in-one solution. For example, a companion application can be
created to display the prescription in a simple format. Also, additional metadata can be
implemented in the PDF prescription so that date-time, location and digital signatures are
visible in the companion application. It can also be programmed to set alarms and reminders
according to the timings given in the e-prescription.

Lastly, we can make it cross-platform so that even the ubiquitous smartphone can be used as
an inlet for the data given by the doctor. Due to the nature of Python used in its development,
the software can be easily ported to any given operating system in existence. It can be made
to work on any device with a good enough microphone.

19
Chapter X: Conclusion

We intend to build a software which will work upon “voice recognition” system and provide
a elite service to the society. This project has a greater future scope to lead in the medical
field. Not only that in future it will be a demanding software as it is going to make tedious
work easier. It also provides security which makes it more attractive as a service software.
Moreover, it is built with the most common object-oriented programming language called
python. Which makes this software simple and easy to use.

20
References

We intend to build a software which will work upon “voice recognition” system and provide
an elite service to the society. This project has a greater future scope to lead in the medical
field. Not only that in future it will be a demanding software as it is going to make tedious
work easier. It also provides security which makes it more attractive as a service software.
Moreover, it is built with the most common object-oriented programming language called
python. Which makes this software simple and easy to use. The websites and research papers
that helped us build this project are:

 Voice based system and its implementation details and probable obstruction: “Voice
 Based Medicine Prescription” by Randhir Jagannath Patil and Dr. S.A.Pardeshi
 Information regarding electronic prescription system database implementation: “An
 Electronic Prescription System powered” by Speech Recognition by Jitender
 Mahatpur and Dr.Mahesh Motwani
 Implementation purpose: https://www.googleadservices.com/pagead/
 Details regarding the working of the voice recognition system and its application.:
 https://codelabs.developers.google.com/codelabs/cloud-speech-text-python3#0
 Details of healthcare and information regarding prescription keeping system:
 https://www.sciencedirect.com/topics/computer-science/prescription-system
 Medical and Biomedical details and its storage system and application:
 https://bmcmedinformdecismak.biomedcentral.com/articles/10.1186/s12911-022-
 01948-w
 For survey and observation for the project:
 https://www.statista.com/statistics/261303/total-number-of-retail-prescriptionsfilled
annually-in-the-us/ :
 Reference for medical errors and how to prevent them in a system:
 9. Patient safety and quality in system:
https://www.ncbi.nlm.nih.gov/books/NBK2648/

21