Professional Documents
Culture Documents
BACHELOR OF TECHNOLOGY
in
Computer Science and Engineering
by
VANKADARA ANUSHA
18K91A05P4
I, Ms.V. Anusha bearing Hall Ticket Number: 18K91A05P4, hereby declare that the main
project report titled PERFORMANCE EVALUATION OF RELATIONAL AND NON-
RELATIONAL DATABASES ON A RASPBERRY PI under the guidance of MS.K.MOUNIKA,
Assistant Professor in Department of Computer Science and Engineering is submitted in par-
tial fulfillment of the requirements for the award of the degree of Bachelor of Technology in
Computer Science and Engineering.
Vankadara Anusha
18K91A05P4
Place: Meerpet
Date:
CERTIFICATE
This is to certify that the main project report entitled PERFORMANCE EVALUATION
OF RELATIONAL AND NON-RELATIONAL DATABASES ON A RASPBERRY PI,
being submitted by Ms.Vankadara Anusha, bearing ROLL.NO:18K91A05P4 in partial ful-
fillment of requirements for the award of degree of Bachelor of Technology in Computer
Science and Engineering, to the TKR College of Engineering and Technology is a record of
bonafide work carried out by her under my guidance and supervision.
Place: Meerpet
Date:
TABLE OF CONTENTS
ABSTRACT i
ACKNOWLEDGEMENTS ii
LIST OF FIGURES iii
LIST OF TABLES iv
LIST OF SYMBOLS AND ABBREVIATIONS v
1 INTRODUCTION 1
1.1 Motivation 1
1.2 Problem definition 1
1.3 Limitations of existing system 2
1.4 Proposed system 2
2 LITERATURE REVIEW 4
2.1 Review of Literature 4
3 REQUIREMENTS ANALYSIS 8
3.1 Functional Requirements 8
3.2 Non-Functional Requirements 8
3.3 Software Requirement specifications 9
3.3.1 Software requirements 9
3.3.2 Hardware Requirements 13
4 DESIGN 16
4.1 DFDs and UML diagrams 16
4.1.1 Sequence diagram 16
4.1.2 Activity diagram 17
4.1.3 Data Flow diagram 17
4.2 Algorithm 18
5 CODING 20
5.1 Pseudo Code 20
8 CONCLUSION 46
REFERENCES 46
ABSTRACT
The goal of this project is to compare and contrast the performance of two widely used
Database Management System technologies: SQL and NoSQL, in order to figure out which is
best for which types of query executions.
The project’s major purpose is to create a guide for application developers who are
working on apps that rely heavily on databases and where data retrieval time is critical. Both
DBMS technologies will be tested under various scenarios and the results will be analysed in
order to attain this aim.
By keeping the same system environment on Raspberry Pi, the measurement will be
done for different sorts of queries for both DBMS(s). The query execution time and memory
space requirements will be used as benchmarks. The limitations will be to keep the same
hardware configurations and system loads, i.e., the same number of records in both DBMSs
(s).
In this project, we used Mockaroo.com to create a test database, since it is not possible
to manually create a large database, for example, 50,000 or more databases, in a short time to
determine the database performance.
i
ACKNOWLEDGEMENTS
The satisfaction and euphoria that accompanies the successful completion of any task
would be incomplete without the mention of the people who made it possible and whose en-
couragement and guidance have crowned my efforts with success.
I am also indebted to the Head of the Department, Dr. A. Suresh Rao, Professor,
Computer Science and Engineering, TKR College of Engineering and Technology, for his
support and guidance throughout my Thesis/Dissertation.
I extend my deep sense of gratitude to the Principal, Dr. D. V. Ravi Shankar, TKR
College of Engineering and Technology, for permitting me to undertake this Thesis/Dissertation.
Finally, I express my thanks to one and all that have helped me in successfully com-
pleting this Thesis/Dissertation. Furthermore, I would like to thank my family and friends for
their moral support and encouragement
Vankadara Anusha
18K91A05P4
Place: Meerpet
Date:
ii
LIST OF FIGURES
iii
LIST OF TABLES
iv
LIST OF SYMBOLS AND ABBREVIATIONS
v
Chapter 1
INTRODUCTION
1.1 Motivation
Because the apps of developers must run quickly, it is vital to choose a database manage-
ment system (DBMS) for each application. This choice should be based on data that compares
various database management systems based on a variety of performance metrics. Different
apps have their own set of functionalities. For the development of their product, application
developers must select acceptable technology. This project will assist them in making a de-
cision about which database management system to choose. Under various constraints, the
project evaluates two distinct database management systems based on some performance pa-
rameters.
The major purpose of this research is to compare and evaluate two common database
management systems. One type of database is SQL, while the other is NoSQL. Both tech-
nologies are being used.
Memory Requirements and Execution Time for conducting various sorts of queries are
the measurement factors. The identical database schema was used to assess both systems.. By
gradually expanding the quantity of records in the database, the same sets of queries have been
analysed. The impact of increased size on performance has also been assessed and compared.
For ease of comparison, performance data for both technologies is provided on line graphs.
The intended audience for this paper is application developers who work in develop-
ment settings where a database serves as a backend and performance is critical. They can
assess project data as well as the conclusion to select which database management system to
use.
1
As relational databases, SQLite and PostgreSQL are used, and MongoDB is used as
a non-relational database. CRUD operations are executed for various number of records and
performance time is calculated. The comparison is presented as a line graph.
2
•the outcomes are simple to comprehend for working-class people.
3
Chapter 2
LITERATURE REVIEW
4
write key-value pairs 4) It’s time to get rid of the key-value pairs. 5) It’s time to collect all
of the keys. The experiments employed anywhere from ten to one hundred thousand records.
In terms of read, insert, and delete operations, Couchbase and MongoDB surpassed all other
databases, according to the findings.
Boicea et al. (2) conducted a comparison of MongoDB and Oracle to have a compre-
hensive understanding of the performance differences between SQL and NoSQL databases.
The databases are tested in three different ways: 1) time taken to insert data, 2) time taken to
update data, and 3) time taken to delete data. MongoDB emerged as the obvious victor in all
operations, according to the findings of the experiments. The amount of records was anything
between ten and a million.
Konstantinou et al (7) examined Cassandra, HBase, and Riak in terms of read and
update operations in order to undertake a study on the elasticity of non-relational databases.
Cassandra is best suited for write operations, while Hbase has stronger elasticity and is better
suited for read operations, according to the study. Riak, on the other hand, does not show any
improvement in performance regardless of the operations.
The failover features of several NoSQL databases, such as Aerospike, Cassandra,
Couchbase, and MongoDB, were explored by Nelubin and colleagues (11) . MongoDB has the
least favourable downtime, followed by Cassandra and Aerospike, according to the findings..
The study’s disadvantage was that the datasets employed for the research were RAM-only
datasets, which made it unsuitable for real-world circumstances. MongoDB was also found to
be unsuitable for highly available systems, according to the study.
Van der Veen et al (13) evaluated Cassandra, MongoDB, and PostgreSQL to see which
database performs the best on single and distributed servers. Cassandra performs best in a
distributed server and MongoDB in a single server
Loureno and colleagues (10) examined a variety of NoSQL databases on the market,
including MongoDB, Cassandra, and HBase. They compared them in terms of the consis-
tency and longevity of the data they saved, as well as performance and scalability. They came
to the conclusion that the MongoDB database might be the SQL database’s successor since it
delivers strong data stability and consistency. Cassandra is the best option when the majority
of the operations are database writes.
5
Chandra (3) contrasts NoSQL databases’ BASE (Basically Available, Soft State, Even-
tual Consistency) traits to the ACID (Atomicity, Consistency, Isolation, and Durability) fea-
tures in his work. He also investigates which databases are best suited for specific applications,
concluding that relational databases are the best option in financial applications. NoSQL tech-
nologies have proven to be superior for data analysis and data mining.
Choi et al (4) compared Oracle and MongoDB performance. They observed that Mon-
goDB outperforms Oracle in terms of speed. Boicea et al. [6] compared the same database.
The authors assert that MongoDB is faster and easier to maintain at the end of their paper.
Oracle, on the other hand, is the preferable option when it comes to mapping complicated data
relationships.
Hecht and Jablonski (6) provided a use-case-based research of NoSQL databases. They
recognise the challenges of selecting a NoSQL database that is appropriate for a certain use
case and devote their work to addressing this issue. The data model, query functionality, par-
titioning, replication, and concurrency controls serve as the foundation for their comparison.
In this light, they examine 14 NoSQL databases, including MongoDB, CouchDB, Cassandra,
and HBase. Boicea et al compare a NoSQL database against a SQL database. They picked
Oracle for the SQL implementation and MongoDB for the NoSQL implementation. They be-
lieve that when dealing with a large number of records, insertion time, as well as update and
deletion times, are more important with Oracle.
While NoSQL databases are more scalable and speedier than traditional relational
databases, they have a number of drawbacks. These issues are outlined by Leavitt (8) . He
points out that, while NoSQL databases are quick for simple activities, they take a long time
to perform complex jobs. Furthermore, complex queries might be difficult to articulate. The
lack of native consistency support is the other disadvantage. According to Leavitt, many busi-
nesses have yet to grasp NoSQL, and there is a scarcity of support and management tools to
assist them. Bartholomew provides an overview of the history of SQL and NoSQL databases,
as well as the contrasts between them. For cloud-based platforms, Sakr et al. examine data
management solutions, including NoSQL. They talk about the problems that data management
software has to deal with in the real world.
Tiwari (9) gives a thorough overview of NoSQL databases. He discusses the imple-
6
mentations’ history, logic, programmability, storage architecture, and performance tweaking.
Indrawan-Santiago analyses the data model, transaction model, ad-hoc query support, index-
ing, shrading, and licencing type as well. She examines eleven NoSQL implementations,
including Cassandra, HBase, Dynamo, MongoDB, and CouchDB, on this foundation. She
also does a qualitative comparison of relational and NoSQL databases, indicating that NoSQL
databases are more likely to complement relational databases and increase an organization’s
database management skills.
7
Chapter 3
REQUIREMENTS ANALYSIS
Security:
We’ve created a username and password for accessing the Raspberry Pi. As a result, with-
out the login and password, no one can access the Raspberry Pi and its code.
Usability:
The application is easy to use. Select, insert, update, and delete buttons are included in the
web interface. End users can easily view the comparison graphs just by clicking the buttons.
8
Reliability:
Because of the qualities inherited from the chosen platform, python, the system is more
reliable. The python-based code is more reliable. The databases used are SQLite, PostgreSQL
and MongoDB.
Supportability:
The system is designed to run on a range of operating systems. The system works with
a variety of hardware and software platforms. Raspberry pi can be easily connected to any
hardware. As the databases required are already installed in the raspberry pi ,it can be easily
connected to any hardware and we can run the code.
Performance:
This system is being developed in high-level languages and will respond to the end user
on the client system in a very short period using powerful front-end and back-end technologies.
Maintainability:
As the test data is created using free websites and by using simple hardware the application
is cost-effective. It can be readily implemented in any scenario by just attaching the Raspberry
Pi to the hardware.
9
provides a number of benefits. It is significantly faster, has more recent software, and will be
much more stable in the near future.
Raspbian is a Debian-based free operating system created exclusively for the Rasp-
berry Pi. An operating system is a collection of programmes and tools that enable your Rasp-
berry Pi to function. Raspbian, on the other hand, is more than just an operating system:
it comes with over 35,000 packages, which are pre-compiled software packages packed in
an easy-to-install format for your Raspberry Pi. The initial build of almost 35,000 Raspbian
packages optimised for the Raspberry Pi was finished in June of 2012. Raspbian, on the other
hand, is continuing in development, with the objective of improving the stability and perfor-
mance of as many Debian programmes as possible.
2.Relational Databases
i. SQLite
SQLite is an in-process library that implements a serverless, zero-configuration trans-
actional SQL database engine. It’s a zero-configuration database, which means you won’t
have to configure it in your system like you would with other databases. The SQLite engine,
unlike other databases, is not a stand-alone process. Depending on your needs, it can be con-
nected statically or dynamically to your application. SQLite has immediate access to its data
files.
• To run SQLite, you don’t need a separate server process or system (serverless).
• SQLite is zero-configuration, which implies it requires no setup or management.
• An full SQLite database is stored in a single cross-platform disc file.
• SQLite is a compact and lightweight database, weighing in at less than 400 kilobits when
fully configured and less than 250 kilobits when extra features are disabled.
• SQLite is self-contained, meaning it does not require any external dependencies.
• SQLite transactions are completely ACID-compliant, which means that several processes or
threads can securely access the database.
• SQLite supports the majority of the SQL92 (SQL2) query language features.
• SQLite is written in ANSI-C and has a straightforward API.
• SQLite is a database management system that runs on UNIX (Linux, Mac OS X, Android,
10
and iOS) and Windows (Win32, WinCE, WinRT).
ii. PostgreSQL
PostgreSQL is a powerful, enterprise-class, and open-source relational database sys-
tem. SQL (relational) and JSON (non-relational) querying are both supported by PostgreSQL.
It is a very stable database that has been developed for over 20 years by the open-source com-
munity. Many web apps, as well as mobile and analytics applications, use PostgreSQL as their
primary database. Extensibility is built into PostgreSQL. You can define your own data types,
index types, functional languages, and so on in PostgreSQL. If there is something about the
system that you don’t like, you may always create a custom plugin to improve it to match your
needs, such as adding a new optimizer.
PostgreSQL has a number of sophisticated capabilities that aren’t found in other enterprise-
class database management systems, including:
• User-defined types
• Table inheritance
• Sophisticated locking mechanism
• Foreign key referential integrity
• Views, rules, subquery
• Transactions that are nested (savepoints)
• Multi-version concurrency control (MVCC)
• Replication that is asynchronous
3. Non-relational databases
i. MongoDB
MongoDB is a document-oriented database with high performance, high availability,
and simple scaling that operates on a number of systems. MongoDB is built on the princi-
ples of collection and document. MongoDB is a scalable, flexible NoSQL document database
platform that was designed to overcome the limitations of earlier NoSQL solutions as well as
the approach of relational databases. The horizontal scaling and load balancing features of
MongoDB have provided app developers with unparalleled flexibility and scalability.
• The purpose of MongoDB was to increase developer productivity and flexibility. It’s a
11
document-oriented database, which implies that data is saved in the form of documents, which
are then organised into collections. Because documents are self-contained and can be handled
as objects, the document model is much more natural for developers to work with.
• MongoDB’s sharding enables for far more horizontal scalability. Sharding is the breaking of
huge datasets into numerous dispersed instances, or ”shards.”
• In MongoDB, authentication is a vital security elementAuthentication ensures that only au-
thorised users have access to the database. Your data can be accessed by anyone without
authentication. Users can access the database using a variety of authentication procedures
provided by MongoDB. The default, Salted Challenge Response Authentication Mechanism
(SCRAM), is the most often used. SCRAM needs the user to give an authentication database,
username, and password before it can be employed.
• MongoDB has a number of capabilities that can assist you in managing time series data.
• MongoDB indexes can be generated on the fly to support real-time, dynamic query patterns
and application requirements. They can also be declared on any field, even those nested within
arrays, in any of your documents.
• Database triggers are a significant feature in MongoDB Atlas that allows you to run code
when certain events in your database occur. MongoDB Atlas makes it simple and intuitive to
design and manage triggers. The Atlas UI allows you to manage your triggers.
4. Flask
Flask is a microweb framework written in Python. Because it does not need the use
of any specific tools or libraries, it is referred to as a microframework. It lacks a database
abstraction layer, form validation, and other components that rely on third-party libraries to
accomplish common tasks. Extensions, on the other hand, may be used to extend the func-
tionality of an application as if it were built into Flask itself. Extensions are available for
object-relational mappers, form validation, upload handling, several open authentication pro-
tocols, and other framework-related features. Flask is a web application framework that is
used to construct backends.
5. Python
12
Python is a high-level, interpreted, interactive, and object-oriented scripting language.
Python is designed to be an easy-to-understand programming language. It has fewer syntacti-
cal features than other languages and often employs English terminology instead of punctua-
tion.
• Python may be used to create web applications on a server.
• Python may be used in combination with other applications to create processes.
• Python has the ability to communicate with databases. It can read and write files as well.
• Python may be used to tackle complicated issues and work with big volumes of data.
• Python can be used to produce production-ready software or for fast prototyping.
• Python can run on a number of operating systems (Windows, Mac, Linux, Raspberry Pi, etc).
• Python’s basic syntax is comparable to that of English.
• Python’s syntax enables programmers to write programs in less lines than other program-
ming languages.
• Python is an interpreter language, which implies that code may be run right away. As a
result, prototyping can be done relatively quickly.
6. VNC Viewer
VNC Viewer is used to operate local PCs and mobile devices from a remote location.
VNC Viewer is a piece of software that may be installed on a computer, tablet, or smartphone
and can access and control a machine at another location. In this project VNC viewer is used
to connect Raspberry Pi to laptop
7. Mockaroo
Mockaroo website is used to create test data for both SQL and NoSQL databases
13
card-sized processor that teaches people of all ages about operating systems and programming
languages such as Scratch and Python. The Raspberry Pi was designed by the Raspberry Pi
Foundation to provide a low-cost platform for learning and exploring computer programming.
Many of the same things that a standard desktop computer can accomplish, such as word pro-
cessing, spreadsheets, high-definition video, gaming, and programming, are available on the
Raspberry Pi. The board’s four USB ports can be used to connect USB devices such as key-
boards and mice. The Raspberry Pi will come in two flavours: Model A and Model B. Model
A is less expensive and has fewer connectors than Model B.
In this application, we’ll be utilising a Raspberry Pi 3 Model B+. The Raspberry Pi
3 Model B+ is a more powerful version of the Raspberry Pi 3 Model B from the previous
year. A 1.4 GHz quad-core ARMv8 64bit processor and a powerful Video Core IV GPU are
included in the BCM2837B0 system-on-chip (SoC). The Raspberry Pi can run a variety of
ARM GNU/Linux distributions. In comparison to the Raspberry Pi 3 Model B, the Raspberry
Pi 3 Model B+ has a higher CPU clock speed (1.4 GHz against 1.2 GHz), enhanced Ethernet
throughput, and dual-band WiFi.
14
Fig. 3.2 Raspberry Pi backview
2. SD Card
Because there is no internal memory on the Raspberry Pi, you must boot it from an SD
card.
3. USB Charger
Pi works only when there is a constant power supply. Here we’ll be using Type-B
charger
4. HDMI Cable
It is the most widely utilised HD signal for transporting both high-definition audio and
video from one device to another over a single connection.
15
Chapter 4
DESIGN
16
4.1.2 Activity diagram
17
The purpose of a DFD is to show the scope and boundaries of a system as a whole. It
may be used as a communication tool between a system analyst and everyone else engaged in
the order, and it can also be utilised as a starting point for system modification. The DFD is
also known as a data flow graph or bubble chart.
The DFD graphically represents the functions, or processes, that capture, alter, store,
and distribute data between a system and its environment, as well as between system compo-
nents. It’s a useful communication tool between the user and the system designer because of
the visual depiction. You may start with a general overview and work your way down to a
hierarchy of individual diagrams thanks to DFD’s structure.
4.2 Algorithm
1. First start PostgreSQL in Raspberry Pi. This can be done using command “docker start
postgres”
18
2. Go to location where app.py is located and run it. Because it is a server with which the user
can interact with.
3. Go to browser and search http://localhost:5000 .Then the web interface opens with four
buttons UPDATE, DELETE, WRITE and SELECT.
4. Click on WRITE button in the webpage. When we click on WRITE it performs all the
operations.
5. Now, in the project folder, execute the main.py file.
6. The select, insert, update and delete operations takes place for the three databases -SQLite,
PostgreSQL and MongoDB for various number of records.
7. Go to the webpage after you’ve completed all of the processes.
8. We can see the comparison graph for three databases with number of records on Y-axis and
performance time on X-axis
19
Chapter 5
CODING
PROCEDURE index:
IF request.method = ’POST’
THEN
OUTPUT request.data.decode(’utf-8’)
return ”OK”
ENDIF
return rendertemplate(’plotPage.html’)
ENDPROCEDURE
@app.route(’/start’)
PROCEDURE startoperation:
optype¡-request.args.get(”type”).upper()
IF optype in app.config[”ALLOWEDOPERATIONS”]
THEN
id¡-randomidgenerator()
conn¡-sqlite3.connect(’bg.sqlite’)
cursor¡-conn.cursor()
create bgtask table if not exists
THEN
ENDIF
Insert taskid,tasktype into bgtask
conn.close()
return ”queueId”: id
20
ENDIF
return ’errorMsg’
ENDPROCEDURE
@app.route(”/check”)
PROCEDURE checkstatus:
Request taskid
IF taskid
THEN
Connect bg.sqlite
SELECT taskstatus, taskresult from bgtask where taskid=’?’
OUTPUT data
IF data and data[0] = ”completed”
THEN
LOAD (data[1]) from json
return ”ok”: True, ”data”: jsondata
ENDIF
ENDIF
return ”ok”: False
ENDPROCEDURE
IF name = ’main’
THEN
app.run(host=”0.0.0.0”,debug=True)
ENDIF
main.py
PROCEDURE dbconnect:
Connect bg.sqlite
cursor¡-conn.cursor()
return cursor, conn
ENDPROCEDURE
PROCEDURE getpendingtask:
21
Connect database
select id,taskid from bgtask where taskstatus=’pending’
latestid.sort(reverse=True)
OUTPUT latestid
taskid¡-””
IF len(latestid)
THEN
taskid¡-latestid[0][1]
ENDIF
conn.close()
return taskid
ENDPROCEDURE
FUNCTION updatetaskstatus(taskid, data):
RETURNS
OUTPUT ”updating task status”
Connect database
update bgtask set taskstatus=?, taskresult=? where taskid=?”
conn.close()
ENDFUNCTION
FUNCTION stringifyresult(data):
RETURNS
return json.dumps(data)
ENDFUNCTION
PROCEDURE runsql:
data¡-psql.runtask()
return data
ENDPROCEDURE
PROCEDURE runnosql:
data¡-mongodb.runtask()
return data
22
ENDPROCEDURE
PROCEDURE main:
ENDPROCEDURE
Get pending task
OUTPUT taskid
IF taskid
THEN
sqlresult¡-runsql()
nosqlresult¡-runnosql()
data¡-
update sql result
update nosql result
data¡-stringifyresult(data)
update taskstatus(taskid, data)
ELSE
OUTPUT ”no new task found”
ENDIF
IF name = ”main”
THEN
OUTPUT main()
ENDIF
mongodb.py
@timer
FUNCTION insert(data, noofdoc):
RETURNS
docs¡-noofdoc
OUTPUT docs
FOR 0 TO 1,docs+1
count¡-randomidgenerator()
FOR x in data:
23
IF (x.get(”id”))
THEN
x[”id”] ¡-count + str(x[”id”])
ELSE
x[”id”] ¡-count + str(x[”id”])
ENDIF
db.fake.insertone(x)
NEXT
ENDFUNCTION
@timer
FUNCTION select(noofdocs):
RETURNS
Count estimated fake data
OUTPUT count
Get random row count
FOR x 0 TO noofdocs//4
Get random row count
Return id
NEXT
ENDFUNCTION
@timer
FUNCTION update(noofdocs):
RETURNS
Count fake document count
FOR x 0 TO noofdocs//4
gender= getrandomgender()
randomrow¡-getrandomrow(count)
filter¡-”id”: randomrow
update random gender
NEXT
24
ENDFUNCTION
@timer
FUNCTION delete(noofdocs):
RETURNS
Get fake estimated document count
FOR x 0 TO noofdocs // 4
randomrow¡-getrandomrow(count)
db.fake.deleteone(”id”: randomrow)
NEXT
ENDFUNCTION
PROCEDURE dropcollection:
db.fake.drop()
ENDPROCEDURE
noofoperations¡-[4000, 4500, 4800]
PROCEDURE main:
data¡getfakecsv()
op¡-’insert’: [], ”select”: [], ”delete”: [], ”update”: []
FOR x in no of operations:
INSERT data
SELECT data
DEETE data
UPDATE data
dropcollection
return op
ENDPROCEDURE
PROCEDURE runtask:
mongo¡-main()
dbop¡-”mongo”: mongo
return dbop
ENDPROCEDURE
25
IF name = ”main”
THEN
delete(1000)
ENDIF
psql.py
@timer
FUNCTION insert(noofrow, cursor):
Get fake data
cursor.execute(data[0])
rows¡-noofrow // len(data[1])
FOR i in range(rows):
FOR x in data[1]:
execute(x)
close connection
ENDFUNCTION
@timer
FUNCTION update(noofrows, cursor):
RETURNS
Select id from personal table
FOR x 0 TO noofrows//4
Get random gender
Get random gender count
Update gender in database
Execute query
NEXT
Close connection
ENDFUNCTION
@timer
FUNCTION delete(noofrow, cursor):
26
RETURN
Select id from personal
count¡-cursor.fetchone()[0]
FOR x 0 TO noofrow//4
Get a random row
Delete the rows in datbase
cursor.execute(query)
NEXT
Close connection
ENDFUNCTION
@timer
FUNCTION select(noofrows, cursor):
RETURNS
Select id from database
count¡-cursor.fetchone()[0]
FOR x 0 TO noofrows//4
Get random row count
select row count details from personal
execute query
NEXT
ENDFUNCTION
FUNCTION droptable(cursor):
RETURNS
drop table Personal
ENDFUNCTION
noofoperations¡-[4000, 4500, 4800]
FUNCTION main(cursor):
RETURNS
op¡-’insert’: [], ”select”: [], ”delete”: [], ”update”: []
FOR x in noofoperations:
27
INSERT x
SELECT x
DELETE x
UPDATE x
droptable(cursor)
return op
ENDFUNCTION
PROCEDURE runtask:
sl¡-main(sqlitecursor)
OUTPUT ”sqlite3 db operations done”
ps¡-main(pcursor)
OUTPUT ”postgres db operations done”
dbop¡-”sqlite”: sl, ”postgres”: ps
OUTPUT dbop
db.commit()
sqlitedb.commit()
return dbop
ENDPROCEDURE
IF name = ’main’
THEN
insert(500, cursor¡-pcursor)
db.close()
sqlitedb.close()
ENDIF
28
Chapter 6
startoperation():
This function creates a background task to run and the created task is added to bgtask
folder in bg.sqlite database. This function executes when we run app.py file and click on
WRITE button in the web page.
getpendingtask():
This function is defined in main.py file. It selects taskid from bgtask where taskstatus
is pending. It helps in completing the tasks that are still pending.
updatetaskstatus():
After all the pending tasks are completed, this function updates the taskstatus as com-
pleted. The arguments for this function are task id and data. It also keeps the data in the bg.
sqlite. The data here refers to the result after execution of the tasks.
checkstatus():
This function finds the taskid and checks its status in bg.sqlite . It returns “ok”:True in
json format if the status is completed. If not completed it returns “ok”:False.
getfakedata():
This function goes to the personal.sql test data and fetches some records for execution.
29
getfakecsv():
It fetches some collections from personal.json for execution.
timer():
It calculates the start time and end time. It returns end time-start time and is used where
ever the performance time is needed to be calculated.
getrandomgender():
It randomly selects the gender – male or female to insert in database while execution
of the queries.
getrandomrow():
similar to getrandomgender, this function randomly generates row numbers.
index():
It determines if the method is POST or GET. If it is Post method then it requests the
data and returns “ok”.
insert():
This function takes number of rows and data as parameters in psql and it takes number
of documents and data as parameters in mongodb. It executes the insert query.
select():
It takes number of documents in mongodb file and number of rows in psql file for exe-
cution of select query.
update():
It also takes number of rows and documents for execution of update query
30
delete():
This function executes delete query in both psql and mongodb files
runtask():
This function is responsible for execution of all select(), insert(), update() and delete()
functions.
runsql():
It is defined in main.py file. In this function the runtask() from psql file is called. The
queries for SQL databases gets executed when we run the main.py file.
runnosql():
In this runtask() from mongodb.py file is called. NoSQL queries gets executed
randomidgenerator():
Unique ID is generated for each task in this function
main():
It is responsible for updating the SQL and NoSQL result
31
Fig. 6.1 snapshot of SQL test data
32
mostly supports the Linux operating system. Raspbian is an operating system for the Rasp-
berry Pi that is based on the Debian Linux distribution. Modern operating systems are rather
large and come on DVD or as ISO images. These documents detail the whole OS installation
process. They begin by running a software that identifies computer hardware and copies all
data to the hard drive. However, The Raspberry Pi lacks a DVD player. As a result, we’ll need
to take a picture of the system or copy the image to an SD card.
Steps for installation are:
•Visit raspberry.org/downloads/ and download Raspberry pi imager
• Choose the OS, SD card and click ‘WRITE’
• After copying it to SD card remove it and connect to raspberry pi
• Now, connect raspberry pi to hardware. It starts boot.
• Green LED blinks if it’s working.
• After boot, welcome screen is displayed
• Raspbian is installed
After installation of Raspbian the interface is as follows,//
After installation of Raspberry Pi, we have to set up the Raspbian. We can choose
33
country, language and timezone. We can also set a password to the oue Raspberry. The de-
fault password will be “raspberry”.
3.We may use HDMI cable to link Raspberry Pi to devices such as a keyboard, mouse,
and monitor. To directly display the Raspberry Pi interface on a laptop we have to install VNC
viewer on the laptop.
After installing VNC server go to File¿New connection and enter the IP address of the
Raspberry Pi and click “Ok”. After that, input the username and password that we created
during the Raspbian installation. The interface looks as follows.
4.We are using docker in this application because, docker is a free and open platform
for creating, distributing, and executing apps. Docker decouples your programmes from your
infrastructure, allowing you to deliver software faster. With Docker, you can manage your
infrastructure in the same way you manage your apps.
5.First open terminal in the Raspberry and execute the “docker start postgres” com-
mand. This will start the PostgreSQL on your server.
6. Next, open the app.py file in your project folder and execute it. app.py file contains
34
the main flask code. Flask is used to interact with the server. It’s used to communicate with
the server through web pages. We have to run app.py first because it activates the webpage
created using the flask. After activation of webpage, if we click any button on webpage then
startoperation() function will be called.
7.Now go to browser and type http://localhost:5000/ and click enter .Then a webpage
displays as follows.
8.Now click on “write” button, because the Raspberry Pi takes a long time to do a
given task. So what we did is, when we click on WRITE it performs all the operations of
UPDATE, DELETE AND SELECT. After clicking on WRITE it creates a background task to
be completed. The bg.sqlie file contains the background task.
9.Next come back to the project folder and open new termina. Now run “main.py file”,
next the bg.sqlite file is created.
35
Fig. 6.6 bg.sqlite
10.main.py just goes to bg.sqlite file, it will go to bgtask table and it searches for the
task which is still pending. This all we be done by getpendingtask() function. A query is
written in this function to select the task.
11.we can mention the number of records to be evaluated and calculate the performance
time. After clicking write button the server gives the response back to the plotPage.html file.
the task is started by making a network call, every task has a unique ID. For every ID, for
every 5 seconds we are going to fetch the task status using fetch() function. The select, insert,
update, and remove procedures are completed if it is pending.
12.After running main.py file the execution gets started one by one.
36
Fig. 6.7 comparison for write
The comparison chart for UPDATE operation is as follows for 5000, 6000 and 7000
records.
The comparison chart for DELETE operation is as follows for 5000, 6000 and 7000
records
37
Fig. 6.9 comparison for delete
The comparison chart for SELECT operation is as follows for 5000, 6000 and 7000
records.
38
6.2.2 Result Analysis
WRITE
We can observe that SQLite takes less time than other two database to insert the
records. MongoDB takes longer than the other two databases to execute.
As SQLite3 does not have a central server, SQLite will open and close its database,
but there is no need to validate the cache. In other databases, however, each query is a distinct
transaction. Cache should be flushed many number of times which is equivalent to number of
transactions. So, SQLite inserts records faster than other databases
UPDATE
39
Here SQLite takes low time for less number of records. But in case of large quantity
MongoDB takes less time to update. This is because MongoDB’s data is in the form of key-
value pairs. So, to update data in MongoDB simply searching the key and updating the value
is enough. So, it takes less time for large number of records.
DELETE
Table 6.3 comparison table for DELETE
Delete results are also similar as compared to update. In case of SQLite, it is taking
less time for less number of records but more time for large number of records.as the data can
be deleted directly using key-value pairs it is taking less time for MongoDB.
SELECT
Table 6.4 comparison table for SELECT
There is a vast difference in performance time for SQL and NoSQL databases for re-
40
trieving the data. To retrieve data from tables it may include several conditions and there may
be many joins required to retrieve the properly. Due to joins the complexity increases a lot.
Getting data from several tables takes a long time. But in case of MongoDB as the data is
in JSON format it takes very less time as the data can be retrieved easily using key values.
Hence, NoSQL performs better than SQL in case of retrieving the data
41
Chapter 7
Table 7.1 comparison table for WRITE for more no.of records
Table 7.2 comparison table for UPDATE for more no.of records
42
Table 7.3 comparison table for DELETE for more no.of records
Table 7.4 comparison table for SELECT for more no.of records
Now let us compare performance time of 5000, 6000 and 7000 records in both Rasp-
berry Pi and Windows
43
Table 7.5 Comparison table between Raspberry Pi and Windows for WRITE
Table 7.6 Comparison table between Raspberry Pi and Windows for UPDATE
Table 7.7 Comparison table between Raspberry Pi and Windows for DELETE
44
Table 7.8 Comparison table between Raspberry Pi and Windows for SELECT
7.2 Validation
We can observe that even with different number of records SQLite3 takes less time than
other two databases
In UPDATE operations MongoDB takes less time as compared to other two databases
MongoDB here also takes less time to delete the records
As usual the MongoDB takes lesser time to retrieve the data
Hence, we can observe that the performance time decreases or increases proportionally
according to the number of records.
In case of windows and Raspberry Pi Clearly we can observe that the performance for
any CRUD operations in windows is less than performance time in Raspberry Pi
45
Chapter 8
CONCLUSION
Databases are now an essential component of any programme. The world revolves
around data. Choosing best database for your application which reduces the complexity and
increases the efficiency is very important. The main idea of this project is to provide a guide
to the developers and give a clear idea on what database to choose.
The major focus of this research was on determining the performance time for SQL
and NoSQL databases. The main constraint considered here to evaluate the performance is
the response time. The databases are evaluated on a Raspberry pi 3 model B+.
The Raspberry Pi was introduced with the aim to introduce the computing environ-
ment to the people of all ages. It is pocket friendly and credit card size computer. As the
world is moving on much data is inserted into the internet. This data includes multimedia
like photos, videos and many more. Relational databases struggle to manage this multimedia.
So overcome these obstacles to perform operations on multimedia, NoSQL was introduced.
Because NoSQL stores data in JSON format, these databases can readily manage it. As the
non-relational databases are evolving fastly they are also main part of this project.
Raspbian operating system was installed on Raspberry Pi. It is a Debian Linux OS.
CRUD operations are executed by SQLite3, MongoDB and PostgreSQL on a Raspberry Pi.
Same number of records were considered for all the databases while evaluating. Same envi-
ronment was also maintained. Even there was lot of difference in performance time when we
execute the same queries with same number of records on Windows.
The results showed that NoSQL databases takes less time compared other databases in
many query executions.
46
REFERENCES
[1] Veronika Abramova, Jorge Bernardino, and Pedro Furtado. Which nosql database? a
performance overview. Open Journal of Databases (OJDB), 1(2):17–24, 2014.
[2] Alexandru Boicea, Florin Radulescu, and Laura Ioana Agapin. Mongodb vs oracle–
database comparison. In 2012 third international conference on emerging intelligent
data and web technologies, pages 330–335. IEEE, 2012.
[3] Deka Ganesh Chandra. Base analysis of nosql database. Future Generation Computer
Systems, 52:13–21, 2015.
[4] Yong-Lak Choi, Woo-Seong Jeon, and Seok-Hwan Yoon. Improving database system
performance by applying nosql. Journal of Information Processing Systems, 10(3):355–
364, 2014.
[5] Mahmoud Moustafa Eyada, Walaa Saber, Mohammed M El Genidy, and Fathy Amer.
Performance evaluation of iot data management using mongodb versus mysql databases
in different cloud environments. IEEE Access, 8:110656–110668, 2020.
[6] Robin Hecht and Stefan Jablonski. Nosql evaluation: A use case oriented survey. In
2011 International Conference on Cloud and Service Computing, pages 336–341. IEEE,
2011.
[8] Neal Leavitt. Will nosql databases live up to their promise? Computer, 43(2):12–14,
2010.
47
[9] Yishan Li and Sathiamoorthy Manoharan. A performance comparison of sql and nosql
databases. In 2013 IEEE Pacific Rim Conference on Communications, Computers and
Signal Processing (PACRIM), pages 15–19. IEEE, 2013.
[10] João Ricardo Lourenço, Bruno Cabral, Paulo Carreiro, Marco Vieira, and Jorge
Bernardino. Choosing the right nosql database for the job: a quality attribute evalua-
tion. Journal of Big Data, 2(1):1–26, 2015.
[11] Denis Nelubin and Ben Engber. Nosql failover characteristics: aerospike, cassandra,
couchbase. Technical report, MongoDB. Technical report, Thumbtack Technology, 25
Broadway, Floor 9, New York, 2013.
[12] Mythili Thiruganam, S Margret Anouncia, and Sachin Kantipudi. Automatic defect
detection and counting in radiographic weldment images. International Journal of Com-
puter Applications, 10(2):1–5, 2010.
[13] Jan Sipke Van der Veen, Bram Van der Waaij, and Robert J Meijer. Sensor data stor-
age performance: Sql or nosql, physical or virtual. In 2012 IEEE fifth international
conference on cloud computing, pages 431–438. IEEE, 2012.
48
The Report is Generated by DrillBit Plagiarism Detection Software
Submission Information
Result Information
Similarity 11 %
1 10 20 30 40 50 60 70 80 90
Student Quotes
Internet
Paper 0.21%
1.01%
0.83%
A-Satisfactory (0-10%)
B-Upgrade (11-40%)
11 49 B C-Poor (41-60%)
D-Unacceptable (61-100%)
SIMILARITY % MATCHED SOURCES GRADE
2 www.diva-portal.org Publication
1
3 support.industry.siemens.com Publication
1
5 support.industry.siemens.com Publication
<1
6 www.diva-portal.org Publication
<1
10 www.diva-portal.org Publication
<1
11 support.industry.siemens.com Publication
<1
12 support.industry.siemens.com Publication
<1
15 tech.ed.gov Publication
<1
19 www.iraj.in Publication
<1
23 arxiv.org Publication
<1
26 support.industry.siemens.com Publication
<1
32 support.industry.siemens.com Publication
<1
38 support.industry.siemens.com Publication
<1
39 support.industry.siemens.com Publication
<1
44 db.in.tum.de Publication
<1
47 A technique for generating natural colour images from false colour Publication
<1
composite ima by Patra-2006