SQLAlchemy

SQLAlchemy

SQLAlchemy is a package which facilitates the communication between Python programs

and databases. Generally, this package is used as an Object Relational Mapper (ORM)
tool, which translates Python classes to tables and also automatically converts
function calls to SQL statements.

A major benefit of using SQLAlchemy is that it enables to write Python code that
can map database schema to applications.

Understanding SQLAlchemy
SQLAlchmey consists of two components; SQLAlchemy Core and SQLAlchemy ORM. The
SQLAlchemy ORM is built on SQLAlchemy Core, and users can utilize any of the
components to build applications, depending on the level of abstraction the user
needs.

The major benefits of using SQLAlchemy over raw SQL are:

Cleaner code.

SQLAlchemy ORM functionalities enable secure code, as it helps to mitigate

vulnerabilities such as SQL injection, etc.

SQLAlchemy enables simpler logic, and allows user's to abstract all database-level
logic to Python objects.

Installing SQLAlchemy and DBAPI

To install SQLAlchemy, use the following:

pip install sqlalchemy

To check if the installation is fine, and the current version of SQLAlchemy, use
the following code in Python shell:

import sqlalchemy
sqlalchemy.__version__

Installing SQLAlchemy and DBAPI

SQLAlchemy works only with an SQLite database by default without adding additional
drivers. To work with other databases, a user must install a DBAPI-compliant driver
specific to the database.

A DBAPI contains many drivers, such as, PyMySQL, MySQL-Connector, CyMySQL, and
MySQL-Python are drivers for MYSQL. Similarly, psycopg2 and pg80000 are drivers for
PostgreSQL.

Example

To install psycopg2 for a PostgreSQL database, use the following:

pip install psycopg2

Creating Engine
In SQLAlchemy, to connect to a database, the first step is to create an engine. An
engine is an object that interacts with a database, and consists of two components:

Dialect
Connection Pool
SQLAlchemy Dialect

Generally, SQL is the standard language used to access databases. However, the SQL
syntax varies from one database to another. Also, database vendors rarely adhere to
database standards. To handle such issues, Dialect is used. Dialect defines the
behavior of a database, and handles tasks like generating SQL statements,
execution, result set handling, etc. The only requirement is to install the
appropriate driver.

Creating Engine
SQLAlchemy Connection Pool

Connection Pool is used as a standard method to store connections in the cache, so

that it can be reused. Creating a new connection every time is expensive and time-
consuming. Whereas, by using Connection Pool, an application can get a connection
from the pool. After performing the appropriate queries, the application releases
the connection back to the connection pool. If all connections available are being
used, a new connection is created, and added to the pool.

To create an engine object, use the create_engine() function available in the

SQLAlchemy package.

Creating Engine
A connection string provides information about the data source. The general format
of a connection string is:

dialect+driver://username:password@host:port/database
Note: The above string is the general format of the createengine() function.

In the above string,

Dialect is the name of the database, like mysql, postgresql, mssql, oracle, sqLite,
etc.
Driver is the DBAPI. If the driver is not specified, the default driver is used.
Username and Password are the credentials required to login to a database.
Host is the location of the database server.
Port is the optional database port.
Database is the name of the database to be connected.
Examples

To connect to a MySQL database, use:

from sqlalchemy import create_engine

engine = create_engine("mysql+mysqldb://root:passwords@localhost/mydb")
To connect to a PostgreSQL database, use:

from sqlalchemy import create_engine

engine = create_engine("postgresql+psycopg2://root:passwords@localhost/mydb")

Connecting to Database
When creating an engine, it will not instantly establish a connection with the
database. To establish a connection with the database, use the connect() method of
the engine object.

engine.connect()
Example
To connect to a database, use:

from sqlalchemy import create_engine

engine= create_engine("postgresql+psycopg2://root:passwords@localhost/mydb")

engine.connect()

print(engine)
The output will be:

Engine(postgres+psycopg2://root:***@localhost/mydb)

Additional Arguments
The following table lists the arguments that can be used in the create_engine()
function:

Argument Description
echo A boolean argument (if set to TRUE) that logs all the SQL being
executed to the standard output.
pool_size Specifies the number of connections to keep in the connection pool.
The default is 5.
max_overflow Specifies the number of connections that can be opened beyond the
pool_size setting. The default is 10.
encoding Specifies the encoding to be used by SQLAlchemy. The default is
utf-8. This setting does not control the encoding scheme of the database.
isolation_level Specifies the isolation level to use. This setting controls how a
transaction is isolated from other transactions. Different databases can support
different isolation levels.

Introduction

This section describes how to perform CRUD operations such as Create, Read, Update,
and Delete by using SQLAlchemy. The two methods we will explore are:

CRUD using Raw SQL

CRUD using SQL ORM

CRUD using Raw SQL

In this method, SQL query syntax is used to perform operations, instead of Python
statements.

Example 1

from sqlalchemy import create_engine

db_string = "sqlite:///test.db"

db = create_engine(db_string)

# Create
db.execute("CREATE TABLE IF NOT EXISTS employee (empid text, empname text,
designation text)")

db.execute("INSERT INTO employee (empid, empname, designation) VALUES ('10000',

'ABC', 'Developer')")

CRUD using Raw SQL

Example 1

# Read
result_set = db.execute("SELECT * FROM employee")

for r in result_set:
print(r)

#Update
db.execute("UPDATE employee SET designation='Tester' WHERE empid='10000'")

#Delete
db.execute("DELETE FROM employee WHERE empid='10000'")
In the above example, basic CRUD operations are executed by using Raw SQL. A
database named 'test' is created. The task is to insert employee data, read the
data, update the designation field, and finally delete the data. Even though the
task is simple, since there are SQL queries, the entire code looks complex, and
will be even more complex for complicated operations and dynamic queries.

Therefore, performing tasks using Raw SQL makes coding complex.

CRUD using Raw SQL

Example 2

from sqlalchemy import create_engine

db_string = "sqlite:///test.db"

db = create_engine(db_string)

#create
db.execute("CREATE TABLE IF NOT EXISTS employee (empid text, empname text,
designation text)")

db.execute("INSERT INTO employee (empid, empname, designation) VALUES ('10001',

'ABC', 'Developer'), ('10002' , 'XYZ', 'Tester'),('10003','DEF','Technical
Support'),('10004','WXY','HR'),('10005','JKL','Manager')")

CRUD using Raw SQL

Example 2

# Read
result_set = db.execute("SELECT * FROM employee")

for r in result_set:
print(r)

#Update
db.execute("UPDATE employee SET designation='Tester' WHERE empid
in('10001','10005')")

#Delete
db.execute("DELETE FROM employee WHERE empid='10005'")

CRUD using SQL ORM

In this method, operations are executed entirely in Python, and code is simpler.
Example 1

from sqlalchemy import create_engine

from sqlalchemy import Column, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

db_string = "sqlite:///test.db"

db = create_engine(db_string)
base = declarative_base()

class Employee(base):
__tablename__ = 'employee'
empid = Column(String, primary_key=True)
empname = Column(String)
designation = Column(String)

Session = sessionmaker(db)
session = Session()

base.metadata.create_all(db)

CRUD using SQL ORM

Example 1

#Create
emp = Employee(empid="10000", empname="ABC", designation="Developer")
session.add(emp)
session.commit()

#Read
employee = session.query(Employee)
for emps in employee:
print(emps.designation)

#Update
emp.designation = "Tester"
session.commit()

#Delete
session.delete(emp)
session.commit()
In the above example, instead of using table, declarative_base and sessionmaker are
used. Instead of creating tables, a Python class is created which subclasses
declarative_base, and instead of making connections to the database, sessions are
used. In Raw SQL, column and column types are used for the employee object. In SQL
ORM, the attributes empid, empname, and designation are used. Data can be inserted
in the database by instantiating the Python objects, and passing them to
session.add(). After making changes to the database, session.commit() is used.

Therefore, performing tasks using SQL ORM makes coding much simpler.

CRUD using SQL ORM

Example 2

from sqlalchemy import create_engine

from sqlalchemy import Column, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

db_string = "sqlite:///restaurantmenuss.db"

db = create_engine(db_string)
base = declarative_base()

class Employee(base):
__tablename__ = 'employee'
empid = Column(String, primary_key=True)
empname = Column(String)
designation = Column(String)

Session = sessionmaker(db)
session = Session()

base.metadata.create_all(db)

CRUD using SQL ORM

Example 2

#Create
emp1 = Employee(empid="10001", empname="ABC", designation="Developer")

emp2=Employee(empid="10002", empname="XYZ", designation="Tester")

emp3=Employee(empid="10003", empname="DEF", designation="Technical Support")

session.add_all([emp1,emp2,emp3])

session.commit()

#Read
employee = session.query(Employee).all()

for emps in employee:

print(emps.designation)

#Update
emp1.designation = "Tester"
session.commit()

#Delete
session.delete(emp1)

session.delete(emp2)

session.delete(emp3)

session.commit()
----------------------------------------------------------
from sqlalchemy import create_engine

db_string = "sqlite:///tests.db"

db = create_engine(db_string)

db.execute("CREATE TABLE IF NOT EXISTS players (plyid text, plyname text, runs
text)")

db.execute("INSERT INTO players (plyid, plyname, runs) VALUES ('10001', 'ply1',

'100')")
db.execute("INSERT INTO players (plyid, plyname, runs) VALUES ('10002', 'ply2',
'80')")
db.execute("INSERT INTO players (plyid, plyname, runs) VALUES ('10003', 'ply3',
'65')")
db.execute("INSERT INTO players (plyid, plyname, runs) VALUES ('10004', 'ply4',
'95')")
db.execute("INSERT INTO players (plyid, plyname, runs) VALUES ('10005', 'ply5',
'99')")

# Read
s = []
result_set = db.execute("SELECT * FROM players")
s = list(result_set)

#Update

q=[]

db.execute("UPDATE players SET runs='100' WHERE plyname='ply5'")

result_set1 = db.execute("SELECT * FROM players")
q = list(result_set1)

#Delete
e=[]
db.execute("DELETE FROM players WHERE plyname='ply5'")
result_set2 = db.execute("SELECT * FROM players")
e = list(result_set2)

print(s)
print(q)
print(e)
s=str(s)
q=str(q)
e=str(e)
with open(".hidden.txt",'w') as f:
f.write(s)

with open(".hidden1.txt",'w') as f:
f.write(q)

with open(".hidden2.txt",'w') as outfile:

outfile.write(e)

[('10001', 'ply1', '100'), ('10002', 'ply2', '80'), ('10003', 'ply3', '65'),

('10004', 'ply4', '95'), ('10005', 'ply5', '99')]

[('10001', 'ply1', '100'), ('10002', 'ply2', '80'), ('10003', 'ply3', '65'),

('10004', 'ply4', '95'), ('10005', 'ply5', '100')]

[('10001', 'ply1', '100'), ('10002', 'ply2', '80'), ('10003', 'ply3', '65'),

('10004', 'ply4', '95')]
============================= test session starts ==============================

platform linux -- Python 3.5.2, pytest-3.2.1, py-1.10.0, pluggy-0.4.0

rootdir: /projects/challenge, inifile:

collected 3 items

test.py ...

=========================== 3 passed in 0.01 seconds ===========================

from sqlalchemy import create_engine

from sqlalchemy import Column, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker

db_string = "sqlite:///tests.db"

db = create_engine(db_string)
base = declarative_base()

class Teacher(base):
#Define table name and column name
__tablename__ = 'students'
stdid = Column(String, primary_key=True)
stdname = Column(String)
subjects = Column(String)
marks = Column(String)

Session = sessionmaker(db)
session = Session()

base.metadata.create_all(db)

#Create
list2 = []
list3 = []
list4 = []
list5 = []
list1 = [['10001', 'std1', 'Maths', '100'], ['10002', 'std2', 'Physics', '80'],
['10003', 'std3', 'English', '65'], ['10004', 'std4', 'Social', '95'], ['10005',
'std5', 'Chemistry', '99']]
std1 = Teacher(stdid=list1[0][0], stdname=list1[0][1], subjects=list1[0][2],
marks=list1[0][3])
std2 = Teacher(stdid=list1[1][0], stdname=list1[1][1], subjects=list1[1][2],
marks=list1[1][3])
std3 = Teacher(stdid=list1[2][0], stdname=list1[2][1], subjects=list1[2][2],
marks=list1[2][3])
std4 = Teacher(stdid=list1[3][0], stdname=list1[3][1], subjects=list1[3][2],
marks=list1[3][3])
std5 = Teacher(stdid=list1[4][0], stdname=list1[4][1], subjects=list1[4][2],
marks=list1[4][3])
session.add_all([std1,std2,std3,std4,std5])
session.commit()

#Read
s1 = session.query(Teacher).all()
for i in s1:
 list4.append([i.stdid,i.stdname,i.subjects,i.marks])
print(list4)
s = list4

#Update
std5.subjects = "Language"
session.commit()
q1 = session.query(Teacher).all()
for i in q1:
if i.stdname == "std5":
list2.append([i.stdid,i.stdname,i.subjects,i.marks])
print(list2[0])
q = list2[0]
#Delete
session.delete(std5)
session.commit()
e1 = session.query(Teacher).all()
for i in e1:
list3.append([i.stdid,i.stdname,i.subjects,i.marks])
print(list3)
e = list3

s=str(s)
q=str(q)
e=str(e)
with open(".hidden.txt",'w') as f:
f.write(s)

with open(".hidden1.txt",'w') as f:
f.write(q)

with open(".hidden2.txt",'w') as outfile:

outfile.write(e)

user@workspacen94kyjpzqyc19nox:/projects/challenge$ python3.5 prog.py

[['10001', 'std1', 'Maths', '100'], ['10002', 'std2', 'Physics', '80'], ['10003',
'std3', 'English', '65'], ['10004', 'std4', 'Social', '95'], ['10005', 'std5',
'Chemistry', '99']]
['10005', 'std5', 'Language', '99']
[['10001', 'std1', 'Maths', '100'], ['10002', 'std2', 'Physics', '80'], ['10003',
'std3', 'English', '65'], ['10004', 'std4', 'Social', '95']]