Python Advanced: © 2020 The Knowledge Academy LTD

Python Advanced
© 2020 The Knowledge Academy Ltd 1

About The Knowledge Academy
• World Class Training Solutions
• Subject Matter Experts
• Highest Quality Training Material
• Accelerated Learning Techniques
• Project, Programme, and Change
Management, ITIL® Consultancy
• Bespoke Tailor Made Training Solutions
• PRINCE2®, MSP®, ITIL®, Soft Skills, and More

Administration
• Trainer
• Fire Procedures
• Facilities
• Days/Times
• Breaks
• Special Needs
• Delegate ID check
• Phones and Mobile devices

Outlines
• Module 1: Core Python
• Module 2: Data Analysis using

Python
• Module 3: Web implementation

Module 1: Core Python

Introduction
• Python is a programming language. It was designed with an emphasis of code
readability which allows the programmers to show their concepts in few lines of code
• It is used for system scripting, software development and web-development (server-

side)
• It works on different platforms(Windows, Linux, Mac)
• It is a object- oriented language and runs on an interpreter system

Introduction
• Python is a programming language. It was designed with an emphasis of code
readability which allows the programmers to show their concepts in few lines of code
• It is used for system scripting, software development and web-development (server-

side)
• It works on different platforms(Windows, Linux, Mac)
• It is a object- oriented language and runs on an interpreter system

Memory Management in Python
Garbage collection in python
• Python uses an automatic method to allocate and deallocate the memory.
• As using dynamic memory allocation in programming languages for example c and c++
the user does not have to pre-allocate or deallocate the memory
• Python follows two ways for memory allocation:
o Reference counting
o Garbage collection

(Continued)
• Former to python version 2.0, Reference counting was used by the python interpreter
for the memory management
• Reference counting works by counting the number of times an object is referenced by

the other objects in the system
• Reference count form an object is decremented, when references to an object are

removed
• The object is being deallocated when the reference count becomes zero show shown
in the following figure

(Continued)
• The literal value 5 is an object as shown in the following figure in the previous slide.
The reference count of object 5 is incremented by 1 in the line 1. As it is dereferenced
in line 2 because its reference count becomes zero. Ultimately, object is deallocated by
the garbage collector
• When there is no method the reference count of the object can reach then a reference
cycle is being created.
• Lists, tuples, instances, classes, dictionaries as well as functions are contained by the
reference cycles

(Continued)
• The simplest way to create a reference cycle is to create an object which refers to itself
as in the following example

(Continued)
• As you can see in the following example create_cycle() creates an object x which refers
to itself, so when the functions returns, the object x will not automatically be freed
• This will cause the memory which is used by the x until the python garbage collector is
invoked

Ways to make an object eligible for garbage collection

(Continued)
• Now, the reference count is two for the created list. Even though, since it cannot be
used again and it cannot be reached from inside the python.
• It is taken as garbage. This list is never freed in the present version of python
Automatic Garbage Collection of Cycles
• Since reference cycles take computational work to find out, garbage collection should
be a scheduled activity.

(Continued)
• Python schedules garbage collection on the basis of a threshold of an object

allocations as well as an object deallocations
• When the threshold number is greater than the result of number of allocations minus
the number of deallocations, the garbage collection run
• By importing the gc module as well as asking for the garbage collection thresholds one
can check the threshold for new objects ( the objects which known as generation 0
objects in the python)

(Continued)
Output:

(Continued)
• In the previous slide example, 700 is the default threshold
• This means when the number of allocations in contrast to the number of deallocations
is greater than 700, then the automatic garbage collector will run
• Any portion of your code which releases large Blocks of memory are good for running
manual garbage collection

Functions
• Functions take one or more arguments and return only one value/object (or none)
• Functions can be stand-alone ex: type(S)
• Function can be member of a class, in which case the function is called a “method”
• Python has a large library of built-in functions: Numeric, String, Conversion, functions
for tuples and lists
• User-defined functions can be created as well

Functions
(Continued)
Example of Function

Functions
Calling a Function
• When we define a function then we specifies the parameters that are to be included in
the function as well as the blocks of code.
• When the basic structure of a function is completed, One can execute it directly from
the python prompt either from the another function as shown in the example in
upcoming slide
• A Function can be called many time

Functions
Calling a Function

Functions
Pass by reference
• The process of passing reference of an argument in the calling function to the

corresponding formal arguments or parameters of the called function is called as pass
by reference
• A called function can modify the values of the arguments or parameter with the help
of reference passed in
• This modification of the values in the called function also reflects back in the calling
function as shown in the following example in upcoming slide

Functions
Pass by reference

Functions Arguments
The following are the types of function arguments:
Required arguments Default arguments
Keyword arguments Variable-length arguments

Functions Arguments
Required arguments
• These are those arguments which are passed to a function in correct positional order
• The number of arguments in the function call must match which the number of
arguments in the definition of the function
• When you will call to the printme() function then you have to pass an argument
otherwise you will get an error as shown in the following example in the upcoming
slide

Functions Arguments
(Continued)
Example: -

Functions Arguments
Keyword arguments
• When a function is called with some values then these values get assigned to the
arguments according to their positions in the function definition.
• In python we can call a function by using keywords. When we use this procedure to
call a function then we can change order of the arguments while calling a function
• This method allows to skip arguments or place them out of the order because the
python interpreter is able to use the keywords provided to match the values with the
parameters

Functions Arguments
(Continued)
• You can also make keyword calls to the printme() function as shown in the following
figure

Functions Arguments
Default arguments
• These are those arguments which assumes a default value when value is not provided
in the function call for that argument.
• You can get better idea about the default function by looking at the following figure

Functions Arguments
Variable-length arguments
• Variable length argument allows a function to receive any number of arguments. It is

helpful when you want a function to handle variable number of arguments to
requirement.
• These functions are not named in the function definition, unlike required and default
arguments
• Before the variable name of the which holds the values of all non keyword variable
arguments an asterisk (*) is placed

Functions Arguments
(Continued)
• During the function call if no additional arguments are specified then this tuple will
remain empty

Lambda Expression/function
• Lambda functions are also called anonymous function. Lambda functions are not
declared in the standard manner by using def keyword.
• For creating small anonymous function you can use lambda keyword
• Lambda cannot contain commands or multiple expressions. Lambda can take any
number of arguments but it returns just one value in the form of an expression
• A direct call to print cannot be made by the anonymous function because it requires an
expression
• Lambda functions have their own local namespace and cannot access variables other
than those in their parameters as well as those in the global namespace

Lambda Expression/function
• Even though it seems that lambda’s are a one-line version of a function. They are not
as same as inline statements in and C++, which purpose is by passing function stack
allocation during innovation for performance reasons
• Example:

Generators and Decorators
Generators
• The are two terms involved in generators:
Generator-Function Generator-Object

Generator – Function
• A generator function is defined as a normal function, It uses yield keyword in spite of

using return whenever it needs to generate a value
• The function automatically becomes a generator function if the body of a def contains
yield
Example:

Generator – Object
• A generator object is returned by the generator functions. Generator objects are used
either by calling the next method on the generator object or using the generator
object in a “for in” loop

Decorators
• In Python programming language functions are known as first class objects which
means that functions are objects; they can be referenced to, passed to a variable as
well as returned from another function too.
• We can defined a function inside the another function as well as can also be passed as
argument to another function
• Decorates allows programmers to change the behavior of function or class so that’s

why decorates considered as useful as well as powerful tools in python

(Continued)
• In order to extend the behavior of the wrapped function, without permanently

modifying it decorates permits us to wrap another function
• In decorates, functions are taken as the arguments into another function and then
called inside the wrapper function

(Continued)
Example:

Map, Reduce and Filter
Map
• After applying the given function to each item of a given iterable (list, tuple excreta.)
map() functions returns a map object (Which is an iterator) of the results.
o Syntax :
map(fun, iter)
Parameters
• iter : It is an iterable which is to be mapped.

• fun : It is a function to which map passes each element of given iterable.

(Continued)
• One or more iterable can be pass to the map() function
Example:

Reduce
• Reduce( fun, seq) function is defined in “functools” module. This function is used to
apply a particular function passed in its argument to all of the list elements mentioned
in the sequence passed along.
Working :
• Firstly , in the sequence first two elements are selected and result is obtained
• Second step is to apply the same function to the previously attained result and the
number just succeeding the second element and the result is again sorted

(Continued)
• Until no more elements are left in the container this this process is continued
• Final result is returned and print on the screen
• Filter_none
Example:

Filter
• The filter() method is used to filter the given sequence by using a function which tests
each element in the sequence to be true or not
Syntax:
filter(function, sequence)
Parameters:
function: function that tests if each element of a
sequence true or not.
sequence: sequence which needs to be filtered, it can
be sets, lists, tuples, or containers of any iterators.
Returns:
returns an iterator that is already filtered.

(Continued)
Example:

OOPS concepts and implementations
• The following are the concepts of OOPS:
Classes Objects Inheritance
Polymorphism Data Abstraction Encapsulation
Destroying
Overriding
Data Hiding Objects (Garbage
Methods
Collection)

Classes
• Python is known as object oriented programming language
• In python everything is an object with its properties as well as methods
• A class is a “blueprint” of an object or a class is like an object constructor

(Continued)
• Use class keyword to create a class:
Example to create a class

Objects
• An object is also called as the instance of the class
• An object is created using constructor of the class
• Object is simply a collection of data (variables) and methods (functions)

(Continued)
Example to create an object

Inheritance
• In spite of making a class from starch You can create a class by driving it from already
made class by listing the parent class (preexisting class) in the parentheses after the
new class name
• Attributes of parent class are inherited by the child class. Moreover, you can use these
attributes as if they were defined in the child class.
• A child class can also override data members as well as methods from the parent

(Continued)
Syntax:
class SubClassName (ParentClass1[, ParentClass2, ...]):

'Optional class documentation string'
class_suite

(Continued)
Example:

Polymorphism
• The word polymorphism defines having several forms. In programming, polymorphism

means same function name ( but various signatures) being uses for different types
• Example:

(Continued)
Polymorphism with class methods
Polymorphism with Inheritance
Polymorphism with a Function and

objects

Data Abstraction
• Data abstraction provides only essential information about the data to the user and
hides background implementations.
• Abstraction is an important concept of Object Oriented Programming (OOPS)
Abstract class in Python
• An abstract class can be considered as a blueprint for other classes, It permits you to
make a group of methods that should be made within any child classes built from your
abstract class

Data Abstraction
• A class is called an abstract class which contain one or more abstract methods. A
method which has declaration but has not any implementation is called as abstract
class
• Abstract classes needs subclasses to provide implementations for those abstract

methods which are already defined in the abstract classes. Moreover, abstract classes
are not able to instantiated.
• For all implementations of a component, when we want to provide a common

implemented functionality then we use an abstract class

(Continued)
Example:

Encapsulation
• Object Oriented Programming has a concept of Encapsulation. In this concept data as

well as methods are wrapped that work on data within one unit
• It does not permit to access variables as well as methods directly and also it can stop
the accidental alteration of data
• An object’s variable can only be changed by an object’s method in order to prevent

accidental alteration. Those type of variables are known as private variables
• As a class encapsulates all the data which are the member functions, variables etc. So a
class is an example of encapsulation

Encapsulation

Data Hiding
• Data hiding permits preventing the functions of a program to access directly the
internal representation of a class type
• By default all members of a class can be accessed outside of class.
• You can make class members private or protected to prevent it.

(Continued)
• Outside the class definition, an object’s attributes may or may not be visible. You need
to name attributes with a double underscore prefix, and those attributes then are not
be directly visible to outsiders
Example:

Overriding Methods
• You can always override your parent class methods when you may want special or
different functionality in your subclass then you override parent’s method

Destroying Objects (Garbage Collection)
• To free the memory space, python deletes unneeded objects automatically.
• The procedure by which python on regular intervals of time reclaims blocks of memory
that on longer are in use is termed garbage collection
• During program execution pythons garbage collector runs and when an object’s
reference count reaches zero, is provoked.
• An object’s reference count changes as the number of pseudonyms that point it

changes

(Continued)
• When a object is assigned a new name or placed in container (list, tuple or dictionary)
it’s reference count increase.
• When an object is deleted with del, it’s reference is reassigned, or it’s reference goes
out of scope than it’s reference count decrease.
• When an object’s reference count reaches zero, python collects it automatically.

(Continued)
Example:

Pickling/Unpickling
Pickling and Unpickling
• To serializing and de-serializing python object structures python’s pickle module is

used.
• The procedure of converts any kind of python objects such as list, dict etc into byte
streams for example 0s and 1s is called pickling or serializing or flattening or
marshalling.
• Similarly, the procedure of converting stream which we were generated by using

pickling back into python objects by a process called as unpickling.

Pickling/Unpickling
(Continued)
Example of pickling:

Pickling/Unpickling
(Continued)
Example of Unpickling:

Exception Handling
Exception Handling
• As other language, even python provides the runtime errors through exception
handlingMethod.
• A few of the standard exceptions which are most recurrent contain IndexError, IOError,
TypeError, ZeroDivisionError, ImportError.
• Exception is the base class for all the exceptions in python.

Exception handling
Output:
• Try to access the array element whose index is out of bound and handle the
corresponding exception.

Multithreading
Multithreading in Python
• Running various threads is as same as running several programs simultaneously.
• Multiple threads within a process share the same data space with the main thread so
that sharing of information or communication between them is easier which was not
possible if they were separate processes
• Sometimes threads are called as light-weight processes. Moreover, they do not require
much memory overhead; they are cheaper than processes
• A thread has a starting, an execution order, and a result. It has also an instruction
pointer that keeps track of where within its context it is currently running

Multithreading
(Continued)
• It can temporarily be put on hold also known as sleeping while other threads are
running - this is called yielding.
• It can be pre-empted (interrupted)

Multithreading
Output:

File I/O
The Input Function
• The input function is alike to raw_input, except that it assumes the input is a valid
python expression as well as returns the calculated result to you
Example:

File I/O
Opening and Closing Files
• Python provides fundamental methods as well as functions essential to manipulate
files by default. Most of the file manipulation can be done by using file object
The open Function
• Before you can write either read a file, you need to open it by using a built-in function
in python which is open() function. This function creates a file object which will be
utilized to call other support methods associated with it
Syntax:
file object = open(file_name [, access_mode][, buffering])

File I/O
(Continued)
Example:

File I/O
The close() Method
• The close method of a file object deletes any unwritten information as well as closes
the file object, after that no more writing is done
• When the reference object of a file is reassigned to another file then python
automatically closes a file. It is good to choose close() method to close a file
Syntax:
fileObject.close()

File I/O
(Continued)
Example:

File I/O
The write() Method
• The write() method writes any string to an open file. It strings can have binary data as
well as not just text it is an essential thing to note
• A newline character (‘\n’) to the end of the string is not added by the write() method
Syntax
fileObject.write(string)

File I/O
(Continued)
Example:

File I/O
The read() Method
• The read() method is used to read a string from an open file. It is essential to note that
apart from text python strings can have binary data
Syntax:
fileObject.read([count])

File I/O
(Continued)
Example:

File I/O
File Positions
• To check current position within the file, tell() method is used.
• To change the current position of file seek (offset, from]) method is used
• The number of bytes to be moved are indicated by the offset argument
• Reference position from where the bytes are to be moved are specified by the from
argument

File I/O
(Continued)
• If from is set to 0, it means use the starting of the file as the reference position as well
as 1 means use the current position as the reference position
• If it is set 2 then the end of the file would be taken as the reference position

File I/O
(Continued)
Example:

File I/O
The rename() Method
• The rename() method takes two arguments, the current filename as well as the new
filename
Syntax:
os.rename(current_file_name, new_file_name)

File I/O
(continued)
Example:

File I/O
The remove() Method
• To delete files by supplying the name of the file which has to be deleted as the
argument remove() method is used
Syntax:
os.remove(file_name)

File I/O
(Continued)
Example:

File I/O
The mkdir() Method
• To create directories in the current directories you can use mkdir() method of the os
module. You have to pass an argument to this method which contains the name of the
directory to be created
Example:

File I/O
The getcwd() Method
• The getcwd() method displays the current working directory.
Syntax:
os.getcwd()

File I/O
(Continued)
Example:

File I/O
The rmdir() Method
• The directory which is passed as an argument in the method is deleted by using rmdir()
method
• Before removing a directory, all the contents in it should be removed.
Syntax:
os.rmdir('dirname')

File I/O
(Continued)
Example:

How to create Modules and packages
Modules
• Assume that Module is same as a code library
• A file which contains a group of functions which you want to include in your
application
Example:

Create and Access a Python Package
• Packages are a way of structuring many packages as well as modules which helps in a
well organized hierarchy of a data set, making the directories as well as modules easy
to access
• As same as the alike drives as well as folders in an operating system to help to store
files, similarly packages helps us in storing sub-package as well as modules, so that user
can use them when essential

(Continued)
Creating and Exploring Packages
• To give information to the python that a specific directory is a package, a file named
_init_.py is created inside it as well as then it is considered as a package and modules
and sub-packages can be created within it
• Initialization code is used to code the _init_.py file for the package. An _int_.py file can
also be left blank

(Continued)
For creating a package in python, follow below written steps:
• Firstly, a directory is created as well as a package name is given to it, preferably related
to its operation
• After that, classes as well as the required functions are put in it
• At the end an _init_.py file is created inside the directory which helps the python to
know that the directory is a package

(Continued)
Example to create package
Step 1: Create the package and save it with “.py” extension
Package 1 Package 2

Step 2: Open a new file for import that packages

Log management (Logging)
Logging
• Logging module in the python is very powerful module which is designed to meet the
needs of the enterprise teams as well as the beginners
• You can integrate your log messages with the ones from the third-party python
libraries to generate a homogeneous log for your application because logging is used
by most of the third-party python libraries
• Adding logging in your python program is very simple as shown below:
import logging

(Continued)
• You can use something called a “logger” to log that messages which you want to see.
By default, there are 5 standard levels demonstrating the severity of events
• Each has a corresponding method which can be used to log events at that level of
severity.

(Continued)
Below written are the defined levels in order of increasing severity
• DEBUG
• INFO
• ERROR
• CRITICAL
• The logging module gives you with a default logger which permits you to get started
without needing to do much configuration.

(Continued)
methods for each level can be called as shown in the following example:

Module 2: Data Analysis using Python

Pandas
Series
• A one-dimensional array which is capable of holding data of any type such as python
objects, float, string, integer etc.) is known as series. The axis labels are collectively
called index
• Following constructor is used to create pandas series
pandas.Series( data, index, dtype, copy)

• Various inputs such as Array, Dict, scalar value or consultant are used to create series

Pandas
Create an Empty Series
• A basic series, which can be created is an Empty Series
Output

Pandas
Create a Series from ndarray
• Index passed must be of the same length if an data is an ndarray. By default index will
be range(n) where n is an array length i.e [0,1,2,3…. Range(len(array))-1) when no
index is passed
Example:
Output

Pandas
Create a Series from dict
• If no index is specified then dict can be passed as input. After that, dictionary keys are
taken in a sorted order to construct index. The values in data corresponding to the
labels in the index will be pulled out if index is passed
Example:
Output

Pandas
Create a Series from Scalar
• An index must be provided if data is a scalar value. To match the length of index the
value will be repeated
Example:
Output

Pandas
DataFrame
• A two dimensional data structure is known as data frame i.e., data is aligned in a
tabular fashion in rows as well as columns
Output

Pandas
Create an Empty DataFrame
• A basic DataFrame, which can be created is an Empty Dataframe.
Example:
Output

Pandas
Create a DataFrame from Lists
• The DataFrame can be created using a single list or a list of lists.
Example:
Output

Pandas
Create a DataFrame from Dict of ndarrays / Lists
• All the ndarrays should be of same length. The length of the index should equal to the
length of the array, if index is passed
• By default index will be range(n), where n is the array length if no index is passed
Example:
Output

Pandas
Create a DataFrame from List of Dicts
• To create a DataFrame list of dictionaries can be passed as input data. The dictionary
keys are by default taken as column names
Example:
Output

Pandas
Create a DataFrame from Dict of Series
• To form a DataFrame a series of dictionary can be passed. The resultant is the union of
all the series indexes passed
Example:
Output

Pandas
Column Selection
• We will understand this by selecting a column from the DataFrame.
Example:
Output

Pandas
Column Addition
• This will be understand by you by adding a new column to an existing data frame
Example:

Pandas
Column Deletion
• Following example is showing that how columns can be deleted or popped
Example:

Pandas
Row Selection, Addition, and Deletion
Selection by integer
Selection by Label Slice Rows
location
Deletion of Rows Addition of Rows

Pandas
Selection by Label
• By passing row label to a loc function rows can be selected
Example:
Output

Pandas
Selection by integer location
• By passing integer to an loc function rows can be selected
Example:
Output

Pandas
Slice Rows
• By using ‘ : ’ operator multiple rows can be selected
Example:
Output

Pandas
Addition of Rows
• By using the append function news rows are added. This function will append the rows
at the end
Example:
Output

Pandas
Steps to read data from CSV file
Step 1: Create two excel files which contains some data as you need
Step 2: Now, convert these excel files into CSV format as shown in the following figure

Pandas
Step 3: Open Jupyter as shown in the following figure and open that folder where you
have saved your excel sheets as we have stored in desktop folder

Pandas
Step 4: As you can see in the following figure excel files are showing

Pandas
Step 5: Now click on New button. After that, click on Python 3

Pandas
Step 6: Write this code to read data from CSV files
Name of CSV file

Pandas
Step 7: You will get this output

Pandas
Group By
Step 1: Create a CSV file as we have created a CSV file named “sheet1.csv”
Step 2: Write a code to open CSV file as shown in the following figure

Pandas
Step 3: Result will show as shown in the following figure

Pandas
Step 4: Now write a code to select a particular column according to which you want to
group data as we have selected “city” column to make groups of the data

Pandas
Step 5: Code to show groups in the data

Pandas
Step 6: Code to show data of ‘Mumbai’ city only

Pandas
Step 7: Code to check maximum values from the data

Pandas
Step 8: Code to check average or mean values from the data

Pandas
Step 9: Code to check average or mean values from the data

Pandas
Step 10: Code to visualize the data in the form of graphs

Pandas
Step 11: Data visualization of Mumbai city

Pandas
Step 12: Data visualization of new York city

Pandas
Step 13: Data visualization of Paris city

Pandas
Concatenation
Step 1: Wire code two make two different tables to which you want to concatenate

Pandas
(Continued)

Pandas
Step 2: Code to concatenate two table as show in the following figure

Pandas
Step 3: Code to concatenate two tables by using another way

Pandas
Merge
Step 1: Create two tables to which you want to merge

Pandas
(Continued)

Pandas
Step: Code to merge two tables

Pandas
Operations over rows/column
Step 1: Create a table according to your need as we have created in the following figure

Pandas
Step 2: Code to get total of particular columns as shown in the following figure. As you can
see in the figure a column named total is added in the table

Pandas
Step 3: Code to drop a particular column as you can see in the following figure a column
named total is dropped. This code will not drop this column from the original table

Pandas
Step 4: Code to drop a column from the original table

Pandas
Step 5: Code to drop more than one column from the table

Pandas
Step 6: Code to drop a row as we have dropped 0 row from the table you can see in the
following figure

Pandas
Step 7: Code to add a particular row in the table. As you can see in the following figure o
row is added at the end

Pandas
Indexing
Step 1: Create a table according to your need as shown in the following figure

Pandas
Step 2: When you want to access data from the table on the basics of labels (whether it is
row label or column label in that case we you loc as shown in the following figure

Pandas
Step 3: Code for accessing more than one record

Pandas
Step 4: Code to access of more than one record with a particular column as shown in the
following figure

Module 3: Numpy

Numpy
Numpy Array
• Numpy is a core library for scientific computing and it stands for ‘Numerical Python’.
Numpy contains a powerful n-dimensional array object which provides tools for
integrating c, c++ etc
• It is also useful in linear algebra and random number capability etc.
• For generic data Numpy array can also be used as an efficient multi-dimensional
container.

Numpy
(Continued)
• Numpy array is a N-dimensional array object in the form of rows and columns. We can
initialize Numpy arrays from nested Python lists and access it elements.
• Numpy’s main object is the homogenous multidimensional array
• It is a table of elements which are of same type and indexed by a tuple of positive
integers
• In NumPy dimensions are called axes. The number of axes is rank
• NumPy’s array class is called ndarray. Even it is known by the alias array

Numpy
(Continued)
Example to create a 1D Numpy array

Numpy
(Continued)
Example to create 2D Numpy Array

Numpy
(Continued)
• Code to check type of the Numpy array
• Code to check no. of rows and columns

Numpy
(Continued)
• Code to check data type

Module 4: Visualization

Visualization
Matplotlib
• Matplotlib is a Python 2D plotting library which make publication quality figures in a

diversity of hardcopy formats aa well as interactive environments across platforms
• Matplotlib can be used in Python scripts, the Python along with IPython shells, the
Jupyter notebook, web application servers as well as four graphical user interface
toolkits.
• Matplotlib tries to make hard things possible as well as easy things easy
• With few lines of code you can generate plots, power spectra, histograms, bar charts,
scatterplots and error charts

Visualization
(Continued)
• The pyplot module provides a MATLAB-like interface for simple plotting, specially when
combined with python
• you have full control of, line styles, font properties, axes properties etc, through an
object oriented interface or through a collection of functions familiar to MATLAB users.

Visualization
(Continued)
Example:

Visualization
Seaborn
• In python, seaborn is a library for making statistical graphcis. It is built on top of

Matplotlib as well as closely integrated with pandas data structures. However, Seaborn
comes with some very essential features
features help in −
• Built in themes for styling matplotlib graphics
• It seaborn works fine with Numpy as well as pandas data structures
• Fitting in as well as visualizing linear regression models

Visualization
(Continued)
• Plotting statistical time series data
• Features of seaborn helps in visualizing univariate as well as bivariate data

Visualization
(Continued)
Example 1: Seaborn bar plot

Visualization
(Continued)

Visualization
(Continued)

Visualization
Sample visualization using DataFrame
Step 1: Create a table according to your need and data of which you want to visualize in
the form of graphs

Visualization
Step 2: Code to visualize the data in graph

Visualization
Sample visualization using Numpy Array
Step 1: Create a list k as shown in figure and pass it through the plot to get visualization
through graph of this list. As we have given label of y axis similarly you can give x axis label
as well

Module 5: Basic Introduction

Basic Introduction
Data Science
• Data science is field of study which includes extraction of meaningful information form
the large amount of data by using several scientific methods, algorithms as well as
processes.
• It helps us to find unknown patterns form the raw data.
• Data science has developed because of the evolution of mathematical statistics, big
data as well as data analysis
• Data science allows you for transmitting a business problem into a research project and
then convert it back into a practical solution

Basic Introduction
Machine Learning
• A system which can learn from example through self-improvement as well as without
being explicitly coded by programmer is an example of machine learning
• Machine learning combines data with statistical tools to predict an output.
• This output is then used by corporate to makes actionable insights.
• Machine learning is related to Bayesian predictive modeling as well as it is related with

data mining as well
• The machine receives data as input, use an algorithm to formulate answers.

Basic Introduction
(Continued)
• Fraud detection, predictive maintenance, portfolio optimization, automatize and many

more is also done with the help of machine learning
• Machine learning is an amazing tool to analyze, understand as well as to recognize a

pattern in the data.
• In machine learning best thing is that the computer can be trained to automate tasks
which are impossible for an human being

Basic Introduction
Deep learning
• A computer software that acts as the network of neurons in a brain is called deep
learning.
• It is called deep learning because it makes use of deep neural network. Even though, it
is a part of machine learning
• Algorithms in deep learning are created with connected layers
o The Input layer is the first layer
o The Output Layer is the last layer

Basic Introduction
(Continued)
• All the layers which comes between these two layers are called as Hidden layers. The
word deep means the network join neurons in more than two layers
• Each Hidden layer is made of neurons.
• The neurons are linked with each other.
• The neuron will process and then propagate the input signal which is received by the
layer above it

Basic Introduction
(Continued)
• The strength of the signal which is given to the neuron in the next layer depend on the
activation function, bias along with weight
• Large amount of data is consumed by the network and operates them through
multiple layers; the network can learn more and more complex features of the data at
each layer

Basic Introduction
Introduction to End to End Pipeline in machine learning
• When machine learning algorithms runs it contains a sequence of tasks which includes
pre-processing, feature extraction, model fitting as well as validation stages
• Such as when classifying text documents may involve text segmentation as well as
cleaning, extracting features, and training a classification model with cross validation
• Even though there are many libraries which we can use for each stage, in large amount
of datasets connecting the dots is not as easy as it may look

Basic Introduction
(Continued)
• Most machine learning are not designed for distributed computation either they do
not provide any maintenance for pipeline creation as well as tuning
• The machine learning pipeline is a high-level API for Mllib which lives under the
“spark.ml” package
• A pipeline be made of a sequence of stages
• Transformer as well as Estimator both are the basic types of pipeline stages

Basic Introduction
(Continued)
• A dataset is got as input by the transformer as well as generates an augmented dataset

as output
• Such as tokenizer is a transformer which transforms a dataset with text into an dataset
with tokenized words
• To procedure a model, an estimator must be first fit on the input dataset. Which is a
transformer that transforms the input dataset for example logistic regression is an
estimator which trains on a dataset with labels as well as features and produces a
logistic regression

Module 5: Web implementation

Web implementation
Django vs Flask
Type of Web Framework
• Django is full-stack python web framework. Batteries included approach is used to

develop it
• As Django contains batteries which makes it easier for Django developers to complete
common web development tasks such as user authentication, URL routing as well as
database schema migration
• By providing built-in template engine, ORM system as well as bootstrapping tool

custom web application is accelerated by the Django as well

Web implementation
(Continued)
• Whereas, flash is lightweight, simple as well as minimalist web framework. It does not
have some built-in features as provided by the Django
• To keep the core of a web application simple as well as extensible it is very helpful for
developers
Functional Admin Interface
• Not like flash, By providing a ready-to-use admin framework Django makes it easier for
users to manage common project administration tasks

Web implementation
(Continued)
• Based on project models it further produces the functional admin module,

automatically
• The developers even have option to customize the admin interface to meet particular
business requirements
• To simplify website content administration as well as user management they can even
take advantage of the admin interface

Web implementation
(Continued)
• Django’s functional admin user interface makes it standout in the crowd
Template Engine
• Flash is made based on jinja2 template engine. Jinja2 is also inspired by Django’s
template system as a fully featured template engine for python
• By taking the advantage of an integrated sandbox execution environment as well as

templates in an expressive language it enables developers to accelerate development
of dynamic web applications

Web implementation
(Continued)
• To define a web application’s user facing layer without putting extra time as well as
efforts Django comes with a built-in template engine which enables the developers to
do this
• By writing templates in Django template language (DTL) it also allows developers to

accelerate custom user interface development

Web implementation
Built-in Bootstrapping Tool
• Not like flash, Django has a built-in bootstrapping tool named Django-admin
• Django-admin permits developers to start making web applications without any

external input
• Django also permits developers to divide a single project into a number of applications
• To create new applications with the project Django-admin can be used by the
developers. To add functionality to the web applications based on varied business
requirements they can further use the applications

Web implementation
Database Support
• Django permits developers to take advantage of a robust ORM system
• To work with widely used databases such as MySQL, Oracle, SQLITE as well as
PostgreSQL developers can use the ORM system
• To perform common database operations without writing lengthy SQL queries the
ORM system also permit the developers
• In contrast Django, flash does not have a built-in ORM system. It requires developers to
work with databases and perform database operations via SQLAlchemy.

Web implementation
(Continued)
• By using SQLAlchemy as SQL toolkit as well as ORM system for python developers can
work with databases
• Moreover, developers can perform common database queries by writing as well as

executing SQL queries

Web implementation
Project Layout
• Flask requires developers to each project as a single application. While, the developers
can add multiple models as well as views to the same application. In contrast, Django
allows developers to divide a project into multiple applications
• Henceforth, it becomes easier for developers to write single applications as well as add
functionality to the web application by integrating the applications into the project
• The small applications are further helpful for developers to extend as well as maintain
the web applications written in python

Web implementation
Django architecture based on MVC pattern
• Django MVC architecture provides solution of lots of problems which were faced in
traditional approach for web development
• On the internet for every website there are three key components either code
partitions which are Input Logic, Business Logic as well as UI Logic
• The input logic is the dataset as well as how the data gets to organized in the database
these are the particular tasks to achieve by these partitions of code
• It just takes that input as well as sends it to the database in the wanted format

Web implementation
(Continued)
• The output from the server in the HTML either desired format is managed by the
business logic which is the key controller
• The UI Logic as the name suggests are the HTML, CSS and JavaScript pages.
• All the code was written in the single file when traditional approach was used. Every
piece of code increases the webpage size, which is downloaded as well as rendered by
the browser

Web implementation
(Continued)
• It was not a big issue in the prior time, the webpages were largely static and websites
as well as did not contain much multimedia and large coding
• Even, this architecture makes difficulty for developers during testing as well as
maintaining the project as everything is inside one file
• MVC is a product development architecture. Traditional approach’s drawback of code

in one file is solved by it i.e., that in MVC for different aspects of our web application
has different files

Web implementation
(Continued)
• The MVC pattern has three components, namely Model, View, as well as Controller
Controller
Model View

Web implementation
(Continued)
• This difference between the components helps the developers to concentrate on one
aspect of the web- application as well as therefore, better code for one functionality
better testing, scalability as well as debugging
working cycle of Django MVC architecture
1. Model
• It is the part of the web application what acts as a mediator in the middle of the
website interface as well as the database

Web implementation
(Continued)
• In technical terms, it is the object which implements the logic for the application’s
data domain
• There are times, when data from a particular dataset may be taken by the application
as well as directly send it to the view without needing any database then the dataset is
considered as a model
• We need to have some kind of database as we must be requiring some user input even
though we are creating a simple blog site if we want any sort of website

Web implementation
(Continued)
The Model is the component which contains Business Logic in Django architecture
For example
• You are actually sending information to the controller which then transfers it to the
models which in turn applies business logic on it as well as store in the database, when
you sign up on any website

Web implementation
(Continued)
2. View
• View component has the UI logic in the Django architecture
• User interface of the web-application is known as view as well as it contains the parts
such as HTML, CSS and frontend technologies. Commonly, this UI creates from the
models components i.e., the content comes from the Models components

Web implementation
(Continued)
For example
• When you interact with the website either click on any link, the new webpages that
website produces is actually the particular views which stores as well as generates
when we are interacting with the specific components
• The controller is the main control component means controller manages the user
interaction as well as selects a view according to the model
• The prime task of the controller is to select a view component according to the user
interaction as well as apply the model component as well

Web implementation
(Continued)
• Django is based on this architecture because it has lots of advantages. Moreover, it

takes the same model to an advanced level
For example
• After combining two previous examples, we can very clearly see that the component
which is actually selecting diverse views as well as transferring the data to the model’s
component is the controller

Congratulations
Congratulations on completing this course!

Keep in touch
info@theknowledgeacademy.com
Thank you

Python Advanced: © 2020 The Knowledge Academy LTD

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Python Advanced: © 2020 The Knowledge Academy LTD

Uploaded by

Copyright:

Available Formats

Python Advanced

© 2020 The Knowledge Academy Ltd 1

© 2020 The Knowledge Academy Ltd 2

© 2020 The Knowledge Academy Ltd 3

• Module 2: Data Analysis using

• Module 3: Web implementation

© 2020 The Knowledge Academy Ltd 4

© 2020 The Knowledge Academy Ltd 5

• It is used for system scripting, software development and web-development (server-

• It works on different platforms(Windows, Linux, Mac)

• It is a object- oriented language and runs on an interpreter system

© 2020 The Knowledge Academy Ltd 6

• It is used for system scripting, software development and web-development (server-

• It works on different platforms(Windows, Linux, Mac)

• It is a object- oriented language and runs on an interpreter system

© 2020 The Knowledge Academy Ltd 7

• Python uses an automatic method to allocate and deallocate the memory.

• Python follows two ways for memory allocation:

© 2020 The Knowledge Academy Ltd 8

• Reference counting works by counting the number of times an object is referenced by

• Reference count form an object is decremented, when references to an object are

© 2020 The Knowledge Academy Ltd 9

© 2020 The Knowledge Academy Ltd 10

© 2020 The Knowledge Academy Ltd 11

© 2020 The Knowledge Academy Ltd 12

© 2020 The Knowledge Academy Ltd 13

Automatic Garbage Collection of Cycles

© 2020 The Knowledge Academy Ltd 14

• Python schedules garbage collection on the basis of a threshold of an object

© 2020 The Knowledge Academy Ltd 15

© 2020 The Knowledge Academy Ltd 16

• In the previous slide example, 700 is the default threshold

© 2020 The Knowledge Academy Ltd 17

• Functions can be stand-alone ex: type(S)

• User-defined functions can be created as well

© 2020 The Knowledge Academy Ltd 18

© 2020 The Knowledge Academy Ltd 19

• A Function can be called many time

© 2020 The Knowledge Academy Ltd 20

© 2020 The Knowledge Academy Ltd 21

• The process of passing reference of an argument in the calling function to the

© 2020 The Knowledge Academy Ltd 22

© 2020 The Knowledge Academy Ltd 23

Required arguments Default arguments

Keyword arguments Variable-length arguments

© 2020 The Knowledge Academy Ltd 24

© 2020 The Knowledge Academy Ltd 25

© 2020 The Knowledge Academy Ltd 26

© 2020 The Knowledge Academy Ltd 27

© 2020 The Knowledge Academy Ltd 28

© 2020 The Knowledge Academy Ltd 29

• Variable length argument allows a function to receive any number of arguments. It is

© 2020 The Knowledge Academy Ltd 30

© 2020 The Knowledge Academy Ltd 31

© 2020 The Knowledge Academy Ltd 32

© 2020 The Knowledge Academy Ltd 33

• The are two terms involved in generators:

© 2020 The Knowledge Academy Ltd 34

• A generator function is defined as a normal function, It uses yield keyword in spite of

© 2020 The Knowledge Academy Ltd 35

© 2020 The Knowledge Academy Ltd 36

• Decorates allows programmers to change the behavior of function or class so that’s

© 2020 The Knowledge Academy Ltd 37