Batch 9

IMPLEMENTATION OF DEEP LEARNING ALGORITHM
WITH PERCEPTRON USING TENSORFLOW LIBRARY

A Project Report
Submitted to
JNTUA, Ananthapuramu
In partial fulfillment of the requirements for the award of the degree of
Bachelor of Technology
in
COMPUTER SCIENCE & ENGINEERING
Submitted by
N.G. CHAITANYA REDDY - 164E1A0504

KORRAKUTI MURALI - 164E1A0522
A. MONISHA - 164E1A0521
P. JUVERIA AJUM - 164E1A0511
Under the esteemed guidance of

Mrs. M. AMUDHA, M.Tech.
Assistant Professor of Computer Science & Engineering
Department of Computer Science & Engineering

SIDDARTHA INSTITUTE OF SCIENCE and TECHNOLOGY
(AUTONOMOUS)
(Approved by AICTE & Affiliated to JNTUA, Ananthapuramu)
(Accredited by NBA for EEE, ECE, MECH & CSE)
(Accredited by NAAC with ‘A’ Grade)
Siddharth Nagar, Narayanavanam Road, Puttur-517583, A.P
2016 -2020
DEPARTMENT OF COMPUTER SCIENCE & ENGINEERING
SIDDARTHA INSTITUTE OF SCIENCE and TECHNOLOGY

(AUTONOMOUS)
(Approved by AICTE & Affiliated to JNTUA, Ananthapuramu)
(Accredited by NBA for EEE, ECE, MECH & CSE)
(Accredited by NAAC with ‘A’ Grade)
Siddharth Nagar, Narayanavanam Road, Puttur-517583, A.P
CERTIFICATE
This is to certify that the Project entitled “LOW- COST CONCRETE BY PARTIAL
REPLACEMENT OF FINE AGGREGATE WITH SAWDUST” is being submitted
by
N.G. CHAITANYA REDDY - 164E1A0504
KORRAKUTI MURALI - 164E1A0522
A. MONISHA - 164E1A0521
P. JUVERIA AJUM - 164E1A0511
in partial fulfillment of the requirements for the award of BACHELOR OF
TECHNOLOGY in COMPUTER SCIENCE & ENGINEERING to JNTUA,
Ananthapuramu. This project work or part thereof has not been submitted to any
other University or Institute for the award of any degree.
Head of the Department Guide

Dr. D. William Albert, M.Tech, Ph.D. Mrs. M Amudha, M.Tech.
Professor & Head Assistant Professor of CSE
Department of CSE
Project viva-voce examination held on ______________
INTERNAL EXAMINER EXTERNAL EXAMINER

ACKNOWLEDGEMENT
We wish to express our profound and sincere gratitude to Mrs. M. Amudha,

Associate Professor of Computer Science & Engineering, Siddartha Institute of
Science and Technology, Puttur, who guided us into the intricacies of this project
with utmost clarity.
We express our deep sense of gratitude to Project Coordinator Mr. M. Doorvasulu

Naidu, Associate Professor of CSE, who evinced keen interest in our effects and
provided his valuable guidance throughout our project work.
We would also like to extend our gratitude to Dr. D. William Albert, Head of the
Computer Science & Engineering Department for his encouragement and for
providing the facilities to carry out the work in a successful manner.
We are thankful to Dr. M. Janardhana Raju, Principal for his encouragement and
support.
We wish to express our sincere thanks to Dr. K. Indiraveni, Vice-Chairman, and Dr.
K. Ashok Raju, Chairman of Siddharth Group of Institutions, Puttur, for providing
ample facilities to complete the project work.
We would also like to thank all the faculty and staff of the Computer Science &
Engineering Department, for helping us to complete the project work.
Very importantly, we would like to place on record our profound indebtedness to our
parents and families for their substantial moral support and encouragement
throughout our studies.
CONTENTS
Chapter No Title Page. No
DEDICATIONS i
QUOTATIONS ii
LIST OF FIGURES iii
ABSTRACT v
1. INTRODUCTION
1.1 Domain Description 1

1.2 About the Project 19
2. SYSTEM ANALYSIS
2.1 Problem Statement 23
2.2 Problem Description 23
2.3 Existing System 23
2.4 Disadvantages of Existing System 24
2.5 Proposed System 24
2.6 Advantages of Proposed System 24
3. SYSTEM REQUIREMENTS SPECIFICATION
3.1 Hardware Requirements 25

3.2 Software Requirements 25
3.3 Feasibility Study 25
4. LITERATURE SURVEY 27
5. SYSTEM DESIGN
5.1 System Architecture 31
5.2 Modules 32
5.2.1 Data Understanding 32
5.2.2 Exploratory Data Analysis 32
5.2.3 Tensor Flow 33
5.2.4 Machine Learning Models 33
5.2.5 Deep Learning Models 33
5.2.6 Validation 34
5.3 Algorithm 34
5.4 Introduction to UML 38
5.5 UML diagrams 39
5.5.1 Class diagram 41

5.5.2 Use case Diagram 42
5.5.3 Sequence Diagram 43
5.5.4 Activity Diagram 44
5.5.5 Deployment Diagram 45
6. SYSTEM IMPLEMENTATION
6.1 Software Description 46
6.1.1 Introduction to Google Colaboratory 46
6.1.2 Introduction to jupyter Notebook 49
6.2 Python Libraries for data Science 55
6.4 Sample code 58
7. SYSTEM TESTING
7.1 Validation 62
7.2 Types of Validation 62
7.2.1 Validation 63
7.2.2 Leave One Out Cross Validation 63
7.2.3 KFold Cross Validation 63
7.3 Cross Validating 65
8. RESULTS
8.1 Execution procedure 67
8.2 Screen Shots 67
9. CONCLUSION AND FUTURE ENHANCEMENT

9.1 Conclusion 74
9.2 Future Enhancement 74
10. BIBLIOGRAPHY
10.1 References 75
10.2 Websites 75
10.3 Text Books 75
11. BASE PAPER

DEDICATED
TO
“Our Parents, Faculty members

And all others who cared
For us in Fulfilling this Project”
i
QUOTATIONS
1. IT WOULD BE BETTER IF YOU BEGIN TO EACH

OTHERS ONLY AFTER YOU YOURSELF HAVE
LEARNED SOMETHING.
-ALBERT EINSTEN
2. THINKING SHOULD BECOME YOUR CAPTIAL

ASSETS, NO MATTER WAHTEVER UP’S AND
DOWN YOU COME ACROSS IN YOUR LIFE.
-APJ ABDUL KALAM
3. DETERMINATION AND CONFIDENCE ARE LIKE
AIR AND WATER FOR SURVIVAL OF PEOPLE.
-ROBERT WILLAM
4. EDUCATION IS THE ABILITY TO LISTEN TO
ALMOST ANYTHING WITHOUT LOSING YOUR
TEMPER OR YOUR SELF- CONFIDENCE.
-ROBERT FROST
ii
LIST OF FIGURES

1.1(a) Various Processes in Data Science 05
1.1(b) Appending Drive to google colab 18
1.2(a) Machine Learning v/s Deep Learning 20
1.2(b) Neural Network 21
1.2(c) Perceptron rule 22
5.1(a) System Architecture 31
5.1(b) Block diagram 32
5.3(a) Perceptron 34
5.3(b) Multi-Layer Perceptron 35
5.3(c) Forward Pass 37
5.5.1 Class Diagram 41
5.5.2 Use case Diagram 42
5.5.3 Sequence Diagram 43
5.5.4 Activity Diagram 44
5.5.5 Deployment Diagram 45
7.2.3 LOOCV Method 64
iii
LIST OF FIGURES
7.3(a) Validation 65
7.3(b) KFold cross validation 66
8.2.1-8.2.11 Screenshots 67
iv
ABSTRACT
As of late, Deep Learning, Machine Learning, and Artificial Intelligence are profoundly
engaged ideas of information science. Deep learning has made progress in the field of Computer
Vision, Speech and Audio Processing, and Natural Language Processing. It has the solid learning
capacity that can improve use of datasets for the information extraction contrasted with
conventional Machine Learning Algorithm. Perceptron is the fundamental structure hinder for
making a profound Neural Network, it breaks down the unsupervised data, making it an important
instrument for information analytics. A key assignment of this paper is to create and analyze
Deep learning Algorithm and compared with the conventional Machine learning Algorithms,
Model starts with Deep learning with perceptron and how to apply it to comprehend different
issues in non-separable information. The fundamental piece of this paper is to make perceptron
learning Algorithm respectful with non-separable training datasets when compared with the
conventional Machine learning Algorithms.
v
Implementation of Deep Learning Algorithm with Perceptron using TensorFlow
Library
1. INTRODUCTION
1.1 Domain Description

Data science is an inter-disciplinary field that uses scientific methods, processes,
algorithms and systems to extract knowledge and insights from many structural
and unstructured data. Data science is related to data mining and big data.
Data science is a "concept to unify statistics, data analysis, machine learning and
their related methods" in order to "understand and analyze actual phenomena" with
data.[3] It employs techniques and theories drawn from many fields within the context
of mathematics, statistics, computer science, and information science. Turing
award winner Jim Gray imagined data science as a "fourth paradigm" of science
(empirical, theoretical, computational and now data-driven) and asserted that "everything
about science is changing because of the impact of information technology" and the data
deluge. In 2015, the American Statistical Association identified database management,
statistics and machine learning, and distributed and parallel systems as the three emerging
foundational professional communities.
Historical overview
The term "data science" has appeared in various contexts over the past thirty years
but did not become an established term until recently. In an early usage, it was used as a
substitute for computer science by Peter Naur in 1960. Naur later introduced the term
"datalogy". In 1974, Naur published Concise Survey of Computer Methods, which freely
used the term data science in its survey of the contemporary data processing methods that
are used in a wide range of applications.
The modern definition of "data science" was first sketched during the second
Japanese-French statistics symposium organized at the University of Montpellier
II (France) in 1992. The attendees acknowledged the emergence of a new discipline with a
SISTK DEPARTMENT OF CSE Page 1

Library
specific focus on data from various origins, dimensions, types and structures. They shaped
the contour of this new science based on established concepts and principles of statistics
and data analysis with the extensive use of the increasing power of computer tools.
In 1996, members of the International Federation of Classification Societies (IFCS)

met in Kobe for their biennial conference. Here, for the first time, the term data science is
included in the title of the conference ("Data Science, classification, and related methods"),
after the term was introduced in a roundtable discussion by Chikio Hayashi.
In November 1997, C.F. Jeff Wu gave the inaugural lecture entitled "Statistics =
Data Science?" for his appointment to the H. C. Carver Professorship at the University of
Michigan. In this lecture, he characterized statistical work as a trilogy of data collection,
data modeling and analysis, and decision making. In his conclusion, he initiated the
modern, non-computer science, usage of the term "data science" and advocated that
statistics be renamed data science and statisticians data scientists. Later, he presented his
lecture entitled "Statistics = Data Science?" as the first of his 1998 P.C. Mahalanobis
Memorial Lectures. These lectures honor Prasanta Chandra Mahalanobis, an Indian
scientist and statistician and founder of the Indian Statistical Institute.
In 2001, William S. Cleveland introduced data science as an independent

discipline, extending the field of statistics to incorporate "advances in computing with data"
in his article "Data Science: An Action Plan for Expanding the Technical Areas of the Field
of Statistics," which was published in Volume 69, No. 1, of the April 2001 edition of the
International Statistical Review / Revue “Internationale de Statistique”. In his report,
Cleveland establishes six technical areas which he believed to encompass the field of data
science: multidisciplinary investigations, models and methods for data, computing with
data, pedagogy, tool evaluation, and theory.
In April 2002, the International Council for Science (ICSU): Committee on Data
for Science and Technology (CODATA) started the Data Science Journal, a publication
focused on issues such as the description of data systems, their publication on the internet,

Library
applications and legal issues. Shortly thereafter, in January 2003, Columbia University
began publishing The Journal of Data Science, which provided a platform for all data
workers to present their views and exchange ideas. The journal was largely devoted to the
application of statistical methods and quantitative research. In 2005, The National Science
Board published "Long-lived Digital Data Collections: Enabling Research and Education
in the 21st Century" defining data scientists as "the information and computer scientists,
database and software and programmers, disciplinary experts, curators and expert
annotators, librarians, archivists, and others, who are crucial to the successful management
of a digital data collection" whose primary activity is to "conduct creative inquiry and
analysis."
Around 2007, Turing award winner Jim Gray envisioned "data-driven science" as
a "fourth paradigm" of science that uses the computational analysis of large data as primary
scientific method and "to have a world in which all of the science literature is online, all of
the science data is online, and they interoperate with each other."
In the 2012 Harvard Business Review article "Data Scientist: The Sexiest Job of
the 21st Century", DJ Patil claims to have coined this term in 2008 with Jeff
Hammerbacher to define their jobs at LinkedIn and Facebook, respectively. He asserts that
a data scientist is "a new breed", and that a "shortage of data scientists is becoming a serious
constraint in some sectors” but describes a much more business-oriented role.
In 2013, the IEEE Task Force on Data Science and Advanced Analytics was
launched. In 2013, the first "European Conference on Data Analysis (ECDA)" was
organized in Luxembourg, establishing the European Association for Data Science
(EuADS). The first international conference: IEEE International Conference on Data
Science and Advanced Analytics was launched in 2014.
In 2014, General Assembly launched student-paid bootcamp and The Data

Incubator launched a competitive free data science fellowship. In 2014, the American
Statistical Association section on Statistical Learning and Data Mining renamed its journal

Library
to "Statistical Analysis and Data Mining: The ASA Data Science Journal" and in 2016
changed its section name to "Statistical Learning and Data Science". In 2015, the
International Journal on Data Science and Analytics was launched by Springer to publish
original work on data science and big data analytics. In September 2015 the Gesellschaft
für Klassifikation (GfKl) added to the name of the Society "Data Science Society" at the
third ECDA conference at the University of Essex, Colchester, UK.
Uses of Data Science
Data science can be used to assist minimize costs, discover new markets and make
better decisions. It will be a bit difficult to pinpoint a single or specific example to make
one understand about Data Science since it mainly works in confluence with other
variables.
There are many algorithms and methods that data experts analyse to help the
managers and directors extend their companies bottom line and strategic positioning. Data
science techniques are incredible at spotting abnormalities, optimizing constraint
problems, predicting and targeting.
For example, data scientists can analyse the data on food habits of the people in a
certain area and can help in identifying the location and type of food joint that would be
successful. They can also predict the amount of sale a particular food joint will earn there.
Various Processes in Data Science
Data science pipelines are sequences of obtaining, cleaning, visualizing, modelling

and interpreting data for a particular purpose. They’re useful in production projects, and
they can also be beneficial if one expects to come across the identical kind of business
question in the future, so as to retailer on design time and coding.

Library
• Obtaining data
• Cleaning or Scrubbing data
• Visualizing and Exploring data will allow us to find patterns and trends
• Modelling data will give us our predictive power as a data magician
• Interpreting data
Fig 1.1(a): Various processes in data science
Obtaining Data
Before delving into any research, we need to collect or obtain our data. We need
to first identify our strong reasons as to why we are collecting data and if we do what will
our sources be? It can be internal or external, there can be an existing database which has
to re-analyzed, create a new database, or use both. Here, the Database Management
System comes into play.
A database management system (DBMS) is a system software for creating and

managing databases. The DBMS presents users and programmers with a systematic way
to create, retrieve, update and control data.

Library
There are two different kinds of DBMS- SQL and NoSQL database. SQL
databases are particularly known as Relational Databases (RDBMS) which characterize
information in form of tables which consists of n range of rows of records.
On the other hand, NoSQL databases are especially called as non-relational or

distributed database which is document based consisting of a series of a key-value pair,
documents, graph databases or wide-column stores.
SQL databases have predefined schema whereas NoSQL databases have a

dynamic schema for unstructured data.
SQL is more conventionally used data based, however, NoSQL has become more
popular in recent years.
Process
• Going through both SQL and NoSQL database and identify one that you need
to follow
• Querying these databases
• Retrieving unstructured data in the form of videos, audios, texts, documents,
etc.
• Storing the information
Cleaning Data
Data cleaning or Data Scrubbing is the technique of detecting and correcting (or
removing) corrupt or inaccurate pieces of information from a record set, table, or
database. It helps in figuring out incomplete, incorrect, inaccurate or irrelevant parts of
the data and then replacing, modifying, or deleting the soiled or coarse data.

Library
Cleaning helps in removing differences or anomalies in data and fills in
empty/missing values, thus making records consistent and fit for further processing.
To perform this action an adept need to have a thorough knowledge of Scripting

and Data wrangling tools like Python & R.
Data Visualization
Once your data is preprocessed, now you have the matter to work upon. Though
the data is clean it is still raw, and it will be hard to convey the hidden meaning behind
those numbers and statistics to the end client. Data Visualization is like a visual
interpretation of the raw data with the help of statistical graphics, plots, information
graphics and other tools.
According to Vitaly Friedman (2008), the “main goal of data visualization is to

communicate information clearly and effectively through graphical means. It does not
mean that data visualization needs to look boring to be functional or extremely
sophisticated to look beautiful.
To convey ideas effectively, both aesthetic form and functionality need to go hand
in hand, providing insights into a rather sparse and complex data set by communicating
its key-aspects in a more intuitive way.
Data Visualization is extremely important as it is an easy and a comprehensive

way to convey the central message to the end users. To execute this process perfectly one
needs to possess certain pre-requisite knowledge-
• Python libraries: Numpy, Matplotlib, Pandas, Scipy

• R libraries: GGplot2, Dplyr

Library
• Inferential statistics
• Data Visualisation
• Experimental design
Data Modelling
Data modelling is used to define and analyze data requirements, it is the framework of
what the relations are within the database. A data model can be thought of as a flowchart
that illustrates the relationships between data. Models are simply general rules in a
statistical sense. There are quite a few special strategies for facts modelling, including:
Conceptual Data Modeling – identifies the highest-level relationships between different

entities.
Enterprise Data Modeling – comparable to conceptual records modelling but addresses

the special requirements of a unique business.
Logical Data Modeling – illustrates the unique entities, attributes and relationships
involved in an enterprise function. Serves as the basis for the creation of the physical
records model. Physical Data Modeling represents an application and database-specific
implementation of a logical information model, One ought to have this particular skill-set
to go about it.
• Machine Learning: Supervised/Unsupervised/Reinforcement learning algorithms
• Evaluation methods
• Machine Learning Libraries: Python (Sci-kit Learn) / R (CARET)
• Linear algebra & Multivariate Calculus

Library
Data Interpretation
All the above steps were restricted to your personal boundary; however, data
interpretation makes your work go public. It is supposedly the toughest of all the steps
followed above and also the culmination point.
Machine Learning
Machine learning is a subfield of artificial intelligence (AI). The goal of machine

learning generally is to understand the structure of data and fit that data into models that
can be understood and utilized by people.
Although machine learning is a field within computer science, it differs from

traditional computational approaches. In traditional computing, algorithms are sets of
explicitly programmed instructions used by computers to calculate or problem solve.
Machine learning algorithms instead allow for computers to train on data inputs and use
statistical analysis in order to output values that fall within a specific range. Because of
this, machine learning facilitates computers in building models from sample data in order
to automate decision-making processes based on data inputs.
Any technology user today has benefitted from machine learning. Facial
recognition technology allows social media platforms to help users tag and share photos of
friends. Optical character recognition (OCR) technology converts images of text into
movable type. Recommendation engines, powered by machine learning, suggest what
movies or television shows to watch next based on user preferences. Self-driving cars that
rely on machine learning to navigate may soon be available to consumers.

Library
Machine learning is a continuously developing field. Because of this, there are
some considerations to keep in mind as you work with machine learning methodologies or
analyze the impact of machine learning processes.
In this tutorial, we will investigate the common machine learning methods of

supervised and unsupervised learning, and common algorithmic approaches in machine
learning, including the k-nearest neighbor algorithm, decision tree learning, and deep
learning. We’ll explore which programming languages are most used in machine learning,
providing you with some of the positive and negative attributes of each. Additionally, we’ll
discuss biases that are perpetuated by machine learning algorithms and consider what can
be kept in mind to prevent these biases when building algorithms.
Machine Learning Methods
In machine learning, tasks are generally classified into broad categories. These
categories are based on how learning is received or how feedback on the learning is given
to the system developed.
Two of the most widely adopted machine learning methods are supervised
learning which trains algorithms based on example input and output data that is labeled by
humans, and unsupervised learning which provides the algorithm with no labeled data in
order to allow it to find structure within its input data. Let’s explore these methods in more
detail.
Supervised Learning
In supervised learning, the computer is provided with example inputs that are
labeled with their desired outputs. The purpose of this method is for the algorithm to be
able to “learn” by comparing its actual output with the “taught” outputs to find errors, and

Library
modify the model accordingly. Supervised learning therefore uses patterns to predict label
values on additional unlabeled data.
For example, with supervised learning, an algorithm may be fed data with images
of sharks labeled as fish and images of oceans labeled as water. By being trained on this
data, the supervised learning algorithm should be able to later identify unlabeled shark
images as fish and unlabeled ocean images as water. A common use case of supervised
learning is to use historical data to predict statistically likely future events. It may use
historical stock market information to anticipate upcoming fluctuations or be employed to
filter out spam emails. In supervised learning, tagged photos of dogs can be used as input
data to classify untagged photos of dogs.
Unsupervised Learning
In unsupervised learning, data is unlabeled, so the learning algorithm is left to find

commonalities among its input data. As unlabeled data are more abundant than labeled
data, machine learning methods that facilitate unsupervised learning are particularly
valuable.
The goal of unsupervised learning may be as straightforward as discovering hidden

patterns within a dataset, but it may also have a goal of feature learning, which allows the
computational machine to automatically discover the representations that are needed to
classify raw data.
Unsupervised learning is commonly used for transactional data. You may have a
large dataset of customers and their purchases, but as a human you will likely not be able
to make sense of what similar attributes can be drawn from customer profiles and their
types of purchases. With this data fed into an unsupervised learning algorithm, it may be
determined that women of a certain age range who buy unscented soaps are likely to be

Library
pregnant, and therefore a marketing campaign related to pregnancy and baby products can
be targeted to this audience in order to increase their number of purchases.
Without being told a “correct” answer, unsupervised learning methods can look at
complex data that is more expansive and seemingly unrelated in order to organize it in
potentially meaningful ways. Unsupervised learning is often used for anomaly detection
including for fraudulent credit card purchases, and recommender systems that recommend
what products to buy next. In unsupervised learning, untagged photos of dogs can be used
as input data for the algorithm to find likenesses and classify dog photos together.
Approaches
As a field, machine learning is closely related to computational statistics, so having

a background knowledge in statistics is useful for understanding and leveraging machine
learning algorithms. For those who may not have studied statistics, it can be helpful to first
define correlation and regression, as they are commonly used techniques for investigating
the relationship among quantitative variables. Correlation is a measure of association
between two variables that are not designated as either dependent or
independent. Regression at a basic level is used to examine the relationship between one
dependent and one independent variable. Because regression statistics can be used to
anticipate the dependent variable when the independent variable is known, regression
enables prediction capabilities.

Library
Approaches to machine learning are continuously being developed. For our
purposes, we’ll go through a few of the popular approaches that are being used in machine
learning.
1. Linear Regression
2. Logistic Regression
3. Decision Tree
4. SVM
5. Naive Bayes
6. kNN
7. K-Means
8. Random Forest
9. Dimensionality Reduction Algorithms
10. 10.Gradient Boosting algorithms
1. GBM
2. XGBoost
3. LightGBM
4. CatBoost
Deep Learning
Deep learning attempts to imitate how the human brain can process light and sound
stimuli into vision and hearing. A deep learning architecture is inspired by biological neural
networks and consists of multiple layers in an artificial neural network made up of
hardware and GPUs. Deep learning uses a cascade of nonlinear processing unit layers in
order to extract or transform features (or representations) of the data. The output of one
layer serves as the input of the successive layer. In deep learning, algorithms can be either
supervised and serve to classify data, or unsupervised and perform pattern analysis.

Library
Among the machine learning algorithms that are currently being used and
developed, deep learning absorbs the most data and has been able to beat humans in some
cognitive tasks. Because of these attributes, deep learning has become the approach with
significant potential in the artificial intelligence space.
Computer vision and speech recognition have both realized significant advances
from deep learning approaches. IBM Watson is a well-known example of a system that
leverages deep learning.
How Deep Learning Gets Better Results?
Deep learning uses layers of neural-network algorithms to decipher higher-level

information at other layers based on raw input data. For example, in an image-recognition
application, one layer could identify features such as sharp edges or contrasts in light, while
another could identify how different distinct shapes appear. Further, a third layer could
decipher what the image is showing. This is all achieved by learning the different ways
information from previous layers are pieced together to form distinguishable objects.
Neural-network algorithms are designed to recognize data patterns based on an

early understanding of how the human brain functions. Neural networks can help cluster
points within a large sample of data based on the similarities of its features, classify data
based on labels from previous data, and extract distinct features from data. The numerical
patterns these networks recognize are stored in vectors that depict real-world inputs. Deep
neural networks can be thought of as components of larger machine-learning applications
involving algorithms for reinforcement learning, classification, and regression.

Library
Deep learning uses self-taught learning and algorithm constructs with many hidden
layers, big data, and powerful computational resources. The algorithmic framework is
called the neural network, while the hidden layers in the network give it the moniker of
deep learning.
The Google Brain Team project and deep learning software like TensorFlow have
given further traction to the development of deep learning techniques. Such techniques are
based on mathematical functions and parameters for achieving the desired output.
Deep Learning Algorithms
1. Multilayer Perceptron Neural Network (MLPNN)

2. Backpropagation
3. Convolutional Neural Network (CNN)
4. Recurrent Neural Network (RNN)
5. Long Short-Term Memory (LSTM)
6. Generative Adversarial Network (GAN)
7. Restricted Boltzmann Machine (RBM)
8. Deep Belief Network (DBN)
Tensor Flow Library
Created by the Google Brain team, TensorFlow is an open source library for
numerical computation and large-scale machine learning. TensorFlow bundles together a
slew of machine learning and deep learning (aka neural networking) models and algorithms
and makes them useful by way of a common metaphor. It uses Python to provide a
convenient front-end API for building applications with the framework, while executing
those applications in high-performance C++.

Library
TensorFlow offers multiple levels of abstraction so you can choose the right one
for your needs. Build and train models by using the high-level Keras API, which makes
getting started with TensorFlow and machine learning easy.
If you need more flexibility, eager execution allows for immediate iteration and
intuitive debugging. For large ML training tasks, use the Distribution Strategy API for
distributed training on different hardware configurations without changing the model
definition.
Google Colaboratory Jupyter Notebook
Colaboratory is a free Jupyter notebook environment that requires no setup and runs
entirely in the cloud.
With Colaboratory you can write and execute code, save and share your analyses,
and access powerful computing resources, all for free from your browser.
Introduction
The Jupyter Notebook is an interactive computing environment that enables

users to author notebook documents that include: - Live code - Interactive widgets - Plots
- Narrative text - Equations - Images – Video.
Components
The Jupyter Notebook combines three components:
• The notebook web application: An interactive web application for writing and
running code interactively and authoring notebook documents.
• Kernels: Separate processes started by the notebook web application that runs users’
code in each language and returns output back to the notebook web application. The
kernel also handles things like computations for interactive widgets, tab completion
and introspection.

Library
• Notebook documents: Self-contained documents that contain a representation of all
content visible in the notebook web application, including inputs and outputs of the
computations, narrative text, equations, images, and rich media representations of
objects. Each notebook document has its own kernel.
Notebook web application

• Edit code in the browser, with automatic syntax highlighting, indentation, and tab
completion/introspection.
• Run code from the browser, with the results of computations attached to the code
which generated them.
• See the results of computations with rich media representations, such as HTML,
LaTeX, PNG, SVG, PDF, etc.
• Create and use interactive JavaScript widgets, which bind interactive user interface
controls and visualizations to reactive kernel side computations.
• Author narrative text using the Markdown markup language.
• Include mathematical equations using LaTeX syntax in Markdown, which are
rendered in-browser by MathJax.
Kernels
Through Jupyter’s kernel and messaging architecture, the Notebook allows code to be run
in a range of different programming languages. For each notebook document that a user
opens, the web application starts a kernel that runs the code for that notebook. Each kernel
can run code in a single programming language and there are kernels available in the
following languages:
• Python(https://github.com/ipython/ipython)
• Julia (https://github.com/JuliaLang/IJulia.jl)
• R (https://github.com/IRkernel/IRkernel)
• Ruby (https://github.com/minrk/iruby)

Library
• Haskell (https://github.com/gibiansky/IHaskell)
• Scala (https://github.com/Bridgewater/scala-notebook)
• node.js (https://gist.github.com/Carreau/4279371)
• Go (https://github.com/takluyver/igo)
The default kernel runs Python code. The notebook provides a simple way for users to pick
which of these kernels is used for a given notebook.
Each of these kernels communicate with the notebook web application and web browser
using a JSON over ZeroMQ/WebSockets message protocol that is described here. Most
users don’t need to know about these details, but it helps to understand that “kernels run
code.”
Fig 1.1(b): Appending drive to Google colab

Library
1.2 About the Project
IN recent year’s advancement in technology, there has been a lot of focus on
adaptive learning which is more fueled by machine learning and deep learning. In
practical terms, deep learning is just a subset of machine learning. In fact, deep learning
technically is machine learning and functions in a similar way (hence why the terms are
sometimes loosely interchanged). However, its capabilities are different. While basic
machine learning models do become progressively better at whatever their function is,
they still need some guidance.
If an AI algorithm returns an inaccurate prediction, then an engineer must step in

and adjust. With a deep learning model, an algorithm can determine on its own if a
prediction is accurate or not through its own neural network. It is one of the effective
methods to predict data analytics with a large amount of sample data. It also helps in
recognizing hidden pattern or features through historical learning and trends in data.
Models are created based on the past result gathered from training data, It studies
the past observation results to make precise predictions. Conventional machine-learning
techniques were restricted in their abilities, for decades Machine learning algorithm is
trained using training data sets to create a model. When the new input data is introduced
to the Machine Learning Algorithm (MLA), it makes predictions based on the model.
The prediction is evaluated for accuracy if accuracy is acceptable, the MLA is

displayed otherwise MLA is trained again with an augmented training data set.
An ML & Deep Learning incorporates the following four steps:
1. Feature extraction is the basis for prediction
2. Select the suitable machine learning algorithm
3. Train and evaluate model performance
4. Use the trained model to predict the obscure data

Library
The steps of Deep Learning (DL) is similar to Machine Learning (ML) as shown in
Fig. 1.1(a), Unlike the machine learning method, its feature extraction is automated rather
than manual. DL is a subset of ML where similar MLA are used to train deep neural
networks to achieve better accuracy while predicting the required result. DL is the only
method to overcome the challenges of feature extraction.
Fig1.2(a): Machine Learning v/s Deep Learning
The Neural Network is divided into two main broad streams:

1. Linearly Separable
2. Non-Linearly Separable
The three different types of ML/DL methods are:

1. Supervised Learning
2. Un-supervised Learning
3. Semi-Supervised Learning
NURAL NETWORK
An artificial neural network learning algorithm, or neural network, or

just neural net, is a computational learning system that uses a network of functions to
understand and translate a data input of one form into a desired output, usually in another
form. The concept of the artificial neural network was inspired by human biology and the

Library
way neurons of the human brain function together to understand inputs from human
senses.
How does it work?
Neural network is made up of neurons connected to each other; at the same time,
each connection of our neural network is associated with a weight that dictates the
importance of this relationship in the neuron when multiplied by the input value. Each
neuron has an activation function that defines the output of the neuron. The activation
function is used to introduce non-linearity in the modeling capabilities of the network. We
have several options for activation functions that we will present in this post. Training our
neural network, that is, learning the values of our parameters (weights and biases) is the
most genuine part of Deep Learning and we can see this learning process in a neural network
as an iterative process of “going and return” by the layers of neurons. The “going” is a
forward propagation of the information and the “return” is a backpropagation of the
information.
Fig1.2(b): Neural Network

Library
Perceptron Learning Algorithm
A Perceptron is a method for learning linear threshold device classifiers. It

computes the weighted sum of all the coordinates of the vector pattern. The produced
output is +1 or -1 which lies below or above the threshold if larger or smaller than the
threshold. In the present study, the perceptron divides the input data sets into two sets. Set
A lies under the decision line when Inputs are having Output as zero though Set B lies over
the decision line when Inputs are having Output as one. Function of weights, Input, and
Bias are mathematically represented as.
f (x) = wx + b (1)
Where w=weight vector, x =Input Vector, b =Bias and f (x) = Transfer capacities.
final output in eq. (1) Bias enables us to move the decision line with the goal that
can best separate the input into two classes.
Fig 1.2(c): Perceptron Rule

A key assignment of this project is to create and analyze Deep learning Algorithm
and compared with the conventional Machine learning Algorithms, Model starts with Deep
learning with perceptron and how to apply it to comprehend different issues in non-
separable information.
The fundamental piece of this project is to make perceptron learning Algorithm
respectful with non-separable training datasets when compared with the conventional
Machine learning Algorithms.

Library
2. SYSTEM ANALYSIS
2.1 Problem statement

As of late, Deep Learning, Machine Learning, and Artificial Intelligence are
profoundly engaged ideas of information science. Deep learning has made progress in the
field of Computer Vision, Speech and Audio Processing, and Natural Language
Processing. It has the solid learning capacity that can improve use of datasets for the
information extraction contrasted with conventional Machine Learning Algorithm.
Perceptron is the fundamental structure hinder for making a profound Neural
Network, it breaks down the unsupervised data, making it an important instrument for
information analytics.
2.2 Problem Description
A key assignment of this project is to create and analyze Deep learning Algorithm
and compared with the conventional Machine learning Algorithms, Model starts with Deep
learning with perceptron and how to apply it to comprehend different issues in non-
separable information. The fundamental piece of this project is to make perceptron learning
Algorithm respectful with non-separable training datasets when compared with the
conventional Machine learning Algorithms.
2.3 Existing System
Conventional machine-learning techniques were restricted in their abilities. For
decades Machine learning algorithm is trained using large training data sets to create a
model. When the new input data is introduced to the Machine Learning Algorithm (MLA),
it makes predictions based on the applied model by collecting the features from the given
input data, If the input data is accurate the prediction will be accurate.
To create a machine learning model you need to train the data set using suitable
Machine learning algorithm by pre processing the data to over come the missing values
and outliers.

Library
2.4 Disadvantages of Existing System
 Time and resources- Machine learning requires massive resources to function.
 Interpretation of results- One needs to exercise caution while choosing
algorithms for their specific purpose.
 Data acquisition- Machine learning needs massive datasets to train on.
2.5 Proposed system

Deep Learning, it is one of the effective methods to predict data analytics even with
a small amount of sample data, Feature extraction is done effectively by pattern
identification. It also helps in recognizing hidden pattern or features through historical
learning.
The prediction is evaluated for accuracy if accuracy is acceptable, the accuracy will
be displayed otherwise MLA is trained again with an augmented training data sets by
changing the weights of the links in the neural network and by calculating the activation
function, TensorFlow library is used to train the data in best possible way to get accurate
results, It is python library developed by google brain team to get 100% assurance while
training.
2.6 Advantages
 No need for Feature Engineering.
 Best results with unstructured data.
 No need for labeling of data.
 Efficient at delivering high quality results.

Library
3.SYSTEM REQUIREMENTS SPECIFICATION
3.1 H/W System Configuration
 Processor : 2.3Ghz Processor
 GPU : 1xTesla K80, 2496 CUDA cores
 RAM : 12GB (min)
 Hard Disk : 120GB
3.2 S/W System Configuration

 Operating System : Windows 8 & above.
 Programming Language : Python.
 Programming Platform : Jupiter Notebook.
 Working Platform : Google Colaboratory.
 Browser Requirements : Internet Explorer 10, Chrome v10.
3.3 Feasibility Study

The feasibility of the project is analyzed in this phase and business proposal is put
forth with a very general plan for the project and some cost estimates. During system
analysis the feasibility study of the proposed system is to be carried out. This is to ensure
that the proposed system is not a burden to the company. For feasibility analysis, some
understanding of the major requirements for the system is essential.
Three key considerations involved in the feasibility analysis are

 Economically Feasibility
 Technical Feasibility
 Social Feasibility

Library
Economically Feasibility
This study is carried out to check the economic impact that the system will have
on the organization. The amount of fund that the company can pour into the research and
development of the system is limited. The expenditures must be justified.
Thus the developed system as well within the budget and this was achieved because
most of the technologies used are freely available. Only the customized products had to be
purchased.
Technical Feasibility
This study is carried out to check the technical feasibility, that is, the technical
requirements of the system. Any system developed must not have a high demand on the
available technical resources. This will lead to high demands on the available technical
resources. This will lead to high demands being placed on the client. The developed system
must have a modest requirement, as only minimal or null changes are required for
implementing this system.
Social Feasibility
The aspect of study is to check the level of acceptance of the system by the user.
This includes the process of training the user to use the system efficiently. The user must
not feel threatened by the system, instead must accept it as a necessity. The level of
acceptance by the users solely depends on the methods that are employed to educate the
user about the system and to make him familiar with it. His level of confidence must be
raised so that he is also able to make some constructive criticism, which is welcomed, as
he is the final user of the system.

Library
4.LITERATURE SURVEY
Literature survey is the most important step in software development process.
Before developing the tool it is necessary to determine the time factor, economy n company
strength. Once these things r satisfied, ten next steps are to determine which operating
system and language can be used for developing the tool. Once the programmers start
building the tool the programmers need lot of external support. This support can be
obtained from senior programmers, from book or from websites. Before building the
system, the above consideration are taken into account for developing the proposed system.
4.1 Yann LeCun, Yoshua Bengio & Geoffrey Hinton, Deep Learning
Deep learning allows computational models that are composed of multiple
processing layers to learn representations of data with multiple levels of abstraction. These
methods have dramatically improved the state-of-the-art in speech recognition, visual
object recognition, object detection and many other domains such as drug discovery and
genomics. Deep learning discovers intricate structure in large data sets by using the
backpropagation algorithm to indicate how a machine should change its internal parameters
that are used to compute the representation in each layer from the representation in the
previous layer. Deep convolutional nets have brought about breakthroughs in processing
images, video, speech and audio, whereas recurrent nets have shone light on sequential
data such as text and speech.
Machine-learning technology powers many aspects of modern society: from web
searches to content filtering on social networks to recommendations on e-commerce
websites, and it is increasingly present in consumer products such as cameras and
smartphones. Machine-learning systems are used to identify objects in images, transcribe
speech into text, match news items, posts or products with users’ interests, and select
relevant results of search. Increasingly, these applications make use of a class of techniques
called deep learning. Conventional machine-learning techniques were limited in their
ability to process natural data in their raw form. For decades, constructing a pattern-

Library
recognition or machine-learning system required careful engineering and considerable
domain expertise to design a feature extractor that transformed the raw data (such as the
pixel values of an image) into a suitable internal representation or feature vector from
which the learning subsystem, often a classifier, could detect or classify patterns in the
input.
4.2 R. Sathya, Annamma Abraham, “Comparison of Supervised and
Unsupervised Learning Algorithms for Pattern Classification”
This paper presents a comparative account of unsupervised and supervised learning
models and their pattern classification evaluations as applied to the higher education
scenario. Classification plays a vital role in machine based learning algorithms and in the
present study, we found that, though the error back-propagation learning algorithm as
provided by supervised learning model is very efficient for a number of non-linear real-
time problems, KSOM of unsupervised learning model, offers efficient solution and
classification in the present study.
4.3 Tang, J., Deng, C. and Huang, G.B., Extreme learning machine for
multilayer perceptron
Extreme learning machine (ELM) is an emerging learning algorithm for the
generalized single hidden layer feedforward neural networks, of which the hidden node
parameters are randomly generated and the output weights are analytically computed.
However, due to its shallow architecture, feature learning using ELM may not be effective
for natural signals (e.g., images/videos), even with a large number of hidden nodes. To
address this issue, in this paper, a new ELM-based hierarchical learning framework is
proposed for multilayer perceptron. The proposed architecture is divided into two main
components:
1) self-taught feature extraction followed by supervised feature classification and
2) they are bridged by random initialized hidden weights. The novelties of this
paper are as follows:

Library
1) unsupervised multilayer encoding is conducted for feature extraction, and an
ELM-based sparse autoencoder is developed via ℓ 1 constraint. By doing so, it
achieves more compact and meaningful feature representations than the original
ELM;
2) by exploiting the advantages of ELM random feature mapping, the hierarchically
encoded outputs are randomly projected before final decision making, which leads
to a better generalization with faster learning speed; and
3) unlike the greedy layerwise training of deep learning (DL), the hidden layers of
the proposed framework are trained in a forward manner. Once the previous layer
is established, the weights of the current layer are fixed without fine-tuning.
Therefore, it has much better learning efficiency than the DL. Extensive
experiments on various widely used classification data sets show that the proposed
algorithm achieves better and faster convergence than the existing state-of-the-art
hierarchical learning methods. Furthermore, multiple applications in computer
vision further confirm the generality and capability of the proposed learning
scheme.
4.4 Zhu, Z., Luo, P., Wang, X. and Tang, X., Multi-view perceptron: a
deep model for learning face identity and view representations.
Various factors, such as identity, view, and illumination, are coupled in face
images. Disentangling the identity and view representations is a major challenge in face
recognition. Existing face recognition systems either use handcrafted features or learn
features discriminatively to improve recognition accuracy. This is different from the
behavior of primate brain. Recent studies discovered that primate brain has a face-
processing network, where view and identity are processed by different neurons. Taking
into account this instinct, this paper proposes a novel deep neural net, named multi-view
perceptron (MVP), which can untangle the identity and view features, and in the meanwhile
infer a full spectrum of multi-view images, given a single 2D face image. The identity
features of MVP achieve superior performance on the MultiPIE dataset. MVP is also
Library
capable to interpolate and predict images under viewpoints that are unobserved in the
training data.
4.6 R. Sathya, Annamma Abraham, “Comparison of Supervised and
Unsupervised Learning Algorithms for Pattern Classification”
This paper presents a comparative account of unsupervised and supervised learning
models and their pattern classification evaluations as applied to the higher education
scenario. Classification plays a vital role in machine based learning algorithms and in the
present study, we found that, though the error back-propagation learning algorithm as
provided by supervised learning model is very efficient for a number of non-linear real-
time problems, KSOM of unsupervised learning model, offers efficient solution and
classification in the present study
4.7 Raiko, T., Valpola, H. and LeCun, Y., Deep learning made easier by
linear transformations in perceptrons.
We transform the outputs of each hidden neuron in a multi-layer perceptron network to
have zero output and zero slope on average, and use separate shortcut connections to model
the linear dependencies instead. This transformation aims at separating the problems of
learning the linear and nonlinear parts of the whole input-output mapping, which has many
benefits. We study the theoretical properties of the transformation by noting that they make
the Fisher information matrix closer to a diagonal matrix, and thus standard gradient closer
to the natural gradient. We experimentally confirm the usefulness of the transformations
by noting that they make basic stochastic gradient learning competitive with state-of-the-
art learning algorithms in speed, and that they seem also to help find solutions that
generalize better. The experiments include both classification of small images and learning
a lowdimensional representation for images by using a deep unsupervised auto-encoder
network. The transformations were beneficial in all cases, with and without regularization
and with networks from two to five hidden layers.

Library
5.SYSTEM DESIGN
5.1 System Architecture
Fig 5.1(a): System Architecture

Description
The System Architecture explains how the predictions will be performed in a
sequential manner, initially the data is read and features and lables should be defined based
on the data taken, then encode the dependent variables and separate the data set into
Training and Testing data.
After dividing the data into Training and Testing Part then apply the TensorFlow
Data Structure for holding features, labels etc.., now implement the model by appending
the data with the algorithm and train the model, based on the trained data validate the
model.
Now find the accuracy score and Mean Square Error in the error is higher again
train the model to reduce the Mean Square Error and make prediction on Test data.

Library
Block Diagram
Fig 5.1(b): Block Diagram

5.2 MODULES
5.2.1 Data understanding
 Considering Sonar dataset containing data of 208 patterns obtained by bouncing
sonar signals at different angles and under different conditions from a metal
cylinder (naval mine) and rock.
 Our aim is therefore to construct a model that can predict whether the object is a
naval mine or a rock.
5.2.2 Exploratory Data Analysis
EDA 1: EDA 2: EDA 3:
1.Variable identification. 1.Importance of variables. 1.Feature extraction.

2.Univariant analysis. 2.Normalization-of 2.Feature construction.
3.Bivariant analysis. variables. 3.Splitting into Training
and Testing Data.

Library
5.2.3 TensorFlow
TensorFlow is a Python-friendly open source library for numerical computation
that makes machine learning faster and easier, it performs multidimensional operations.
Tensor - Arrays.
Flow - Flow of execution.
5.2.4 Machine Learning models
➢ Building a model to train and test the data by using different Machine Learning
classification algorithms to show the difference between Machine Learning
classification and Deep Learning.
➢ Here the data is trained and validated using different classification algorithms to
predict the classification accuracy.
5.2.5 Deep Learning Modules
Neural network
A neural network is a series of algorithms that endeavors to recognize underlying
relationships in a set of data through a process that mimics the way the human brain
operates.
Perceptron
A Perceptron is an algorithm used for supervised learning of binary classifiers.
Binary classifiers decide whether an input, usually represented by a series of vectors,
belongs to a specific class.
Multi-Layer Perceptron
➢ A multilayer perceptron (MLP) is a class of feedforward artificial neural
network (ANN).
➢ An MLP consists of at least three layers of nodes: an input layer, a hidden
layer and an output layer.
➢ Except for the input nodes, each node is a neuron that uses a
nonlinear activation function. MLP utilizes a supervised learning technique
called backpropagation for training.

Library
5.2.6 Validation
In machine learning, we could not fit the model on the training data and cannot say
that the model will work accurately for the real data. For this, we must assure that our
model got the correct patterns from the data, and it is not getting up too much noise. For
this purpose, we use the cross-validation technique.
Cross-Validation
Cross-validation is a technique in which we train our model using the subset of the
dataset and then evaluate using the complementary subset of the dataset.
1. Reserve some portion of sample dataset.

2. Using the rest dataset train the model.
3. Test the model using the reserve portion of the dataset.
5.3 Algorithm
If you are aware of the Perceptron Algorithm, in the perceptron we just multiply
with weights and add Bias, but we do this in one layer only.
Fig 5.3(a): Perceptron

Library
We update the weight when we found an error in classification or miss-classified. Weight
update equation is this…
weight = weight + learning_rate * (expected - predicted) * x
In the Multilayer perceptron, there can more than one linear layer (combinations
of neurons). If we take the simple example the three-layer network, first layer will be
the input layer and last will be output layer and middle layer will be called hidden layer. We
feed our input data into the input layer and take the output from the output layer. We can
increase the number of the hidden layer as much as we want, to make the model more
complex according to our task.
Fig 5.3(b): Multi-Layer perceptron
Feed Forward Network, is the most typical neural network model. Its goal is to
approximate some function f (). Given, for example, a classifier y = f ∗ (x) that maps an
input x to an output class y, the MLP find the best approximation to that classifier by
defining a mapping, y = f(x; θ) and learning the best parameters θ for it. The MLP networks
are composed of many functions that are chained together. A network with three functions
or layers would form f(x) = f (3)(f (2)(f (1)(x))). Each of these layers is composed of units

Library
that perform an affine transformation of a linear sum of inputs. Each layer is represented as
y = f(WxT + b). Where f is the activation function (covered below), W is the set of
parameter, or weights, in the layer, x is the input vector, which can also be the output of the
previous layer, and b is the bias vector. The layers of an MLP consists of several fully
connected layers because each unit in a layer is connected to all the units in the previous
layer. In a fully connected layer, the parameters of each unit are independent of the rest of
the units in the layer, that means each unit possess a unique set of weights.
In a supervised classification system, each input vector is associated with a label, or

ground truth, defining its class or class label is given with the data. The output of the network
gives a class score, or prediction, for each input. To measure the performance of the
classifier, the loss function is defined. The loss will be high if the predicted class does not
correspond to the true class, it will be low otherwise. Sometimes the problem of overfitting
and underfitting occurs at the time of training the model. In this case, Our model performs
very well on training data but not on testing data. In order to train the network, an
optimization procedure is required for this we need loss function and an optimizer. This
procedure will find the values for the set of weights, W that minimizes the loss function.
A popular strategy is to initialize the weights to random values and refine them
iteratively to get a lower loss. This refinement is achieved by moving on the direction
defined by the gradient of the loss function. And it is important to set a learning rate defining
the amount in which the algorithm is moving in every iteration
Activation function
Activation functions also known non- linearity, describe the input-output relations
in a non-linear way. This gives the model power to be more flexible in describing arbitrary
relations.

Library
Training the Model
There are basically three steps in the training of the model.
1. Forward pass
2. Calculate error or loss
3. Backward pass
1.Forward Pass
In this step of training the model, we just pass the input to model and multiply with
weights and add bias at every layer and find the calculated output of the model.
Fig 5.3(c): Forward Pass
2. Loss Calculate
When we pass the data instance(or one example) we will get some output from the
model that is called Predicted output(pred_out) and we have the label with the data that is

Library
real output or expected output(Expect_out). Based upon these both we calculate the loss that
we have to backpropagate(using Backpropagation algorithm). There is various Loss
Function that we use based on our output and requirement.
3. Backward Pass
After calculating the loss, we backpropagate the loss and updates the weights of the
model by using gradient. This is the main step in the training of the model. In this step,
weights will adjust according to the gradient flow in that direction.
5.4 Introduction to UML
UML (Unified Modeling Language)
UML is a language for visualizing, specifying, constructing and documenting the

artifacts of a software intensive system. UML is simply another graphical representation
of a common semantic model. UML provides a comprehensive notation for the full
lifecycle of object-oriented development.
Advantages
• To represent complete systems (instead of only the software portion) using object-
oriented concepts.
• To establish an explicit coupling between concepts and executable code.
• To consider the scaling factors that are inherent to complex and critical systems.
• To creating a modeling language usable by both humans and machines.
As UML describes the real time systems it is very important to make a conceptual
model and then proceed gradually. Conceptual model of UML can be mastered by learning
the following three major elements:

Library
• UML building blocks
• Rules to connect the building blocks
• Common mechanisms of UML
The building blocks of UML can be defined as
• Things
• Relationships
• Diagrams
5.5UML Diagrams
UML diagrams are the ultimate output of the entire discussion. All the elements,
relationships are used to make a complete UML diagram and the diagram represents a
system.
The visual effect of the UML diagram is the most important part of the entire process. All
the other elements are used to make it a complete one.
UML includes the following nine diagrams and the details are described in the following
chapters.
• Class diagram
• Object diagram
• Use case diagram
• Sequence diagram
• Collaboration diagram
• Activity diagram
• State chart diagram
• Deployment diagram
• Component diagram

Library
UML defines several models for representing s ystems
1. Use case diagrams: represent the functions of a system from the user's point of view.
2. Class diagrams : represent the static structure in terms of classes and relationships
3. Object diagrams : represent objects and their relationships and correspond to

simplified collaboration diagrams that do not represent message broadcasts.
4. Sequence diagrams: are a temporal representation of objects and their interactions.
5. Collaboration diagrams: spatial representation of objects, links, and interactions.

6. State chart diagrams: represent the behavior of a class in terms of states at run time.
7. Activity diagrams: represent the behavior of an operation as a set of actions
8. Component diagrams: represent the physical components of an application
9. Deployment diagrams: represent the deployment of components on pieces of

hardware

Library
5.5.1 Class Diagram
AIM: To implement class diagram for Implementation of Deep Learning Algorithm
with Perceptron using TensorFlow Library.
DESCRIPTION: A class diagram shows a set of classes, interfaces and collaborations and
their relationships.
OBJECTIVE: The main objective of the class diagram to illustrate the static design of a
view system.
THINGS: class, interfaces, collaboration, active class.
RELATIONSHIPS: Dependency, generalization and association
class C# Model
User
Data Understanding
- Data Base
- Exploratory Data Analysis - Algorithms
explores data - Cross Validation
+ Variable Identification()
1.. 1 + Upload Data()
+ Multi Varient Analysis()
+ Importance Of Variables() + View Dataset()
+ Normalization Of Variables() + Evaluate Model()
+ Compare Model()
modeling
1
1
Modeling
- Machine Learning
Algorithms - Deep Learning
using data
+ Train Model()
+ Test Model()
+ Select Algorithm()
algorithm
selection
Algorithms
- Libraries
1 1
select algorithn + Select Algorithm()
1 select algorithm
ML Classification
- Logistic Regression 1
- Linear Discriminate Analysis
- Decission Tree Model Ev aluation DL Classification
- Random Forest cross validation cross validation
- KNeighborsClassifier - Cross Validation - Multi Layer Perceptron Classifier
- GaussianNB - KFold Validation
- SVM
- AdaBoostClassifier
- GradientBoostingClassifier
- ExtraTreesClassifier
Fig 5.5.1: Class Diagram

Library
5.5.2 USE CASE DIAGRAM
AIM: To implement use case diagram for Implementation of Deep Learning
Algorithm with Perceptron using TensorFlow Library.
DESCRIPTION: Use case diagrams are central to modeling the behavior of the system
or a class. Use case diagrams are important for testing executable systems through
reverse engineering.
OBJECTIVE: Use case diagram organizes the behavior of the system.
THINGS: Use cases, Actors.
RELATIONSHIPS: Association.
uc Manage Users
System
Authentication &
Authorization
Google Colaboratory
Select Dataset
Load Dataset
Analyse Dataset
Split Dataset
(Training & Testing)
User
Execution Env ironment
Select Algorithm
Train Dataset
Validate Dataset
Cross v alidate
Result
Compare Accuracy
Fig 5.5.2: Use Case Diagram

Library
5.5.3 SEQUENCE DIAGRAM
AIM: To implement sequence diagram for Implementation of Deep Learning
DESCRIPTION: A sequence diagram is an interaction diagram that emphasizes the time
ordering messages. A sequence diagram shows a set of objects and messages sent and
received by the by those objects.
OBJECTIVE: To illustrate the dynamic view of system
THINGS: Objects and Messages
RELATIONSHIP: Time and lifeline, Links
sd 12.Algorithm
:User :Googl e :Executi on

col aboratory Envi ronment
1:User Logi n
()
2:Accept/Rej ect()
3:Import Li brari es()
4:Sel ect Dataset()
5:Load Dataset()
6:Anal yse Dataset()
7:Spl i tDataset()
8:Sel ect Al gori thm()
9:T rai n Data()
10:Val i date Data()
11:Predi cti on()
12:Cross Val i date()
13:Cross Val i dati on Resul t()
Stop Stop
Stop
Fig 5.5.3: Sequence Diagram

Library
5.5.4 ACTIVITY DIAGRAM:
AIM: To implement activity diagram for Implementation of Deep Learning Algorithm
with Perceptron using TensorFlow Library.
DESCRIPTION: An activity diagram is essentially a flow chart, showing flow of
control from activity to activity. It involves modeling the sequential steps in
computational process. Activity diagram not only important for modeling dynamic
aspects of a system, but also for constructing executable system through forward and
reverse engineering.
Objective: Focused on floe of control from activity to activity
THINGS: State and object
RELATIONSHIPS: Transitions
act Customer Process
User Google Colaboratory Execution Env ironment
User
Authenti cate
Val i d
User Login Import Libraries
Inval i d
Authenti cati on Fai l ed
Analyse Dataset Load Dataset
Split (Training and

Testing)
Select Algorithm Train Dataset
Validate Dataset
Cross Validation
Comparision Of Result Result
Stop
Fig 5.5.4: Activity Diagram

Library
5.5.5 DEPLOYMENT DIAGRAM:
AIM: To implement Deployment diagram for Implementation of Deep Learning
DESCRIPTION: A UML deployment diagram is a diagram that shows the
configuration of run time processing nodes and the components that live on them.
Deployment diagrams is a kind of structure diagram used in modeling the physical
aspects of an object-oriented system. They are often be used to model the static
deployment view of a system (topology of the hardware).
OBJECTIVE: To illustrate the dynamic view of system
THINGS: Nodes
RELATIONSHIP: Associations, Dependencies, Generalizations and Realizations.
deployment Deployment Model
Google
Colaboratory
Python
Notebook
Algorithms Libraries Cross

Validation
Deep Learning Machine

Algorithm Learning
Classification
Algorithms
Fig 5.5.5: Deployment Diagram

Library
6.SYSTEM IMPLEMENTATION
6.1 Software Description
6.1.1 Introduction to Google Colaboratory
Colaboratory is a Google research project created to help disseminate machine

learning education and research. It's a Jupyter notebook environment that requires no setup
to use and runs entirely in the cloud. This means that if you have a Google account, you
can freely train your models on a K80 GPU.
When you log in and connect to the Colaboratory runtime, you will get your own
virtual machine with a K80 GPU (not always entirely yours, read below) and a Jupyter
notebook environment. You can use it to your heart's content for up to 12 hours. Or until
you close your browser window. Be warned, sometimes the runtime can disconnect
randomly.
If you're lucky and you get access to the full card you'll see something like this:
Gen RAM Free: 11.6 GB | Proc size: 666.0 MB

GPU RAM Free: 11439MB | Used: 0MB | Util 0% | Total 11439MB
If you see RAM usage, that means that you are sharing the card with someone. Since by
default Tensorflow allocates almost 100% of the GPU's memory this is a problem. To get
a new instance and a new chance to get you own GPU, run the following command in a
cell:
!kill -9 -1

Library
Then wait 30-60 seconds and reconnect and check with the above script if you got your
own GPU.
Making sure you’re training on the GPU
This code trains a sample model on the GPU.
import tensorflow as tf
with tf.device('/gpu:0'):
model.fit(X_train, y_train, epochs=10,batch_size=128, verbose=1)
File transfer to Google Colaboratory
Although the first method described below seems more involved, I think it's the best one
as I have had problems with the second method in the past.
Mounting a Google Drive folder inside your Jupyter Notebook
Mount your Google Drive inside the Notebook.
import os
from google.colab import drive
drive.mount('/gdrive')
os.symlink('/gdrive/My Drive', 'my_drive')

!ls -l my_drive

Library
Saving Data using the Google Colaboratory ython library
To download data you can also use the files library provided by Google.
from keras.models import model_from_json

#save model and weights
model_json = model.to_json()
with open("model.json", "w") as json_file:
json_file.write(model_json)
# serialize weights to HDF5
model.save_weights("model.h5")
print("Saved model to disk")
from google.colab import files
files.download('model.json')
files.download('model.h5')
Loading Data using the Google Colaboratory Python library
You can also upload your own files using the same library.
from google.colab import files

uploaded = files.upload()
for fn in uploaded.keys():
print('User uploaded file "{name}" with length {length} bytes'.format
name=fn, length=len(uploaded[fn])))
from keras.models import model_from_json
# load model and weights

json_file = open('model.json', 'r')
loaded_model_json = json_file.read()
Library
json_file.close()
loaded_model = model_from_json(loaded_model_json)
#load weights
loaded_model.load_weights("model.h5")
print("Loaded model from disk")
model = loaded_model
model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy'])
6.1.2 Introduction to jupyter Notebook
The notebook extends the console-based approach to interactive computing in a

qualitatively new direction, providing a web-based application suitable for capturing the
whole computation process: developing, documenting, and executing code, as well as
communicating the results. The Jupyter notebook combines two components:
A web application: a browser-based tool for interactive authoring of documents which

combine explanatory text, mathematics, computations and their rich media output.
Notebook documents: a representation of all content visible in the web application,

including inputs and outputs of the computations, explanatory text, mathematics, images,
and rich media representations of objects.
Main features of the web application
• In-browser editing for code, with automatic syntax highlighting, indentation, and tab
completion/introspection.
• The ability to execute code from the browser, with the results of computations attached
to the code which generated them.

Library
• Displaying the result of computation using rich media representations, such as HTML,
LaTeX, PNG, SVG, etc. For example, publication-quality figures rendered by
the matplotlib library, can be included inline.
• In-browser editing for rich text using the Markdown markup language, which can
provide commentary for the code, is not limited to plain text.
• The ability to easily include mathematical notation within markdown cells using
LaTeX, and rendered natively by MathJax.
Notebook documents
Notebook documents contains the inputs and outputs of a interactive session as well
as additional text that accompanies the code but is not meant for execution. In this way,
notebook files can serve as a complete computational record of a session, interleaving
executable code with explanatory text, mathematics, and rich representations of resulting
objects. These documents are internally JSON files and are saved with
the .ipynb extension. Since JSON is a plain text format, they can be version-controlled
and shared with colleagues.
Notebooks and privacy

Because you use Jupyter in a web browser, some people are understandably
concerned about using it with sensitive data. However, if you followed the standard install
instructions, Jupyter is actually running on your own computer. If the URL in the address
bar starts with http://localhost: or http://127.0.0.1: , it’s your computer acting as the
server. Jupyter doesn’t send your data anywhere else—and as it’s open source, other people
can check that we’re being honest about this.

Library
Creating a new notebook document
A new notebook may be created at any time, either from the dashboard, or using the File
>New menu option from within an active notebook. The new notebook is created within
the same directory and will open in a new browser tab. It will also be reflected as a new
entry in the notebook list on the dashboard.
Notebook user interface

When you create a new notebook document, you will be presented with
the notebook name, a menu bar, a toolbar and an empty code cell.
Notebook name: The name displayed at the top of the page, next to the Jupyter logo,
reflects the name of the .ipynb file. Clicking on the notebook name brings up a dialog
which allows you to rename it. Thus, renaming a notebook from “Untitled0” to “My first
notebook” in the browser, renames the Untitled0.ipynb file to My first notebook.ipynb .
Toolbar: The tool bar gives a quick way of performing the most-used operations within
the notebook, by clicking on an icon.
Code cell: the default type of cell; read on for an explanation of cells.
Structure of a notebook document

The notebook consists of a sequence of cells. A cell is a multiline text input field, and its
contents can be executed by using Shift-Enter, or by clicking either the “Play” button the
toolbar, or Cell , Run in the menu bar. The execution behavior of a cell is determined by
the cell’s type.

Library
There are three types of cells: code cells, markdown cells, and raw cells. Every
cell starts off being a code cell, but its type can be changed by using a drop-down on the
toolbar (which will be “Code”, initially), or via keyboard shortcuts.
Code cells
A code cell allows you to edit and write new code, with full syntax highlighting
and tab completion. The programming language you use depends on the kernel, and the
default kernel (IPython) runs Python code, When a code cell is executed, code that it
contains is sent to the kernel associated with the notebook. The results that are returned
from this computation are then displayed in the notebook as the cell’s output. The output
is not limited to text, with many other possible forms of output are also possible,
including matplotlib figures and HTML tables (as used, for example, in the pandas data
analysis package). This is known as IPython’s rich display capability.
Markdown cells
You can document the computational process in a literate way, alternating
descriptive text with code, using rich text. In IPython this is accomplished by marking up
text with the Markdown language. The corresponding cells are called Markdown cells. The
Markdown language provides a simple way to perform this text markup, that is, to specify
which parts of the text should be emphasized (italics), bold, form lists, etc.
If you want to provide structure for your document, you can use markdown headings.
Markdown headings consist of 1 to 6 hash # signs # followed by a space and the title of
your section. The markdown heading will be converted to a clickable link for a section of
the notebook. It is also used as a hint when exporting to other document formats, like PDF.

Library
When a Markdown cell is executed, the Markdown code is converted into the
corresponding formatted rich text. Markdown allows arbitrary HTML code for formatting.
Within Markdown cells, you can also include mathematics in a straightforward

way, using standard LaTeX notation: $...$ for inline mathematics and $$...$$ for
displayed mathematics. When the Markdown cell is executed, the LaTeX portions are
automatically rendered in the HTML output as equations with high quality typography.
This is made possible by MathJax, which supports a large subset of LaTeX functionality
Standard mathematics environments defined by LaTeX and AMS-LaTeX

(the amsmath package) also work, such as \begin{equation}...\end{equation} ,
and \begin{align}...\end{align} . New LaTeX macros may be defined using standard
methods, such as \newcommand , by placing them anywhere between math delimiters in a
Markdown cell. These definitions are then available throughout the rest of the IPython
session.
Raw cells
Raw cells provide a place in which you can write output directly. Raw cells are not
evaluated by the notebook. When passed through nbconvert, raw cells arrive in the
destination format unmodified. For example, you can type full LaTeX into a raw cell,
which will only be rendered by LaTeX after conversion by nbconvert.
Keyboard shortcuts
All actions in the notebook can be performed with the mouse, but keyboard
shortcuts are also available for the most common ones. The essential shortcuts to remember
are the following:

Library
Shift+Enter: Run Shell
Execute the current cell, show any output, and jump to the next cell below. If Shift-
Enter is invoked on the last cell, it makes a new cell below. This is equivalent to
clicking the Cell , Run menu item, or the Play button in the toolbar.
Esc: command mode
In command mode, you can navigate around the notebook using keyboard
shortcuts.
Enter: Edit mode
In edit mode, you can edit text in cells.
Plotting
One major feature of the Jupyter notebook is the ability to display plots that are the
output of running code cells. The IPython kernel is designed to work seamlessly with
the matplotlib plotting library to provide this functionality. Specific plotting library
integration is a feature of the kernel.
There are a variety of ways to get help for Python
 Inside the Python interpreter, the help() function pulls up documentation strings for
various modules, functions, and methods. These doc strings are similar to Java's
javadoc. The dir() function tells you what the attributes of an object are. Below are
some ways to call help() and dir() from the interpreter:
 help(len) — help string for the built-in len() function; note that it's "len" not "len()",
which is a call to the function, which we don't want
 help(sys) — help string for the sys module (must do an import sys first)

Library
 dir(sys) — dir() is like help() but just gives a quick list of its defined symbols, or
"attributes"
 help(sys.exit) — help string for the exit() function in the sys module
 help('xyz'.split) — help string for the split() method for string objects. You can
call help() with that object itself or an example of that object, plus its attribute. For
example, calling help('xyz'.split) is the same as calling help(str.split).
 help(list) — help string for list objects
 dir(list) — displays list object attributes, including its methods
 help(list.append) — help string for the append() method for list objects
6.2Python Libraries for data science

1.Data Processing and Modeling
NumPy
NumPy (Numerical Python) is a perfect tool for scientific computing and
performing basic and advanced array operations. The library offers many handy features
performing operations on n-arrays and matrices in Python. It helps to process arrays that
store values of the same data type and makes performing math operations on arrays (and
their vectorization) easier. In fact, the vectorization of mathematical operations on the
NumPy array type increases performance and accelerates the execution time.
SciPy
This useful library includes modules for linear algebra, integration, optimization,
and statistics. Its main functionality was built upon NumPy, so its arrays make use of this
library. SciPy works great for all kinds of scientific programming projects (science,
mathematics, and engineering). It offers efficient numerical routines such as numerical

Library
optimization, integration, and others in submodules. The extensive documentation makes
working with this library really easy.
Pandas
Pandas is a library created to help developers work with "labeled" and "relational"
data intuitively. It's based on two main data structures: "Series" (one-dimensional, like a
list of items) and "Data Frames" (two-dimensional, like a table with multiple columns).
Pandas allows converting data structures to DataFrame objects, handling missing data, and
adding/deleting columns from DataFrame, imputing missing files, and plotting data with
histogram or plot box. It’s a must-have for data wrangling, manipulation, and visualization.
Keras
Keras is a great library for building neural networks and modeling. It's very
straightforward to use and provides developers with a good degree of extensibility. The
library takes advantage of other packages, (Theano or TensorFlow) as its backends.
Moreover, Microsoft integrated CNTK (Microsoft Cognitive Toolkit) to serve as another
backend. It's a great pick if you want to experiment quickly using compact systems – the
minimalist approach to design really pays off!
SciKit-Learn
This is an industry-standard for data science projects based in Python. Scikits is a
group of packages in the SciPy Stack that were created for specific functionalities – for
example, image processing. Scikit-learn uses the math operations of SciPy to expose a
concise interface to the most common machine learning algorithms.

Library
Data scientists use it for handling standard machine learning and data mining tasks
such as clustering, regression, model selection, dimensionality reduction, and
classification. Another advantage? It comes with quality documentation and offers high
performance.
Tensor-Flow
TensorFlow is a popular Python framework for machine learning and deep learning,
which was developed at Google Brain. It is the best tool for tasks like object identification,
Pattern recognition, speech recognition, and many others. It helps in working with artificial
neural networks that need to handle multiple data sets.
2. Data Visualization
Matplotlib
This is a standard data science library that helps to generate data visualizations such
as two-dimensional diagrams and graphs (histograms, scatterplots, non-Cartesian
coordinates graphs). Matplotlib is one of those plotting libraries that are really useful in
data science projects — it provides an object-oriented API for embedding plots into
applications, It's thanks to this library that Python can compete with scientific tools like
MatLab or Mathematica. However, developers need to write more code than usual while
using this library for generating advanced visualizations. Note that popular plotting
libraries work seamlessly with Matplotlib.
Seaborn
Seaborn is based on Matplotlib and serves as a useful Python machine learning tool
for visualizing statistical models – heatmaps and other types of visualizations that

Library
summarize data and depict the overall distributions. When using this library, you get to
benefit from an extensive gallery of visualizations.
Plotly
This web-based tool for data visualization that offers many useful out-of-box
graphics – you can find them on the Plot.ly website. The library works very well in
interactive web applications. Its creators are busy expanding the library with new graphics
and features for supporting multiple linked views, animation, and crosstalk integration.
6.4 Sample code
import os
import numpy as np
import pandas as pd
from google.colab import drive
drive.mount('/content/gdrive')
df=pd.read_csv("/content/sonar.all-data.csv")
import tensorflow as tf
from sklearn import preprocessing
df.head()
le=preprocessing.LabelEncoder()
for i in range(len(df.columns)-1,len(df.columns)):
df.iloc[:,i]=le.fit_transform(df.iloc[:,i]).astype(float)
df.head()
seed = 7
np.random.seed(seed)
x_train,x_test,y_train,y_test=train_test_split(x,y,test_size=0.20)
from sklearn.model_selection import KFold

from sklearn.model_selection import cross_val_score
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
from sklearn.naive_bayes import GaussianNB

Library
from sklearn.svm import SVC
# Spot Check Algorithms

models = []
models.append(('LR', LogisticRegression()))
models.append(('LDA', LinearDiscriminantAnalysis()))
models.append(('KNN', KNeighborsClassifier()))
models.append(('DT', DecisionTreeClassifier()))
models.append(('NB', GaussianNB()))
models.append(('SVM', SVC()))
results = []
names = []
for name, model in models:
kfold = KFold(n_splits=num_folds)
cv_results = cross_val_score(model, x_train, y_train, cv=kfold, scori
ng=scoring)
results.append(cv_results)
names.append(name)
msg = "%s: %f (%f)" % (name, cv_results.mean(), cv_results.std())
print(msg)
# Compare Algorithms
fig = plt.figure(figsize = (12,12))
fig.suptitle('Algorithm Comparison')
ax = fig.add_subplot(111)
plt.boxplot(results)
ax.set_xticklabels(names)
plt.show()
from sklearn.metrics import accuracy_score
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report
# Finalize Model
# prepare the model
model.fit(x_train, y_train)
# estimate accuracy on validation dataset

predictions = model.predict(x_test)
print(accuracy_score(y_test, predictions))
print(confusion_matrix(y_test, predictions))

Library
print(classification_report(y_test, predictions))
from sklearn.neural_network import MLPClassifier
# limited memory broyden fletcher goldfarb shanno "solver"

clf = MLPClassifier(solver='lbfgs',hidden_layer_sizes=(40,40),max_iter=
200,random_state=None)
from sklearn.linear_model import Perceptron

import matplotlib.pyplot as plt
class Perceptron(object):
def __init__(self,eta=0.01, n_iter=200):
self.n_iter = n_iter
self.eta = eta
def fit(self,x,y, chooseWeightVector, x_test,y_test):
if chooseWeightVector == 1:
self.w_ = np.random.rand(1 + x.shape[1])
else:
self.w_ = np.zeros(1 + x.shape[1])
self.w_[0] = 1
self.errors_ = []
self.accuracies_ = []
for _ in range(self.n_iter):
#zip: Make an iterator that aggregates elements from each of the iterab
les.
for xi, target in zip(x, y):
# w <- w + α(y — f(x))x or alternately
# w <- w + α(t - o)x
# predict is: o = sign(w * x) != t
o = self.predict(xi)
update = self.eta * (target - o)
self.w_[1:] += update * xi
self.w_[0] += update
self.calc_error(x_test,y_test)
def calc_error(self, x, y):

errors = 0
sumOfAccuracy = 0
for x_t, y_t in zip(x,y):

Library
y_pred = self.predict(x_t)
errors += np.square(y_t-y_pred)
sumOfAccuracy += 1 if y_pred == y_t else 0
self.errors_.append(errors/(2*len(x)))
self.accuracies_.append(sumOfAccuracy/len(x))
def net_input(self,x):
# sum(wi * xi)
# w · x + b
return np.dot(x, self.w_[1:]) + self.w_[0]
def predict(self, x):
#sign(net)
return np.where(self.net_input(x) >= 0.0, 1, -1)
print('Mean accuracy on test set with {} epochs and learning rate={}: {
} '.format(p_null.n_iter,i,sum(p_null.accuracies_)/(len(p_null.accuraci
es_))))
kf = KFold(n_splits = 10, shuffle = True)
rf_reg = MLPClassifier(solver='lbfgs',hidden_layer_sizes=(40,40),max_it
er=200,random_state=None)
for i in range(10):
result = next(kf.split(df), None)
x_train = df.iloc[result[0]]
x_test = df.iloc[result[1]]
y_train = y.iloc[result[0]]
y_test = y.iloc[result[1]]
model = rf_reg.fit(x_train,y_train)
predictions = rf_reg.predict(x_test)
scores.append(model.score(x_test,y_test))
print('Scores from each Iteration: ', scores)
print('Average K-Fold Score :' , np.mean(scores))

Library
7.SYSTEM TESTING
7.1. Validation
Once we are done with training our model, we just cannot assume that it is going
to work well on data that it has not seen before. In other words, we cannot be sure that
the model will have the desired accuracy and variance in production environment. We
need assurance of the accuracy of the predictions that our model is putting out. For this,
we need to validate our model. This process of deciding whether the numerical results
quantifying hypothesized relationships between variables, are acceptable as descriptions
of the data, is known as validation.
To evaluate the performance of any machine learning model we need to test it on

some unseen data. Based on the model’s performance on unseen data we can say weather
our model is Under-fitting/Over-fitting/Well generalized.
7.2 Types of Validation
In machine learning, we couldn’t fit the model on the training data and can’t say
that the model will work accurately for the real data. For this, we must assure that our
model got the correct patterns from the data, and it is not getting up too much noise. For
this purpose, we use the cross-validation technique.

Library
Cross-Validation
Cross-validation is a technique in which we train our model using the subset of the dataset
and then evaluate using the complementary subset of the dataset.
The three steps involved in cross-validation are as follows:
4. Reserve some portion of sample dataset.

5. Using the rest dataset train the model.
6. Test the model using the reserve portion of the dataset.
7.2.1. Validation
In this method, we perform training on the 50% of the given dataset and rest 50%
is used for the testing purpose. The major drawback of this method is that we perform
training on the 50% of the dataset, it may possible that the remaining 50% of the data
contains some important information which we are leaving while training our model i.e
higher bias.
7.2.2. LOOCV (Leave One Out Cross Validation)
In this method, we perform training on the whole dataset but leaves only one data-
point of the available dataset and then iterates for each data-point. It has some advantages
as well as some disadvantages also. An advantage of using this method is that we make use
of all data points and hence it is low bias. The major drawback of this method is that it
leads to higher variation in the testing model as we are testing against one data point. If the
data point is an outlier it can lead to higher variation. Another drawback is it takes a lot of
execution time as it iterates over ‘the number of data points’ times.
7.2.3. KFold Cross validation

In this method, we split the data-set into k number of subsets (known as folds) then we
perform training on the all the subsets but leave one(k-1) subset for the evaluation of the trained
model. In this method, we iterate k times with a different subset reserved for testing purpose
each time.

Library
Note:
It is always suggested that the value of k should be 10 as the lower value
of k is takes towards validation and higher value of k leads to LOOCV method.
Example
The diagram below shows an example of the training subsets and evaluation subsets
generated in k-fold cross-validation. Here, we have total 25 instances. In first iteration we
use the first 20 percent of data for evaluation, and the remaining 80 percent for training([1-
5] testing and [5-25] training) while in the second iteration we use the second subset of 20
percent for evaluation, and the remaining three subsets of the data for training([5-10]
testing and [1-5 and 10-25] training), and so on.
Fig 7.2.3: LOOCV Method
Advantages of train/test split

1. This runs K times faster than Leave One Out cross-validation because K-fold cross-
validation repeats the train/test split K-times.
2. Simpler to examine the detailed results of the testing process.

Library
Advantages of cross-validation
1. More accurate estimate of out-of-sample accuracy.

2. More “efficient” use of data as every observation is used for both training and testing.
7.3 Cross Validating
Validation
In this method, we perform training on the 80% of the given data-set and rest 20%
is used for the testing purpose. The major drawback of this method is that we perform
training on the 80% of the dataset, it may possible that the remaining 20% of the data
contains some important information which we are leaving while training our model i.e
higher bias.
Score
Classification report
Fig 7.3(a): Validation

Library
KFold Cross Validation
In this method, we split the data-set into k number of subsets (known as folds) then we
perform training on the all the subsets but leave one(k-1) subset for the evaluation of the trained
model. In this method, we iterate k times with a different subset reserved for testing purpose
each time.
Note:
It is always suggested that the value of k should be 10 as the lower value
of k is takes towards validation and higher value of k leads to LOOCV method.
Fig 7.3(b): KFold cross validation

Library
8. RESULTS
8.1 EXECUTION PROCEDURE
Procedure
Data science pipelines are sequences of obtaining, cleaning, visualizing, modelling

and interpreting data for a particular purpose. They are useful in production projects, and
they can also be beneficial if one expects to come across the identical kind of business
question in the future, so as to retailer on design time and coding.
• Obtaining data
• Cleaning or Scrubbing data
• Visualizing and Exploring data will allow us to find patterns and trends
• Modelling data will give us our predictive power as a data magician
• Interpreting data
8.2 Screenshots
8.2.1. Google colab Jupiter Notebook
Figure 8.2.1: Google colab Jupiter Notebook

Library
Description
The above figure represents Google colab Jupiter Notebook, here python code can be
edited and executed.
8.2.2. Mounting Google Drive:
Fig 8.2.2: Mounting Google Drive
Description
The above figure represents how to mount a google drive to the working environment to
access data.
8.2.3. Reading Data
Fig 8.2.3: Reading Data

Library
Description
The above figure represents how to read the csv file to the jupiter Notebook and print the
top five rows of the csv data.
8.2.4. Classes of Target Variable
Fig 8.2.4. Classes of Target Variable

Description
The above figure represents the classes of the target variable, the target variable contains
two classes Mine(M) and Rock(R), it is a categorical variable.
8.2.5. Describing the data
Fig 8.2.5. Describing the data

Description
The above figure represents the description about each and every variable in the data, by
this you can find the count, mean, standard deviation, minimum value, 25 percentile value,
50 percentile value, 75 percentile value, and maximum value of each and every column in
the data.
Library
8.2.6. Encoding the Target variable
Fig 8.2.6. Encoding the Target variable

Description
The above figure represents the Encoding the target variable to float from categorical
because all the values in the variables is float type.
8.2.7. Data Splitting
Fig 8.2.7. Data Splitting
Description
The above figure represents the splitting of data into training data and testing data to build
a model.

Library
8.2.8. Machine Learning classification
8.2.8.1
Fig 8.2.8.1 Machine learning Classification

8.2.8.2
Fig 8.2.8.2: Machine Learning Classification

Library
Description
The above figure represents the accuracy score of all Conventional classification
algorithms using the KFold Cross Validation technique.
8.2.9. Deep Learning classification
Fig 8.2.9: Deep Learning Classification
Description
The above figure represents the accuracy score of all Deep Learning classification
algorithms using the Multi-Layer Perceptron Classifier algorithm.
8.2.10. Deep Learning classification Report
Fig 8.2.10. Deep Learning classification Report

Library
Description
The above figure represents the Classification report of the Multi-Layer Perceptron
algorithm by displaying the precision, Recall, F1 score and support for both classes in the
target variable Mine and Rock.
8.2.11. Deep Learning Cross Validation
Fig 8.2.11. Deep Learning Cross Validation

Description
The above figure represents the cross-validation score of a Deep Learning Algorithm by
using the KFold Cross Validation technique.

Library
9.CONCLUSION AND FUTURE ENHANCEMENT
9.1 CONCLUSION
In this paper, we have learnt what is perceptron and how the perceptron learning
algorithm is implemented using TensorFlow library to gain the accuracy when it is
compared with the conventional classification algorithms. Also, we discussed how a
perceptron can be used as a linear classifier and how to implement AND Gate using
perceptron. We have observed that the Perceptron learning algorithm guarantees the
performance depending on a notion of “separability” of training datasets. The solution
describes the convergence property and could be applied to other models too in the
nonlinear separable problem.
9.2 FUTURE ENHANCEMENT

It can be extended in such a way that using more number of hidden layers and
altering the iterations and neurons.

Library
10. BIBLIOGRAPHY
10.1 References
1. Yann LeCun, Yoshua Bengio & Geoffrey Hinton,”Deep Learning”,436
NATURE, Vol. 521, 28 May 2015.
2. R. Sathya, Annamma Abraham, “Comparison of Supervised and Unsupervised
Learning Algorithms for Pattern Classification”, (IJARAI)
International Journal of Advanced Research in Artificial Intelligence, Vol. 2, No.
2, 2013.
3. Kai-Yeung Siu, Amir Dembo, Thomas Kailath. “On the Perceptron Learning
Algorithm on Data with High Precision”, Journal Of Computer And System
Sciences 48, 347-356 (1994).
4. Tang, J., Deng, C. and Huang, G.B., 2016. Extreme learning machine for
multilayer perceptron. IEEE transactions on neural networks and learning
systems, 27(4), pp.809-821.
5. Yang xin, Lingshuang Kong, Zhi Liu, (member, IEEE), Yuling Chen, Yanmiao Li,
Hongliang Zhu, Mingcheng Gao,Haixia Hou, and Chunhua Wang , “Machine
Learning and Deep Learning Methods for Cyber security”,IEEE Access,Vol. 6
page 35365 – 35381(15 May 2018).21
6. Raiko, T., Valpola, H. and LeCun, Y., 2012, March. Deep learning made easier
by linear transformations in perceptrons. In Artificial intelligence and statistics
(pp. 924-932).
10.2 Websites
1. http://wikipedia.org/
2. https://archive.ics.uci.edu/ml/index.php
3. https://towardsdatascience.com/how-do-artificial-neural-networks-learn-
773e46399fc7
4. https://colab.research.google.com/notebooks/intro.ipynb#recent=true
10.3 Textbooks
1. Machine Learning: An Algorithmic Perspective, Stephen Marsland.

2. Deep Learning with python 3rd Edition, Francois Chollet.

International Conference on Communication and Signal Processing, April 4-6, 2019, India
Implementation of Deep Learning Algorithm

with Perceptron using TenzorFlow Library
Arshiya Begum, Farheen Fatima and Asfia Sabahath

Abstract—In recent years, Deep Learning, Machine Learning,
and Artificial Intelligence are highly focused concepts of data
science. Deep learning has achieved success in the field of
Computer Vision, Speech and Audio Processing, and Natural
Language Processing. It has the strong learning ability that can
improve utilization of datasets for the feature extraction
compared to traditional Machine Learning Algorithm.
Perceptron is the essential building block for creating a deep
Neural Network. The perceptron model is the more general
computational model. It analyzes the unsupervised data, making
it a valuable tool for data analytics. A key task of this paper is to
develop and analyze learning algorithm. It begins with deep
learning with perceptron and how to apply it using TensorFlow
to solve various issues. The main part of this paper is to make
perceptron learning algorithm well behaved with non-separable
training datasets. This type of algorithm is suitable for Machine
Learning, Deep Learning, Pattern Recognition, and Connectionist
Expert System. Fig. 1. Machine Learning Model
Index Terms—Deep Learning, Machine Learning, Perceptron

The prediction is evaluated for accuracy if accuracy is
Learning algorithm, TensorFlow. acceptable, the MLA is displayed otherwise MLA is trained
again with an augmented training data sets. An ML
I. INTRODUCTION incorporates the following four steps:
I N recent year’s advancement in technology, there has been a

lot of focus on adaptive learning which is more fuelled by
machine learning and deep learning. It is one of the effective
1. Feature extraction is the basis for prediction
2. Select the suitable machine learning algorithm
methods to predict data analytics with a large amount of 3. Train and evaluate model performance
sample data. It also helps in recognizing hidden pattern or 4. Use the trained model to predict the obscure data
features through historical learning and trends in data. Models
are created based on the past result gathered from training The steps of Deep Learning(DL) is similar to Machine
data, It studies the past observation results to make precise Learning(ML) as shown in Fig. 1, Unlike the machine learning
predictions. Deep learning technique could likewise be method, its feature extraction is automated rather than
uutilized to upgrade the online learning environment for the manual[2]. DL is a subset of ML where similar MLA are used
educators to more readily assess the learning process [1]. to train deep neural networks to achieve better accuracy while
Conventional machine-learning techniques were restricted in predicting the required result. DL is the only method to
their abilities, for decades Machine learning algorithm is overcome the challenges of feature extraction. The three
trained using training data sets to create a model. When the different types of ML/DL methods are:
new input data is introduced to the Machine Learning A. Supervised learning
Algorithm (MLA), it makes predictions based on the applied
model. The most common form of ML/DL is supervised learning.
Presume that, we want to build a model that classifies the
object based on their pattern. At first, we collect large data set
similar to jigsaw puzzle where the one large object is divided
into number of small images and labeled each with its
Ms.Arshiya Begum and Asfia Sabahath are with Computer Science category. Once the output is produced it is then exposed to the
Department of King Khalid University, Abha, KSA.(e-mail: arshiyabegum@ machine to obtained desired output in the form of vector score
ymail.com, asfia.saba@gmail.com)
Ms.Farheen Fatima is a Data Analyst in Accenture Private Ltd.,
[3].
Hyderabad, India (e-mail: fatima.farheen711@gmail.com).
978-1-5386-7595-3/19/$31.00 ©2019 IEEE

0172
The error between the output and the desired pattern is final output in eq. (1) Bias enables us to move the decision line
calculated using its objective function and if the attained result with the goal that can best separate the input into two classes.
is not accurate the machine modifies the internal adjustable
B. Implementation of AND Gate in Perceptron Learning
parameter to reduce the error, these parameters are called as
Algorithm
weights [4].The supervised learning algorithm analyses the
training data set and use the predicted result to map the new Below we have implemented the following code using
instances [5-10]. Phython 3.7 using Anaconda navigator, where we create
variable t_in and t_out vectors to store input, output, and bias.
B. Unsupervised Learning
A Self-organizing neural network is learned using an
unsupervised learning algorithm to identify the hidden pattern
of the unlabeled data [6]. Here the sample data is unlabeled
therefore the accuracy of data also cannot be predicted.
Unsupervised learning organizes the data without any errors in
the desired output pattern [2].
C. Semi-Supervised Learning
It uses a large dataset of unlabeled data and can reduce the
efforts of labeling data while achieving high accuracy [2].
The rest of this paper is organized as below, Section II
about Neural Network and Section III about how to implement For three inputs (Input 1, Input 2, Bias) three weight values
Neural network in perceptron Learning.In section IV are defined for each input. For these inputs tensor variable of
concludes the paper. 3*1 vectors for weight will be initialized with random values.
In TensorFlow, variables update the learning procedure by
II. NEURAL NETWORK changing their weights.
The Neural Network is divided into two main broad
streams as shown in Fig. 2. Linear Separable problem: It
shows in Fig. 2(a) the datasets are classified into two basic
categories or sets using a single line. Non-Linear Separable
problem: It shows in Fig. 2(b) It is the datasets contains
multiple categories or sets and require a non-linear line to
separate them in respective sets.
Fig. 2. (a).Linear and (b).Non-Linear Separable Problem

In the above code, Input and output of the AND Gate are
A. Perceptron Learning Algorithm given the placeholders as a and b respectively.The input
A Perceptron is a method for learning linear threshold received by perceptron is multiplied by the respective weights
device classifiers [5]. It computes the weighted sum of all the and summed together. The summed value is fed to activation
coordinates of the vector pattern .The produced output is +1 or function to obtain the final output as shown in Fig. 3.
-1 which lies below or above the threshold if larger or smaller
than the threshold [6]. In the present study, the perceptron
divides the input data sets into two sets. Set A lies under the
decision line when Inputs are having Output as zero though Set
B lies over the decision line when Inputs are having Output as
one.Function of weights, Input, and Bias are mathematically
represented as.
f ( x) wx b (1)
Where w =weight vector, x =Input Vector, b =Bias and
f (x) = Transfer functions.
Each input received from the perceptron has been weighted
dependent on the measure of its contribution for obtaining Fig. 3. AND Gate and Activation Function
0173
To calculate the error value for perceptron output and the
desired output, minimize TensorFlow error where it provides
optimizers that gradually change each variable weight and bias
in the successive iteration. Perceptron is trained to update
weights and bias values in the successive iteration in order to
minimize the error or loss. An accuracy of 83.84% is achieved during this process. A
C. Activation Functions graph in Fig. 5 shows reduced cost or error in successive
The activation function is applied to the perceptron’s output. epochs in a graph where Cost vs Number of Epochs are
Activation function is given below to perform linear plotted.
classification on the AND gate linear perceptron in eq. (2) and
(3) and input dataset.
z x1 w1 x2 w2 ................. xn wn (2)
y f (z ) (3)
Where =Activation Function, = Final Output

III. USE CASE: SONAR DATA USING SINGLE LAYER PERCEPTRON
As a testbed we have taken dataset containing data of 208
patterns obtained by bouncing sonar signals at different angles
and under different conditions from a metal cylinder (naval
mine) and rock. Our aim is therefore to construct a model that
can predict whether the object is a naval mine or a rock. A Fig. 5. Cost vs Number of Epochs
brief walk-through of all SONAR data set linear classification But, in use cases such as SONAR, will be provided with
steps using single Layer Perceptron is shown in Fig. 4. the raw data sets that has to read and preprocess to train the
In the perceptron learning algorithm input and output model. It is divided into two data sets during preprocess of
variable with respect to AND Gate are defined explicitly. data.
Training Subset: Used to train a model
Testing subset: Used to validate trained model
The model is trained and the cost or error is calculated
using Cross Entropy Mean squared error method. The
model is trained in successive epoch by calculating the cost
of weight and bias to minimize the error. The trained model
calculates the error or cost to attained accurate predicted
output.
IV. CONCLUSION
In this paper, we have learnt what is perceptron and how
X2
the perceptron learning algorithm is implemented using
X1 y
TensorFlow library. Also, we discussed how a perceptron
0 0 0
can be used as a linear classifier and how to implement
0 1 0 AND Gate using perceptron. We have observed that the
1 0 0 Perceptron learning algorithm guarantees the performance
1 1 1 depending on a notion of “separability” of training datasets.
The solution describes the convergence property and could
Fig. 4. SONAR dataset using Single Layer Perceptron be applied to other models too in the nonlinear separable
problem.
0174
REFERENCES
[6] Nicol`o Cesa-Bianchi, Alex Conconi, and Claudio Gentile “A Second-
[1] O.R. Zaiane,”Web usage mining for better web-based learning
Order Perceptron Algorithm”,Siam J.Comput., Vol. 34, No. 3, pp. 640–
environment”, in:Prof. of Conference on Advanced Technology for
668.
Education, Banff, AB, June2001, pp. 60–64.
[7] Tang, J., Deng, C. and Huang, G.B., 2016. Extreme learning machine
[2] Yang xin, Lingshuang Kong, Zhi Liu, (member, IEEE), Yuling Chen,
for multilayer perceptron. IEEE transactions on neural networks and
Yanmiao Li, Hongliang Zhu, Mingcheng Gao,Haixia Hou, and Chunhua
learning systems, 27(4), pp.809-821.
Wang , “Machine Learning and Deep Learning Methods for Cyber
security”,IEEE Access,Vol. 6 page 35365 – 35381(15 May 2018).21 [8] Raiko, T., Valpola, H. and LeCun, Y., 2012, March. Deep learning
made easier by linear transformations in perceptrons. In Artificial
[3] Yann LeCun, Yoshua Bengio & Geoffrey Hinton,”Deep Learning”,436
intelligence and statistics (pp. 924-932).
NATURE, Vol. 521, 28 May 2015.
[9] Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I.,
[4] R. Sathya, Annamma Abraham, “Comparison of Supervised and
Wierstra, D. and Riedmiller, M., 2013. Playing atari with deep
Unsupervised Learning Algorithms for Pattern Classification”, (IJARAI)
reinforcement learning. arXiv preprint arXiv:1312.5602.
International Journal of Advanced Research in Artificial Intelligence,
[10] Zhu, Z., Luo, P., Wang, X. and Tang, X., 2014. Multi-view perceptron:
Vol. 2, No. 2, 2013.
a deep model for learning face identity and view representations.
[5] Kai-Yeung Siu, Amir Dembo, Thomas Kailath. “On the Perceptron In Advances in Neural Information Processing Systems (pp. 217-225).
Learning Algorithm on Data with High Precision”, Journal Of Computer
And System Sciences 48, 347-356 (1994).
0175

Batch 9

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Batch 9

Uploaded by

Copyright:

Available Formats

IMPLEMENTATION OF DEEP LEARNING ALGORITHM

WITH PERCEPTRON USING TENSORFLOW LIBRARY

In partial fulfillment of the requirements for the award of the degree of

N.G. CHAITANYA REDDY - 164E1A0504

Under the esteemed guidance of

Department of Computer Science & Engineering

SIDDARTHA INSTITUTE OF SCIENCE and TECHNOLOGY

Head of the Department Guide

Project viva-voce examination held on ______________

INTERNAL EXAMINER EXTERNAL EXAMINER

We wish to express our profound and sincere gratitude to Mrs. M. Amudha,

We express our deep sense of gratitude to Project Coordinator Mr. M. Doorvasulu

Chapter No Title Page. No

1.1 Domain Description 1

3. SYSTEM REQUIREMENTS SPECIFICATION

3.1 Hardware Requirements 25

5.5.1 Class diagram 41

9. CONCLUSION AND FUTURE ENHANCEMENT

11. BASE PAPER

“Our Parents, Faculty members

1. IT WOULD BE BETTER IF YOU BEGIN TO EACH

2. THINKING SHOULD BECOME YOUR CAPTIAL

Chapter No Title Page. No

1.1(b) Appending Drive to google colab 18

1.2(a) Machine Learning v/s Deep Learning 20

1.2(b) Neural Network 21

1.2(c) Perceptron rule 22

5.1(a) System Architecture 31

5.1(b) Block diagram 32

5.3(b) Multi-Layer Perceptron 35

5.3(c) Forward Pass 37

5.5.1 Class Diagram 41

5.5.2 Use case Diagram 42

5.5.3 Sequence Diagram 43

5.5.4 Activity Diagram 44

5.5.5 Deployment Diagram 45

7.2.3 LOOCV Method 64

Chapter No Title Page. No

7.3(b) KFold cross validation 66

1.1 Domain Description

SISTK DEPARTMENT OF CSE Page 1

In 1996, members of the International Federation of Classification Societies (IFCS)

In 2001, William S. Cleveland introduced data science as an independent

SISTK DEPARTMENT OF CSE Page 2

In 2014, General Assembly launched student-paid bootcamp and The Data

SISTK DEPARTMENT OF CSE Page 3

Uses of Data Science

Various Processes in Data Science

Data science pipelines are sequences of obtaining, cleaning, visualizing, modelling

SISTK DEPARTMENT OF CSE Page 4

Fig 1.1(a): Various processes in data science

A database management system (DBMS) is a system software for creating and

SISTK DEPARTMENT OF CSE Page 5

On the other hand, NoSQL databases are especially called as non-relational or

SQL databases have predefined schema whereas NoSQL databases have a

SISTK DEPARTMENT OF CSE Page 6

To perform this action an adept need to have a thorough knowledge of Scripting

According to Vitaly Friedman (2008), the “main goal of data visualization is to

Data Visualization is extremely important as it is an easy and a comprehensive

• Python libraries: Numpy, Matplotlib, Pandas, Scipy

SISTK DEPARTMENT OF CSE Page 7

Conceptual Data Modeling – identifies the highest-level relationships between different

Enterprise Data Modeling – comparable to conceptual records modelling but addresses