Professional Documents
Culture Documents
Machine Learning
( y = cx + b )
b = model.intercept_
Image Processing
Machine Learning
Y = real output
Y - Ŷ = error
Model accuracy
Feature selection
Feature elimination
Machine Learning
Feature selection is described in detail. Feature selection classified in to Two Manual method
and Program Method
Filter Method
Filter method –
Constant Constant is mainly used when a feature has all the data in the dataset
constant manner
( Detailed Explanation of Multi Linear Regression using the example of House Prices in USA)
Computer Vision
Machine Learning
marks analysis ,how much time they is used in cafeteria and all how mark depends .
Computer vision
Machine Learning
Filter method
Wrapper Method
Embaded method
Feature Engineering
Label encoding
Dummy Variable
One-Hot encoding
Ilock function
Machine Learning
Feature selection
Argument function
No argument Function
Machine Learning
Introduction to the GRETL software and its use in the dataset analysis.
feature selection using wrapper method and understand about OLS algorithm and p-values.
Computer Vision
a.assign(value)to assign
In neural networks, we have input layer, output layer & hidden layer.
optimiser model
framework and how it works and how lazy execution take part in Neural Network
he lazy execution, why it is slow but it increases the performance of the program.
Data Visualization –
functions in folium like Map(), Marker(), add_to() and also different options
Heatmap
Histogram
Bar Plot
Factor Plot
Density Plot
Old approach—(Sklearn)
Multi classification
A sigmoid function :- is a type of activation function, and more specifically defined as a squashing
function. Squashing functions limit the output to a range between 0 and 1, making these functions
useful in the prediction of probabilities.
Machine Learning
Kaggl.com : - The website contains may type of Datasets (real world data sets)
Detailed explanation of Binary classification using Titanic: Machine Learning from Disaster
Devops
Rapid or agile
Automation
Monitoring
Provisioning
Machine Learning
confusion matrix
False Positive: (Type 1 Error)Interpretation: You predicted positive and it’s false.
False Negative: (Type 2 Error)Interpretation: You predicted negative and it’s false.
Devops
Devops
30-04-2020 Session1(summary)
Machine Learning
Explain about different libraries that are using for data visualization
Devops
Deploying Webpage
01-05-2020 Session20(summary)
Deep Learning
Explained a about how human brain works , How to Neural Network Identifying or Learning an
Image
Devops
Introduced a tool : ngrok -it facilitate private ip into public ip( It is called Tunneling )
02 -05-2020 Session21(summary)
Deep Learning
Activation Function : Activation functions are mathematical equations that determine the output of
a neural network. Activation functions also help normalize the output of each neuron to a range
between 1 and 0 or between -1 and 1.
Step Function:
Sigmoid Function
Leaky ReLU
Devops
When ever an update in made in git hub git hub send a message to Jenkins jenkis download the code
and execute it
04 -05-2020 Session22(summary)
#MLOPS
#Devops
Integration of:
GitHub
Jenkins
Docker
Webserver
How Jenkins user convert to power user using Sudo command (/etc/sudoers)
Bash if command.
05 -05-2020 Session23(summary)
#MLOPS
CNN are mostly focused on big data. But it also uses in audio, video ,images
#DEVOPS
08 -05-2020 Session24(summary)
MLOPS
Kernel create the box for detecting the feature and go ahead after 1 or 2 pixel it create new box this
jump go to one box to another box is called as stride
Devops
Exit 0 – Success
Exit 1 – Fail
Upstream
Downstream
09-05-2020 Session25(summary)
MLOPS
Max Pooling:
Average Pooling
Global pooling
Local Binary Patterns Histogram algorithm (LBPH) for face recognition. It is based on local binary
operator and is one of the best performing texture descriptors.
a kernel, convolution matrix, or mask is a small matrix. It is used for blurring, sharpening, embossing,
edge detection, and more.
DEVOPS
Merge conflict: -Conflicts generally arise when two people have changed the same lines in a file, or if
one developer deleted a file while another developer was modifying it. In these cases, Git cannot
automatically determine what is correct
12-05-2020 Session26(summary)
MLOPS
Devops
Scalability of K8 is demonstrated .
MLOPS
Discussed about the Home Work project,
2) integrate to GitHub
3)integrate to Jenkins,
4)create jobs in Jenkins 1 job will download the data 2 job will detect that if it is CNN code then
launch the env 3 job will find accuracy if it is less than 4 job will add one more layer and again upload
to GitHub and this process will continue in loop until we get the desired accuracy..
There are some model discussed Len Net & Alex Net
DEVOPS
MLOPS
VGG has two versions :VGG 16 having 13 convolutional layers and 3 fully connected layers and VGG
19 having 16 convolutional layers and 3 fully connected layers.
Python Imaging Library (abbreviated as PIL) (in newer versions known as Pillow)
DEVOPS
In two ways we can create the pod one is manual method other is using script program
MLOPS
If you want to create a model through CNN we needed two things dataset & Computing Power to
train the model .So In the industrial use case we are using cloud computing .
Transfer learning is when a model developed for one task is reused to work on a second task. Fine
tuning is one approach to transfer learning.
In Transfer Learning or Domain Adaptation we train the model with a dataset and after we train the
same model with another dataset that has a different distribution of classes, or even with other
classes than in the training dataset).
In Fine-tuning, an approach of Transfer Learning, we have a dataset, and we use let's say 90% of it in
training. Then we train the same model with the remaining 10%. Usually, we change the learning
rate to a smaller one, so it does not have a significant impact on the already adjusted weights.
DEVOPS
RC – Replication Controller
RS – replication Set
Deployment
The main duty of Replication Controller is keep track of all the pods
Zero down time – to get zero down time we have to extend the capability of R.S
Update strategy mainly classified in to four in that we are using “rampd” today
21-05-2020: (summary)
MLOPS
Feature Scaling is a technique to standardize the independent features present in the data in a fixed
range. It is performed during the data pre-processing.
Standardization: It is a very effective technique which re-scales a feature value so that it has
distribution with 0 mean value and variance equals to 1.
K-means clustering is one of the simplest and popular unsupervised machine learning algorithms.
Typically, unsupervised algorithms make inferences from datasets using only input vectors without
referring to known, or labelled, outcomes.
We can speed up gradient descent by scaling because θ descends quickly on small ranges and slowly
on large ranges, and oscillates inefficiently down to the optimum when the variables are very
uneven.
21-05-2020 (summary)
persistent volume (PV) is a cluster-wide resource that you can use to store data in a way that it
persists beyond the lifetime of a pod. The PV is not backed by locally-attached storage on a worker
node but by networked storage system such as EBS or NFS or a distributed filesystem like Ceph.
A persistent volume claim (PVC) is a request for storage, similar to how a pod requests compute
resources. A PVC provides an abstraction layer to underlying storage.
A Secret is an object that contains a small amount of sensitive data such as a password, a token, or a
key. Such information might otherwise be put in a Pod specification or in an image. Users can create
secrets and the system also creates some secrets.
To use a secret, a Pod needs to reference the secret. A secret can be used with a Pod in two ways:
Pod is asking for the space it called (claim ) it is also called mount.
22-05-2020 (summary)
MLOPS
The Role of Computer Vision create CNN module by detection and classification.
Mask-RCNN which is also known as State of Art. The mask-rcnn is used for object detection in which
concept of segmentation is used
Concept of clustering and In which k-mean is used to find the random mean point
the real use case if for forensic and for same we have to do the task
Mask RCNN it uses pretrained model called Coco and it can also be used for transfer learning
WCSS(Within Cluster Squad Sum) formula which gives us the right number of clusters to be used for
ous dataset.
the Elbow method which helps us to find the stable number of clusters. And sir also introduces a
new function in WCSS called Model .inertia
Explained the Scenario of How to over come SPOF Single Point Of Failure Using Multiple Node
23-05-2020 (summary)
1) Profile Monitoring
2) Trace Monitoring
3) Log monitoring
4) Matrices Monitoring
26-05-2020 (summary)
MLOPS
Discussed about generative adversarial network is a class of machine learning frameworks invented
by Ian Goodfellow and his colleagues in 2014. Two neural networks contest with each other in a
game. Given a training set, this technique learns to generate new data with the same statistics as the
training set.
If we want to generate a image which doesn't really exists but looks real or feels real then we
reverse our CNN network
Examples of GAN
Image-to-Image Translation
Text-to-Image Translation
Semantic-Image-to-Photo Translation
26-05-2020 (summary)
Create a file deamon.json and add ip to the docker host address for exporting
Kill -HUP processid command for restart process without downtime or exit
Introduced graphana tool to visualize the data that are collected by Prometheus
MLOPS
Generative adversarial networks (GANs) are algorithmic architectures that use two neural networks,
pitting one against the other (thus the “adversarial”) in order to generate new, synthetic instances of
data that can pass for real data. They are used widely in image generation, video generation and
voice generation.
This generated image is fed into the discriminator alongside a stream of images taken from
the actual, ground-truth dataset.
The discriminator takes in both real and fake images and returns probabilities, a number
between 0 and 1, with 1 representing a prediction of authenticity and 0 representing fake.
Market Basket Analysis is conducted at the transaction level (what’s in their basket). There are three
main steps:
1. Evaluate the strength of the relationship between each of your products and every other
product you offer using the algorithms of Association math.
2. Identify those pairings that have very strong affinity (typically an affinity score of 2 or
higher). For example, a customer with a credit card might be found to be twice or three
times as likely to have an auto loan than an auto loan customer selected at random.
3. Create a personalized offering for customers who have one product of a strongly associated
pair but not the other.
27-05-2020 (summary)
Devops
A regular expression is a sequence of characters that define a search pattern. Usually such patterns
are used by string searching algorithms for "find" or "find and replace" operations on strings, or for
input validation. It is a technique developed in theoretical computer science and formal language
theory
Regex are used in Google analytics in URL matching in supporting search and replace in most popular
editors like Sublime,
Character Classes
A character class matches any one of a set of characters. It is used to match the most basic element
of a language like a letter, a digit, space, a symbol etc.
28-05-2020 (summary)
Devops
29-05-2020 (summary)
Devops
Creating Connection Between Docker Server & Docker Client Using TCP Socket Support
After Creating the connectivity we have to replace docker commands with Jenkins
2-06-2020 (summary)
Apache Maven is a software project management and comprehension tool. Based on the concept of
a project object model (POM), Maven can manage a project's build, reporting and documentation
from a central piece of information.
validate - validate the project is correct and all necessary information is available
test - test the compiled source code using a suitable unit testing framework. These tests
should not require the code be packaged or deployed
package - take the compiled code and package it in its distributable format, such as a JAR.
verify - run any checks on results of integration tests to ensure quality criteria are met
install - install the package into the local repository, for use as a dependency in other
projects locally
deploy - done in the build environment, copies the final package to the remote repository
for sharing with other developers and projects.
Setup the scenario with windows as a client We are using Oracle Virtual Box as VM for operations
Kubeadm is the official tool for “first-paths” when creating your first Kubernetes cluster. With the
ease of getting up and running
flannel is a virtual network that gives a subnet to each host for use with container runtimes.
Platforms like Google's Kubernetes assume that each container (pod) has a unique, routable IP
inside the cluster. The advantage of this model is that it reduces the complexity of doing port
mapping
--discovery-token-ca-cert-hash
sha256:ee3a81ca44366176e8d81e6498de4c3984c343eed7be61de773c52c9e1800cee” }