Padmashri Annasaheb Jadhav Bharatiya Samaj Unnati Mandal’s

B. N. N. College of Arts, Science & Commerce, Bhiwandi.

(Department of M.Sc. Computer Science)

CERTIFICATE
This is to certify that

Mr. Momin Naofil Ahmad Nehal Ahmad

PRN No. 201701641121002 Roll No: 6 Exam Seat No. has

Satisfactorily completed the Practical in _Machine Learning – II (Advanced Machine Learning)

As laid down in the regulation of University of Mumbai for the purpose of M.Sc. Computer

Science sem. III Examination 2021 - 2022

Date: 13 / 11 / 2021

Place: Bhiwandi Professor In Charge

 INDEX

Sr
Practical Date
No

Find probability density function or probability mass function, cumulative 22/10/2021

distribution function and joint distribution function to calculate probabilities and
quantiles for standard statistical distributions.
1

Create a Directed Acyclic Graph (DAG) using (i) set of formulae (ii) set of vectors 22/10/2021
and (iii) set of matrices. Find parents and children of nodes. Read conditional
independence from DAG. Add and remove edges from graph.
2

Create a Bayesian network for a given narrative. Set findings and ask queries [One 23/10/2021
may use narratives like ‘chest clinic narrative’ and package gRain for the purpose].
3

Demonstrate SVM as a binary classifier 23/10/2021

Create a random graph and find its page rank. 23/10/2021

Apply random walk technique to a multivariate time series. 24/10/2021

Implement two stages Gibbs Sampler. 24/10/2021

7
 Practical No 1

AIM: Find probabilitydensity function or probabilitymass function, cumulative distribution function and joint distribution
function to calculate probabilities and quantiles for standard statistical distributions.

Binomial Distribution

Problem

Suppose there are twelve multiple choice questions in an English class quiz. Each question has five
possible answers, and only one of them is correct. Find the probability of having four or less correct
answers if a student attempts to answer every question at random.
Solution
Since only one out of five possible answers is correct, the probability of answering a question
correctly by random is 1/5=0.2. We can find the probability of having exactly 4 correct answers by
random attempts as follows.NOTE: NO PACKAGES REQUIRED)
> dbinom(4, size=12, prob=0.2)
> dbinom(0, size=12, prob=0.2) +
+ dbinom(1, size=12, prob=0.2) +
+ dbinom(2, size=12, prob=0.2) +
+ dbinom(3, size=12, prob=0.2) +
+ dbinom(4, size=12, prob=0.2)
> dbinom(4, size=12, prob=0.2)
> pbinom(4, size=12, prob=0.2)
Poisson Distribution

Problem
If there are twelve cars crossing a bridge per minute on average, find the probability of having seventeen
or more cars crossing the bridge in a particular minute.
Solution
The probability of having sixteen or less cars crossing the bridge in a particular minute is given by
the function ppois.

> ppois(16, lambda=12)

> ppois(16, lambda=12, lower=FALSE)

Answer
If there are twelve cars crossing a bridge per minute on average, the probability of having seventeen or
more cars crossing the bridge in a particular minute is 10.1%.

Continuous Uniform Distribution

Problem
Select ten random numbers between one and three.

Solution
We apply the generation function runif of the uniform distribution to generate ten random numbers
between one and three.

> runif(10, min=1, max=3)

Exponential Distribution

Problem
Suppose the mean checkout time of a supermarket cashier is three minutes. Find the probability of a customer
checkout being completed by the cashier in less than two minutes.
Solution
The checkout processing rate is equals to one divided by the mean checkout completion time. Hence the
processing rate is 1/3 checkouts per minute. We then apply the function pexp of the exponential
distribution with rate=1/3.

Probability mass function

> pexp(2, rate=1/3)

Answer
The probability of finishing a checkout in under two minutes by the cashier is 48.7%
Normal Distribution

Problem
Assume that the test scores of a college entrance exam fits a normal distribution. Furthermore, the mean
test score is 72, and the standard deviation is 15.2. What is the percentage of students scoring 84 or more
in the exam?
Solution
We apply the function pnorm of the normal distribution with mean 72 and standard deviation 15.2. Since
we are looking for the percentage of students scoring higher than 84, we are interested in the upper tail
of the normal distribution.

> pnorm(84, mean=72, sd=15.2, lower.tail=FALSE)

Answer
The percentage of students scoring 84 or more in the college entrance exam is 21.5%.
Chi-squared Distribution

Problem
Find the 95th percentile of the Chi-Squared distribution with 7 degrees of freedom.

Solution
We apply the quantile function qchisq of the Chi-Squared distribution against the decimal values 0.95.

> qchisq(.95, df=7)

Answer
The 95th percentile of the Chi-Squared distribution with 7 degrees of freedom is 14.067.
Student t Distribution

Problem
Find the 2.5th and 97.5th percentiles of the Student t distribution with 5 degrees of freedom.
Solution
We apply the quantile function qt of the Student t distribution against the decimal values 0.025 and 0.975.
> qt(c(.025, .975), df=5)

Answer
The 2.5th and 97.5th percentiles of the Student t distribution with 5 degrees of freedom are -2.5706 and 2.5706
respectively.
 Practical No 2
AIM: Create a Directed Acyclic Graph (DAG) using (i) set of formulae (ii) set of vectors and (iii) set
of matrices. Find parents and children of nodes. Read conditional independence from DAG. Add and
remove edges from graph. (NOTE : Install & Load Packages – gRbase, graph)
CODE:

> library(gRbase)
> g1 <- ug(~a:b:e + a:c:e + b:d:e + c:d:e + c:g + d:f)
> class(g1)
[1]
"graphNEL"
attr(,"packag
e")
[1] "graph"
> as(g1,
"matrix") a b
ecdgf
a0111000
b1010100
e1101100
c1010110
d0111001
g0001000
f0000100
> plot(g1)
# ‘0 1’ Count Must be 25 in count otherwise it shows error.

> m <- matrix(c(0, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0,

+ 1, 1, 0, 1, 1, 1, 1, 1, 0), nrow = 5)
> rownames(m) <- colnames(m) <- c("a", "b", "c", "d", "e")
>m
abc
dea0
1101
b10011
c10011
d01101
e11110
> as(m, "graphNEL")
A graphNEL graph with undirected edges
Number of Nodes = 5
Number of Edges = 8
#addEdge function is present in graph library(must be loaded)
> g1a <- addEdge("a", "d", g1)
> g1b <- removeEdge("c", "d", g1)
> par(mfrow = c(1, 3))
> plot(g1, main = "g1")
> plot(g1a, main = "g1a")

> plot(g1b, main = "g1b")

> g1c <- subGraph(c("b", "c", "d", "e"), g1)

> par(mfrow = c(1, 3))
> plot(g1c, main = "g1c")
> is.complete(g1,set=c("a","b","e"))
[1] TRUE
> is.complete(g1)
[1] FALSE
(NOTE : Load RBGL package)
> str(maxClique(
g1)) List of 1
$ maxCliques:List of 6
..$ : chr [1:3] "e" "b" "a"
..$ : chr [1:3] "e" "b" "d"
..$ : chr [1:3] "e" "c" "a"
..$ : chr [1:3] "e" "c" "d"
..$ : chr [1:2] "g" "c"
..$ : chr [1:2] "f" "d"
> g2 <- ug(~a:b:e + a:c:e + b:d:e + c:d:e)
> plot(g2)

> separates("a", "d", c("b", "c", "e"), g2)

[1] TRUE
> plot((g3 <- ug(~ A:B + B:C:D + C:E + D:E)))

> dag0 <- dag(~a, ~b * a, ~c * a * b, ~d * c * e, ~e * a)

>plot(dag0)
#All Representation of Directed Acrylic Graph is Same Format
> dag0 <- dag(~a + b:a + c:a:b + d:c:e + e:a)
> dag0 <- dag("a", c("b", "a"), c("c", "a", "b"), c("d", "c", "e"), c("e", "a"))
> dag0

A graphNEL graph with directed edges

Number of Nodes = 5
Number of Edges = 6
> plot(dag0)

> (m <- matrix(c(0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0,

+ 1, 0, 0, 0, 0, 0, 1, 1, 0), nrow = 5))
[,1] [,2] [,3] [,4] [,5]
[1,] 0 1 0 0 0
[2,] 0 0 1 1 0
[3,] 0 0 0 0 1
[4,] 0 0 0 0 1
[5,] 0 0 0 0 0
> rownames(m) <- colnames(m) <- letters[1:5]
> dg <- as(m, "graphNEL")
> plot(dg)
> parents("d",
dag0) [1] "c" "e"
> children("c", dag0)
[1] "d"
> par(mfrow=c(3,1),mar=c(3,1,2,1))
> plot(dag(~a+b:a+c:b),"circo")

> plot(dag(~c+b:c+a:b),"circo")

> plot(dag(~b+a:b+c:b),"circo")
> plot(dag(~a+c+b:a:c),"circo")

> ancestralSet(c("a", "c", "e"), dag0)

[1] "a" "b" "c" "e"
 Practical No 3
AIM: Create a Bayesian network for a given narrative. Set findings and ask
queries [One may use narratives like ‘chest clinic narrative’ and package gRain
for the purpose]

CODE:

R>
library("gRain")
R> library(“grid”)
R> library("Rgraphviz")
#1. Specify conditional probability tables with values as given in L&S:
R> yn <- c("yes", "no")
R> a <- cptable(~ asia, values = c(1, 99), levels = yn)
R> t.a <- cptable(~ tub + asia, values = c(5, 95, 1, 99), levels =
yn) R> s <- cptable(~ smoke, values = c(5,5), levels = yn)
R> l.s <- cptable(~ lung + smoke, values = c(1, 9, 1, 99), levels =
yn) R> b.s <- cptable(~ bronc + smoke, values = c(6, 4, 3, 7),
levels = yn) R> x.e <- cptable(~ xray + either, values = c(98, 2, 5,
95), levels = yn) R> d.be <- cptable(~ dysp + bronc + either,
+ values = c(9, 1, 7, 3, 8, 2, 1, 9), levels = yn)
R> e.lt <- ortable(~ either + lung + tub, levels = yn)
#Notice that the + operator used above is slightly misleading in the sense, for example,
that the operator does not commute (the order of the variables is important). We use
the + operator merely as a separator of the variables. The following forms are also
valid speci_cations:
R> cptable(~ dysp | bronc + either,
+ values = c(9, 1, 7, 3, 8, 2, 1, 9), levels = yn)

R> cptable(c("dysp", "bronc", "either"),

+ values = c(9, 1, 7, 3, 8, 2, 1, 9), levels = yn)

#Notice also that since E is a logical variable which is true if either T or L are
true and false otherwise, the corresponding CPT can be created with the special
function ortable().
#2. Create an intermediate representation of the CPTs:
 PRACTICAL NO: 4

AIM: Demonstrate SVM as a binary classifier.

Theory: support vector machines (SVMs) are supervised learning models with
associated learning algorithms that analyze data used for classification and regression analysis.

Package e1071 : It consist of libsvm to allow multiclass classification . libsvm uses one-against-one technique by fitting all
binary sub classifiers and finding correct class by voting mechanism.

Package mblench: This mblench package contains a collection of real-world and artificial machine learning benchmark
problems. E.g.: datasets from the UCI repository; functions for creating artificial datasets.

CODE:
Install packages e1071 and
mblench library(e1071)
library(mlbench)
> data(Glass, package="mlbench")
> index <- 1:nrow(Glass)
> testindex <- sample(index, trunc(length(index)/3))
> testset <- Glass[testindex,]
> trainset <- Glass[-testindex,]
>svm.model <- svm(Type ~ ., data = trainset, cost = 100, gamma = 1)
> svm.pred <- predict(svm.model, testset[,-
10]) #compute svm confusion matrix
> table(pred = svm.pred, true = testset[,10])
true
pred 1 2 3 5 6 7
1 16 3 1 0 0 0
2 7 17 2 3 3 1
3 3 1 2 0 0 0
5 0 0 0 1 0 0
6 0 0 0 0 2 0
7 0 0 0 0 0 9
 Practical No 5
AIM: Create a random graph and find its page rank.

Pagerank: PageRank (PR) is a mathematical formula that judges the “value of a page” by looking at the
quantity and quality of other pages that link to it. Its purpose is to determine the relative importance of a
given webpage in a network (i.e., the World Wide Web). It is a link analysis algorithm and it assigns a
numerical weighting to each element of a hyperlinked set of documents.

CODE:

Library(igraph)

#Creating a Random Graph

g <- random.graph.game(20, 5/20, directed=TRUE)

> page.rank(g)$vector
[1] 0.04610213 0.06614367 0.05894461 0.05695411 0.05587227 0.04037344 0.06015475 0.05044331
0.02475410

[10] 0.03077680 0.13450033 0.01404007 0.04370260 0.03636246 0.06787678 0.05759629 0.03560976

0.03518419

[19] 0.03004123 0.05456712

>plot(g)
> g2 <- graph.star(10)

> page.rank(g2)$vector

[1] 0.49008499 0.05665722 0.05665722 0.05665722 0.05665722 0.05665722

[7] 0.05665722 0.05665722 0.05665722 0.05665722

>plot(g2)
 Practical No 6
AIM: Apply random walk technique to a multivariate time series.

Theory: Random Walk is an algorithm that provides random paths in a graph. A random walk means that
we start at one node, choose a neighbor to navigate to at random or based on a provided probability
distribution, and then do the same from that node, keeping the resulting path in a list. A Multivariate Time
Series consist of more than one time-dependent variable and each variable depends not only on its past
values but also has some dependency on other variables.

CODE ON CONSOLE:

First-order differences of a random walk are a white noise series, the correlogram of the series of
differences can be used to judge whether a given series is reasonably modelled as a random walk.
acf(): The development of the spatial parameters involves the use of the mathematical technique of the
autocorrelation function (ACF). ACF is a measure of how similar the texture is at a given distance from
the original location.
The above random model is used to fit a time series of exchange rate:

As can be seen from the correlogram of the time series difference, a significant value occurs at lag 1,
suggesting random walk model is not adequate and a more complex model may be needed.

> Z<-read.table("C:/rtools40/home/pounds_nz.dat",header=F)

> plot(Z.ts)

> acf(diff(Z.ts))
And additional term can be added to the random walk model using the Hilt-Winters procedure, allowing
non- zero \(\beta\,\) but zero \(\gamma\)

> Z.hw<-HoltWinters(Z.ts,alpha=1,gamma=FALSE)

> Z.hw

> acf(resid(Z.hw))
 Practical No 7
AIM: Implement Two Stage Gibbs Sampler

Theory: The Gibbs Sampling is a Monte Carlo Markov Chain method that iteratively draws an instance
from the distribution of each variable, conditional on the current values of the other variables in order to
estimate complex joint distributions. It is a criterion for choosing a probability distribution for the statistical
ensemble of microstates of a thermodynamic system by minimizing the average log probability

R CODE:
(NOTE: Write this code either in RSTUDIO Console or RGui
Console) Example 1 (Gibbs sampler: Bivariate distribution)

#initializes constants and

parameters N <- 5000

#length of

chain
burn <- 1000 #burn-in length
X <- matrix (0, N, 2) #the chain, a bivariate sample

rho <- -.75

#correla
tion mu1 <- 0
mu2 <- 2
sigma1 <- 1
sigma2 <- .5
s1 <- sqrt(1-
rho^2)*sigma1 s2 <-
sqrt(1-rho^2)*sigma2

###### generate the chain #####

X[1, ] <- c(mu1, mu2) #initialize

for (i in 2:N) {
x2 <- X[i-1, 2]
m1 <- mu1 + rho * (x2 - mu2) *
sigma1/sigma2 X[i, 1] <- rnorm(1, m1, s1)
x1 <- X[i, 1]
m2 <- mu2 + rho * (x1 - mu1) *
sigma2/sigma1 X[i, 2] <- rnorm(1, m2, s2)
}

b <- burn
+ 1 x <-
X[b:N, ]

# compare sample statistics to

parameters colMeans(x)

cov(x)

cor(x)

plot(x, main="", cex=.5,

xlab=bquote(X[1]),
ylab=bquote(X[2]),
ylim=range(x[,2]))
OR

A simple Gibbs sampler

simulating from a bivariate normal with zero mean and unit variance for the marginals, but a correlation of
rho between the two components, we don't need a Gibbs sampler to simulate this - we could just simulate
from the marginal for X, and then from the conditional for Y|X. In R, we could do this as follows:
CODE:

rbvn<-function (n, rho)

{
x <- rnorm(n, 0, 1)
y <- rnorm(n, rho * x, sqrt(1 -
rho^2)) cbind(x, y)
}
This creates a vector of X values, then uses them to construct a vectors of Y values
conditional on those X values. These are then bound together into a n by 2 matrix. We can
test it with:
bvn<-
rbvn(10000,0.98)
par(mfrow=c(3,2))

plot(bvn,col=1:100
00)
plot(bvn,type="l")

plot(ts(bvn[,1]))

plot(ts(bvn[,2]))
hist(bvn[,1],40)
hist(bvn[,2],40)

par(mfrow=c(1,1))
This gives a couple of scatter plots of the points, time series plots of the marginals to confirm that we are
independently sampling, and then histograms of the two marginals.

However, this practical is supposed to be about Gibbs sampling! We can construct a Gibbs sampler for
this problem by successively sampling from the conditional distributions.

Implementation in R

A function for the Gibbs sampler for this problem is given

below. gibbs<-function (n, rho)
{
mat <- matrix(ncol = 2, nrow
= n) x <- 0
y <- 0
mat[1, ] <-
c(x, y) for (i
in 2:n) {
x <- rnorm(1, rho * y, sqrt(1 -
rho^2)) y <- rnorm(1, rho * x,
sqrt(1 - rho^2)) mat[i, ] <- c(x, y)
}
mat
}
A matrix for the results is created, then the chain is initialised at (0,0). The main loop then successively
samples from the full conditionals, storing the results in the matrix. We can test this as follows:

CODE:
bvn<-
gibbs(10000,0.98)
par(mfrow=c(3,2))

plot(bvn,col=1:10000)
[Type text]

plot(bvn,type="l")

plot(ts(bvn[,1]))

plot(ts(bvn[,2]))
[Type text]

hist(bvn[,1],40)
[Type text]

hist(bvn[,2],40)

par(mfrow=c(1,1))
With a bit of luck, this will give results which look very similar to those obtained earlier, apart from the time
series plots of the marginals, which show distinct autocorrelation between successive values.