6 views

Uploaded by Avijit Ghosh

Q Paper

- Ya Seen 2015
- fuzzy svm
- Water Quality - support vector machines
- CS583 Supervised Learning
- Fuzzy twin support vector machine based on iterative method
- Face Detect
- ROBOTIC MANIPULATOR CONTROL BY USING MACHINE LEARNING ALGORITHMS: A REVIEW
- spam_detection.pdf
- IRJET-ARTIFICIAL NEURAL NETWORKS VS SUPPORT VECTOR MACHINES FOR MEMBRANE DISEASES DETECTION
- Managing Intrusion Detection Alerts Using Support Vector Machines
- motor svn
- Intelligent Prediction of Sieving Efficiency in Vibrating Screens
- Bioinformatics 2009 Alexandrov 643 9
- SAAIP05
- A Decision Tree Based Word Sense
- The Study and Analysis of Classification Algorithm for Animal Kingdom Dataset
- Text Content Dependent Writer Identification
- A Multi-classifier Prediction Model for Phishing Detection
- Transfer functions hidden possibilities for better neural networks
- 9_BoW.pdf

You are on page 1of 5

- -- ,:~

"- -.--.-.'

.--, '. ":" .---,-. "":"'" . ''' .~~

CS60050 Machine Learning

Endterm Examination, Spring 2013

Time: 3 hours Full Marks: 95

1. (a) Suppose that you train a classifier with training sets of size m. As m --too, what [16]

do you expect will be the behavior of the training error? What would you expect

for the behavior of the test error? Draw a picture to illustrate.

(b) Suppose that you have a linear SVM binary classifier. Consider a point that is

currently classified correctly, and is far away from the decision boundary. If you

remove the point from the training set, and re-train the classifier, will the decision

boundary change or stay the same? Explain your answer in one sentence.

(c) Suppose that you have a decision tree binary classifier. Consider a point that is

currently classified correctly, and is far away from the decision boundary. If you

remove the point from the training set, and re-train the classifier, will the decision

boundary change or stay the same? Explain your answer in one sentence.

(d) True or false: Given enough training data, feed-forward neural networks can

learn to solve any binary classification problem. Explain.

2. A publisher has decided to run a marketing campaign and send free samples of books [6+9]

from their newly published books, to people who are likely to be very interested in

them. For each customer, they know the age, gender, occupation, education level,

salary, city and state. Each book that they publish has a title, keywords describing it

(e.g. fantasy, scienec fiction, historical, biography, etc), author and year of publication,

as well as a unique ISBN code. Some of the customers have provided in the past ratings

for books they have bought. The company has roughly 1000 past ratings available

(a) Suppose you have to set this up as a supervised learning problem. Explain how

you would construct the data set:

What attributes would you use?

What would you aim to predict?

What would be the training data?

(b) Suppose that you decided to phrase this as a classification problem. For each of

the methods below, explain in at most 2 sentences if it is appropriate or not. If

yes, describe any data preprocessing and other choices that you would need (in

at most 2 other sentences).

i. Support vector machines

ii. Neural networks

iii. !-nearest neighbour

[4+8+4]

3. (a) Let F be a set of classifiers whose VC-dimension is 5. Suppose we have four

training examples and labels, {(x1, YI), (x2, Y2), (x3, y3), (x4, y4)}, and select a

classifier f from F by minimizing classification error on the training set. In the

.,

IJ

!I

'I

absence of any other information about the set of classifiers F , can we say that

the prediction /(x5) for a new example X5 has any relation to the training set?

Briefly justify your answer.

(b) Consider the space of points in the plane. Consider the class of hypotheses

defined by conjunctions of two perceptrons (each with two inputs). An example

of such a hypothesis is shown in the figure below.

',I ,'

1. Show a set of 3 points in the plane that can be shattered by this hypothesis

class.

ii. Show a set of points in the plane that cannot be shattered by this hypothesis

class

iii. What is the exact VC-dimension of this hypothesis class? Show your rea-

soning.

(c) We learned that if a consistent learning algorithm for a finite hypothesis space

1-l is provided with

1 1

m 2: ~ (ln 1-l + ln

5)

randomly drawn training instances, then we can state a certain guarantee. What

is that guarantee? Make sure to clearly indicate the roles of E and 8.

II

4. (a) Define what you mean by the the support vectors of a linear SVM classifier [3+4+6]

when using a hard margin SVM, assuming that the input instances are linearly

separable.

(b) Define a kernel function. Give an example of a kernel function.

(c) Given the following dataset in 1-d space, which consists of 4 positive data points

{0, 1, 2, 3} and 3 negative data points {4, 5, 6}. suppose that we want to learn

a soft-margin linear SVM for this data set. Remember that the soft-margin

linear SVM can be formalized as the following constrained quadratic optimization

problem. In this formulation, Cis the regularization parameter, which balances

the size of margin vs. the violation of the margin (i.e., smaller 2:::::, 1 Ei)

1 m

argmin{w,b} 2wtw + C '2.::: Ei

i=l

subject to

Yi(WtXi +b) :::: 1- Ei

Ei:::: 0 Vi

2

i. if C = 0, which means that we only care the size of the margin, how many

support vectors do we have? What is the margin in this case?

u. If C ---7 oo, which means that we only care the violation of the margin, how

many support vectors do we have?-

5. Consider building an ensemble of decision stumps (decision boundaries) Gm with the [6]

AdaBoost algorithm,

The figure below dispalys a few labeled point in two dimensions as well as the first

classifier boundary we have chosen. A boundary predicts binary 1 values, and de-

pends only on one coordinate value (the split point). The little arrow in the figure is

the normal to the decision boundary indicating the positive side where the boundary

line predicts +1. All the points start with uniform weights.

X2

0

+1. -1

+1 0-1

+1

0-1

x1

(a) Circle all the point(s) in the figure whose weight will increase as a result of

incorporating the first stump (the weight update due to the first stump).

(b) :Praw in the same figure a possible stump (boundary) that we could select at the

next boosting iteration. You need to draw both the decision boundary and its

positive orientation.

(c) Will the second stump receive higher coefficient in the ensemble than the first?

In other words, will a2 > a 1 ? Briefly explain your answer. (no calculation should

be necessary).

6. (a) Let H be a hidden Markov model with state space Sand observation space 0. [6+3]

Suppose we are given a sequence of observations (yr, Y2, ... , Yn) and we would

like to find the MAP estimate of the hidden states (x 1 , x2, ... , xn). The Viterbi

algorithm can be used to compute the MAP estimate in O(nk 2 ) time where

k= lSI.

This algorithm uses the following: Ot,i is defned as the probability of the most

likely sequence that emits 01, o2, ... , Ot, and ends on state Si.

State the formula by which the o values are defined recursively using dynamic

programming. Satte briefly how the most likely path is found using this algo-

rithm.

3

(b) Suppose the transition matrix M has the following special structure. M (i, i) = a

and M (i, j) = b when j # i. Suppose b < a. Show how the Viterbi algorithm

works in this case. Try to find an efficient algorithm that runs in O(nk) time in

this case.

7. Consider the following deterministic Markov Decision Process (MDP), describing a [14]

simple robot grid world. Notice the values of the immediate rewards are written next

to transitions. Transitions with no value have an immediate reward of 0. Assume the

discount factor 1 = 0.8.

s1 s2 9

r=100

s4 s5 s6

(a) For each states, write the value for V*(s) inside the corresponding square in the

diagram.

(b) Mark the state-action transition arrows that correspond to one optimal policy.

If there is a tie, always choose the state with the smallest index.

(c) Give a different value for 1 which results in a different optimal policy and the

number of changed policy actions should be minimal. Give your new value for

/, and describe the resulting policy by indicating which 1r(s) values (i.e., which

policy actions) change.

For the remainder of this question, assume again that 1 = 0.8.

(d) How many complete loops (iterations) of value iteration are sufficient to guaran-

tee finding the optimal policy for this MDP? Assume that values are initialized

to zero, and that states are considered in an arbitrary order on each iteration.

(e) Is it possible to change the immediate reward function so that V* changes but the

optimal policy 1r* remains unchanged? If yes, give such a change, and describe

the resulting change to V*. Otherwise, explain in at most 2 sentences why this

is impossible.

(f) Unfortunately for our robot, in January, a patch of ice has appeared in its world,

making one of its actions non-deterministic. The resulting MDP is shown below.

Note that now the result of the action "go north" from state s6 results in one

of two outcomes. With probability p the robot succeeds in transitioning to state

s3 and receives immediate reward 100. However, with probability (1- p) it slips

on the ice, and remains in state s6 with zero immediate reward. Assume the

discount factor 1 = 0.8.

4

~

.~

,.

,.

~-~-

~

' .

.

r

IJ

...~- cJ ,-:;

'~,1

'

"

'

>~

'

., ' Assume p = 0.4 .. Writ~ the. value of V*

,_ ' f ,,., each state, and cirele the actio11r:; .il).

the optihuiC policy.

. , . il

8. 'In .this problem tvyo linear dimensionality r~!luctidn methods will be considered: prin- (6]

cipal component analysis (~CA) .and Fisher linear discriminant analysi~ (LDA), LDA

reduces/the dimensionality given labels by m~imizing the overall interClass V?J'!an:ce

reJative to intraclass variance. Plot the appro~imate directions ci( the first PCA and

'\~

LPA !3omponehts,.in the following figure. lj

lj

~-

r1~~

,,

..

i ' ,~ 0

0'

I,,

!i

lI!

' , 0 6 6

.. oo

i~

.. ,. 0 0 . !Ii

0 0 0 l.

. oo. o" 1

~

' 0 , 0

0 0 .0

~ ~

~

4~

~

1; ~

~

i\ ~

'"

~

p

' ~jr

'l1.

1

' ~ ~

,,

~II

__........._._,~. ,~,,...- ,..... ~

~-'~"'7" ""'\~~~"'-""""~.:~~--- .....,.~~-,.......,.... "{

II

1:''

- Ya Seen 2015Uploaded byAntonio Soto Leon
- fuzzy svmUploaded bySudheer Ds
- Water Quality - support vector machinesUploaded byaquatierra2007
- CS583 Supervised LearningUploaded byNghe Nhin
- Fuzzy twin support vector machine based on iterative methodUploaded byAnonymous vQrJlEN
- Face DetectUploaded byMohit Khatri
- ROBOTIC MANIPULATOR CONTROL BY USING MACHINE LEARNING ALGORITHMS: A REVIEWUploaded byTJPRC Publications
- spam_detection.pdfUploaded byAli Rao
- IRJET-ARTIFICIAL NEURAL NETWORKS VS SUPPORT VECTOR MACHINES FOR MEMBRANE DISEASES DETECTIONUploaded byIRJET Journal
- Managing Intrusion Detection Alerts Using Support Vector MachinesUploaded byAI Coordinator - CSC Journals
- motor svnUploaded bymh1999
- Intelligent Prediction of Sieving Efficiency in Vibrating ScreensUploaded bySHAIJU
- Bioinformatics 2009 Alexandrov 643 9Uploaded byLee Chi Chan
- SAAIP05Uploaded bypatel_musicmsncom
- A Decision Tree Based Word SenseUploaded byAnonymous IlrQK9Hu
- The Study and Analysis of Classification Algorithm for Animal Kingdom DatasetUploaded bySEP-Publisher
- Text Content Dependent Writer IdentificationUploaded byesatjournals
- A Multi-classifier Prediction Model for Phishing DetectionUploaded byInternational Journal of Research in Engineering and Technology
- Transfer functions hidden possibilities for better neural networksUploaded byStruct DesignPro
- 9_BoW.pdfUploaded byKiran Reddy
- IndexUploaded byAnonymous elPiJMro
- Consolidated ReportUploaded byShahid Manzoor
- 06569069Uploaded bysenthilnathan
- Feature Extraction Techniques for Facial Expression Recognition SystemsUploaded byGRD Journals
- CollabLoc - Privacy-Preserving Multi-Modal Localization via Collaborative Information FusionUploaded byvshssvs7
- aim10Uploaded byJorge Nereu
- ga6Uploaded byAnonymous TxPyX8c
- IJITCS-V4-N7-6Uploaded byKarthigaMurugesan
- green dieselUploaded byJay Jayden
- 7class&PredUploaded byYogesh Bansal

- Intro to Electronics P1Uploaded byAvijit Ghosh
- Ass4_SolUploaded byAvijit Ghosh
- PS_Ass_1_2017Uploaded bykanav
- Credit+Risk+ModellingUploaded byAvijit Ghosh
- Design of Triple Effect Evaporator for GlycerolUploaded byI. Murali Krishna
- Tutorial 1Uploaded byAvijit Ghosh
- Q_&_A_Set_8Uploaded bySusliana Esra M Silaban
- Android JDUploaded byAvijit Ghosh
- Minor CurriculaUploaded byAvijit Ghosh
- Supply and DemandUploaded byAvijit Ghosh
- Computer Architecture, A Quantitative Approach (Solution for 5th Edition)Uploaded byHarish Athuru
- SGRIPUploaded byAvijit Ghosh
- Week 2 Lecture NotesUploaded byAvijit Ghosh
- ApplicationUploaded byAvijit Ghosh
- HE Design Intro LectUploaded byAvijit Ghosh
- Report grtygUploaded byAvijit Ghosh
- Asearch v8Uploaded byAvijit Ghosh
- AI Assignment2 TheoryUploaded byAvijit Ghosh
- WebSelect: A Research Prototype for Optimizing Ad Exposures based on Network StructureUploaded byAvijit Ghosh
- box-modelUploaded byAvijit Ghosh
- mkscUploaded byAvijit Ghosh
- mkscUploaded byAvijit Ghosh
- Block Chain HackathonUploaded byAvijit Ghosh
- 2010_Mid_Autumn_HS20001Uploaded byAvijit Ghosh
- IC BBAUploaded byPrashant
- Registration Document Final 05May15Uploaded byAvijit Ghosh
- Document(1)Uploaded byAvijit Ghosh
- IIT JEE 2011 PAPER-2 FIITJEEUploaded bytanmay100
- IIT JEE 2011 paper-1 FIITJEEUploaded bytanmay100

- A Landing Gear Actuator Using a Dual Output Power Converter Based Motor DriveUploaded byR2os
- matbalUploaded bypendexx
- AkersUploaded byPhiladelphiaMagazine
- EEG383 Measurement - Problem Set 2 - Measurement_ErrorsUploaded bydromaki
- Operational Excellence Management SystemUploaded byChevano Luna
- Ets3-pdfUploaded byKimberly
- An Lift 0019v116enUploaded byTroubleshooting
- Class GuideUploaded byapi-3860591
- Ethical WorkshopUploaded byVikas Chaudhary
- property workingUploaded byapi-314809600
- handout_1335_PD1335Uploaded bysunilas218408
- Liquidity Forecast App Configuration SettingsUploaded byOSCAR RAMIREZ
- MET410 (English)[1]Uploaded byida_power
- Order to Cash Business FlowUploaded byjayanthm
- 30034765Uploaded by陳奕儒
- Site to Site VPN in Packet TracerUploaded byTâm Tồ
- May 12 2015 Advanced MatlabUploaded byyasir hamid
- 30221_indxUploaded bySusana Rojas
- HAWK Sultan Acoustic Wave Series.pdfUploaded byJuan Carlos David
- 102380726 Erp in Apollo and MrfUploaded byMrg Pune
- Https---tabs Ultimate-guitar Com-tab-stone Temple Pilots-trippin on a Hole in a Paper Heart Bass 1250000Uploaded bySebastian Ortiz Reyes
- Pcb TutorialUploaded byAmit Pujar
- Innovation NotesUploaded byMamta Janjria
- E Waste RecyclingUploaded byaskmee
- Scientific-Hangman Final Thesis Waqas-MoazzamUploaded byVeny Redulfin
- Sample Research PaperUploaded bysaad321
- Catalogue Complete ML 100119Uploaded byPercival Castrillo
- Amino A140.pdfUploaded bysernikos
- 1619SW Aoc Service ManualUploaded byjavier_valdivia_42
- R-30iB Controller Product Information_179Uploaded byAnonymous 5pURWf