CS464 Machine Learning, Spring 2014: Homework 1

Due: February 24 Mon, 17:00 pm

Instructions You will submit a hard copy for the answers of the write up questions (including the gures) and will upload the code online on Moodle by the due date. You may hand in the hard copy in classes to the instructor or may drop it in the box in room EA427 (please stick to this submission routes) and please STAPLE your write up. If you are submitting the homework late (see the late submission policy in syllabus), prepare a soft copy for all the parts of the homework and submit it on Moodle. Moodle will allow late submissions untill after 4 days of submission, but we will grade your homework based on the time stamp and your remaining late days. You may code in any programming language you would prefer, we suggest Matlab as there are useful built in functions. In submitting the code on Moodle, package your code as a gzipped TAR le or a ZIP le with the name CS464 hwX erengolge where you substitute in your rst and last names into the lename in place of erengolge and X with the homework number. The code you submit should be in a format easy to run. For example if you are using Matlab do not copy paste your commands in a word document. Instead submit a script le so that we can easily run them. Please refer to the syllabus for policies regarding collaboration, and extensions policy.

Online Blogger [10 pts]

An online blogger, on the average, makes one typo in every 500 words in his blog. A typical blog page contains 300 words. What is the probability that there will be no more than two errors in ve blog pages?

Parallel System [10 pts]

A system composed of n separate components is said to be a parallel system if it functions when at least one of the components functions. For such a system, if component i, independent of other components, functions with probability pi , where i = 1, . . . , n, what is the probability that the system functions.

Slot Machine [10 pts]

Imagine you and your friend, playing the slot machine in a casino. Having played on two separate machines for a while, you decide to switch machines between yourselves to measure for dierences in luck. The wins/losses of you and your friend for each machine are tabulated below. Machine1 You Friend Wins 40 30 Losses 60 70 Machine2 You Friend Wins 210 14 Losses 830 70

Assuming that the outcome of playing the slot machine is independent of its history and that of the other machine, answer the following questions: 1. If the outcome of a game is a success, whats the probability that it was played using machine1? 2. Estimate the winning probability of you and your friend for each of the machines. Compare your winning probability with your friends on dierent machines. Who is more likely to win? 3. Suppose both of you didnt keep track of the wins/losses for each machine, but only of the total in the casino. Estimate the overall winning probability of you and your friend in the casino (assume that there are only two slot machines in the casino). Who is more likely to win?

Multiple Choice Probability [10 pts]

Ali has to answer a True or False question. The probability that he knows to answer correctly is p. Since there is no negative score in the exam, he decides to guess in the case he does not know the answer. With probability q he chooses the true option. However, he is ignorant that the Professor is biased in deciding which one is the correct answer. To be precise, the Professor with probability assigns the correct answer to the true option and with 1 to the false one. John gives an answer in the question and he answers correctly! What is the chance that he actually knew the answer? Hint/Sanity Check: After computing this probability as a function of the parameters that you are given ,do the following sanity check to make sure you gave the right answer: substitute in the formula = 1, q = 0,and = 1, q = 1. What should be the right values? Did you get them?

Matrix Strikes Back [10 pts]

To test Neos worthiness, Morpheus decided to give him a new task: rather than simply choosing between a red and blue pill, Neo needs to draw a pill from a bag with dierent proportions of red and blue pills. This bag has 7 red and 3 blue pills. Now assume that after Neo draws a blue pill from the bag it will be replaced into the bag. Whats the expected number of times Neo needs to draw from the bag until he gets a red pill?

Programming Question [50 pts]

A famous sweet brand hides a picture of 100 football players in each sweets. Each picture could either be star or regular shaped. We can treat the shape of a picture as random variable Y which can either be star or regular. For each football player X 1, , 100 the probability that the sweet has a star shaped picture is given by; x 100 The probability for the picture of the 1st football player to show up with a star shape is 0.01 and the football player numbered 100 with probability 1. Therere three dierent kinds of the sweets, lets denote it with S random variable, and those have varying number of pictures of each football player. You can nd the counts of each football players picture for all three types of sweets in footballplayers.csv. The rst column is the football player id X, and the remaining three columns has the counts of each picture for all three sweets. P (Y = star | X = x) =

6.1

Plotting the Data

Plot bar graphs of the following distributions. (If you are using Matlab, you may use the bar function. You may also use subplot function to plot multiple plots on the same page) 1. The probability of Y = star for each football player. 2

2. The probability of drawing each type of player from the 1st sweet. 3. The probability of drawing each type of player from the 2nd sweet. 4. The probability of drawing each type of player from the 3rd sweet.

6.2

The Likelihood of Multiple Observations

Assume that you randomly opened 5 pictures of each player. Plot the probability of obtaining the following sequence {star, star, regular, star, regular}. (Similiary the bar function of Matlab will be useful)

6.3

Writing Bayes Rule

Suppose I give you P (Y = star | X ) and P (X ). Show the equations to compute P (X | Y = regular) and P (X | Y = star).

6.4

Posterior Dependence on Number of Observations

The posterior probability of a random event is the conditional probability that is assigned after the evidence is taken into account. Suppose that I buy sweets of the rst type S = 1 in dierent amounts and the following shapes are observed in each case. For each of these cases plot the posterior distribution over all footballplayers. 1. I opened one sweet of the 1st type and encountered a star-shaped picture. 2. I opened three sweets of the 1st type and encountered all star-shaped pictures. 3. I opened ve sweets of the 1st type sweet and encountered 2 star-shaped pictures and 3 regular shaped ones. 4. I opened fourth of the 1st type of sweet and encountered 25 star-shaped pictures and 15 regular shaped ones. Based on the last (largest) set among the results, which football player X is selected? How does increasing the number of observations aect the posterior distribution and your ability to make a prediction as to which of the footballplayer is encountered?

6.5

Marginalization

Suppose that I randomly choose a type of sweet S and the probability for each sweet to be chosen is P (S = 1) = 0.25, P (S = 2) = 0.25, P (S = 3) = 0.5. Then I draw 10 sweets from the selected type and I encountered 8 stars and 2 regular shaped pictures. 1. Plot the posterior distribution over each football player given the outcome of the shapes. 2. Plot the posterior distribution over each football player of sweet given the outcome of the shapes. 3. Which football player has the largest posterior? 4. Which type of sweet attains the largest posterior?