NPTEL » Introduction To Machine Learning

Week 1: Assignment 1 Answer

Explanation
Question 1
Which of the following is a supervised learning problem?

Grouping related documents from an unannotated corpus.

Predicting credit approval based on historical data.
Predicting if a new image has cat or dog based on the historical data of other images of cats and dogs,
where you are supplied the information about which image is cat or dog.
Fingerprint recognition of a particular person used in biometric attendance from the fingerprint data of
various other people and that particular person.

Answer: A supervised learning problem is one where the algorithm is trained on a

labeled dataset, meaning it is provided with input data along with the corresponding
correct output or target labels. The goal is to learn a mapping between the input and
output data so that it can make accurate predictions on new, unseen data.

Among the options provided, the following is a supervised learning problem:

• Predicting credit approval based on historical data: In this case, the algorithm
is provided with historical data of credit applications, including information about
applicants and whether their applications were approved or denied. The goal is to
train the algorithm to predict whether a new credit application should be
approved or not based on the historical data.

The other options mentioned involve tasks that are not inherently supervised learning
problems:

• Grouping related documents from an unannotated corpus: This is typically an

unsupervised learning problem where the algorithm tries to find patterns or
similarities among documents without any labeled data.
• Predicting if a new image has a cat or dog based on the historical data of
other images of cats and dogs: This seems like a semi-supervised or transfer
 learning problem, where some labeled data may be available, but the model
would likely rely on both labeled and unlabeled data to improve its performance.
• Fingerprint recognition of a particular person used in biometric attendance
from the fingerprint data of various other people and that particular
person: This is likely a pattern recognition problem, but it doesn't fit the
definition of a supervised learning problem as described above, unless there is
labeled fingerprint data specifying which fingerprints belong to the particular
person and which belong to others. If such labeled data exists, then it can be
considered a supervised learning problem.

Overall, the clearest example of a supervised learning problem from the given options is
"Predicting credit approval based on historical data."

Question 2.
Which of the following are classification problems?

Predict the runs a cricketer will score in a particular match.

Predict which team will win a tournament.
Predict whether it will rain today.
Predict your mood tomorrow.

Answer: Classification problems involve predicting a discrete class label or category for a
given input. Among the options provided, the following are classification problems:

1. Predict which team will win a tournament: This is a classification problem

because the algorithm is tasked with predicting the class label (the winning team)
from a set of possible discrete options (the teams participating in the
tournament).
2. Predict whether it will rain today: This is also a classification problem. The
algorithm needs to predict a binary outcome (rain or no rain) based on input data
(e.g., weather conditions, humidity, temperature).

The other two options are not classification problems:

• Predict the runs a cricketer will score in a particular match: This is a

regression problem. The algorithm is asked to predict a continuous value (the
runs scored), not a discrete class label.
• Predict your mood tomorrow: This is more of a personal prediction rather than
a traditional machine learning classification problem. Moods are subjective and
not easily quantifiable into discrete classes. Even if you were to map moods to
 discrete labels (e.g., happy, sad, neutral), predicting personal emotions would
require a deeper understanding of the individual's context and feelings, making it
a challenging and complex task.

Question 3
Which of the following is a regression task?

Predicting the monthly sales of a cloth store in rupees.

Predicting if a user would like to listen to a newly released song or not based on historical data.
Predicting the confirmation probability (in fraction) of your train ticket whose current status is waiting list
based on historical data.
Predicting if a patient has diabetes or not based on historical medical records.
Predicting if a customer is satisfied or unsatisfied from the product purchased from ecommerce website
using the the reviews he/she wrote for the purchased product.

Answer: Aregression task involves predicting a continuous numerical value based on

input data. Among the options provided, the following is a regression task:

• Predicting the monthly sales of a cloth store in rupees: This is a regression

task because the algorithm is asked to predict a continuous numerical value (the
monthly sales in rupees) based on input data (e.g., historical sales data, time of
year, promotions, etc.).

The other options are not regression tasks:

• Predicting if a user would like to listen to a newly released song or not

based on historical data: This is a binary classification task. The algorithm needs
to predict a discrete label (like or dislike) based on historical data of the user's
song preferences.
• Predicting the confirmation probability (in fraction) of your train ticket
whose current status is waiting list based on historical data: This is a
regression task with a twist. While it involves predicting a numerical value
(confirmation probability in fraction), it also contains elements of classification
(the ticket getting confirmed or not). One approach could be to treat it as a
regression problem where the algorithm predicts the probability directly, or it
could be converted into a binary classification problem by setting a threshold for
the probability.
 • Predicting if a patient has diabetes or not based on historical medical
records: This is a binary classification task. The algorithm needs to predict a
discrete label (diabetes or non-diabetes) based on historical medical records of
patients.
• Predicting if a customer is satisfied or unsatisfied from the product
purchased from the e-commerce website using the reviews he/she wrote for
the purchased product: This is a binary classification task. The algorithm needs
to predict a discrete label (satisfied or unsatisfied) based on the customer's
review of the purchased product.

In summary, only "Predicting the monthly sales of a cloth store in rupees" is a regression
task. The rest are binary classification tasks, except for the "confirmation probability"
prediction, which could be treated as a regression problem with elements of
classification.

Question 4.
Which of the following is an unsupervised learning task?

Group audio files based on language of the speakers.

Group applicants to a university based on their nationality.
Predict a student’s performance in the final exams.
Predict the trajectory of a meteorite
Answer:
An unsupervised learning task involves finding patterns or relationships within data
without the use of labeled examples. Among the options provided, the following is an
unsupervised learning task:

• Group audio files based on the language of the speakers: This is an

unsupervised learning task because the algorithm needs to cluster or group the
audio files based on similarities in the data without any explicit labels indicating
which language each file contains.

The other options are not unsupervised learning tasks:

• Group applicants to a university based on their nationality: This is a

supervised learning task because applicants are labeled with their nationalities,
and the goal is to predict which group (nationality) new applicants belong to.
• Predict a student's performance in the final exams: This is a supervised
learning task because the algorithm would need labeled data of past student
 performances in the final exams to learn and predict how future students may
perform.
• Predict the trajectory of a meteorite: This is not a typical machine learning
task. Predicting the trajectory of a meteorite is more accurately described as a
physics or celestial mechanics problem and usually involves applying
mathematical models and simulations based on physical laws rather than
machine learning algorithms.

To summarize, only "Group audio files based on the language of the speakers" is an
unsupervised learning task from the given options.

Question 5.
Which of the following is a categorical feature?

Number of rooms in a hostel.

Gender of a person
Your weekly expenditure in rupees.
Ethnicity of a person
Area (in sq. centimeter) of your laptop screen.
The color of the curtains in your room.
Number of legs an animal.
Minimum RAM requirement (in GB) of a system to play a game like FIFA, DOTA.

Answer: A categorical feature is a type of data that represents qualitative or nominal

variables, which can take on a limited and fixed number of distinct categories or labels.
Among the options provided, the following are categorical features:

1. Gender of a person: This is a categorical feature because it represents a nominal

variable with distinct categories, such as "male" and "female."
2. Ethnicity of a person: Similar to gender, ethnicity is a categorical feature as it
represents different groups or categories of people based on their cultural or
regional background.
3. The color of the curtains in your room: This is a categorical feature because it
represents different colors as categories, such as "blue," "red," "green," etc.
4. Number of legs an animal: This may be a bit deceptive because the number of
legs is a numerical value. However, it represents a limited set of distinct
categories (e.g., 2 legs for birds, 4 legs for mammals), making it a discrete
categorical feature.
The following are not categorical features:

• Number of rooms in a hostel: This is a numerical feature representing a count

of rooms, which is a quantitative variable.
• Your weekly expenditure in rupees: This is a numerical feature representing a
continuous variable, as it can take on a wide range of real-number values.
• Area (in sq. centimeter) of your laptop screen: This is a numerical feature
representing the size of the laptop screen, which is a continuous variable.
• Minimum RAM requirement (in GB) of a system to play a game like FIFA,
DOTA: This is a numerical feature representing a discrete variable, as it deals with
specific values of RAM in gigabytes.

To summarize, the categorical features among the given options are: Gender of a
person, Ethnicity of a person, The color of the curtains in your room, Number of legs an
animal.

Question 6.
Which of the following is a reinforcement learning task?

Learning to drive a cycle

Learning to predict stock prices
Learning to play chess
Leaning to predict spam labels for e-mails

Answer: The reinforcement learning task among the options provided is:

• Learning to play chess: This is a reinforcement learning task. In reinforcement

learning, an agent learns to take actions in an environment to maximize a
cumulative reward over time. In the context of learning to play chess, the agent
(an AI player) interacts with the chessboard environment and learns by playing
numerous games against itself or other opponents. The agent receives rewards
(positive or negative) based on the outcome of the games (e.g., winning, losing,
or drawing) and uses this feedback to improve its strategy over time.

The other options are not reinforcement learning tasks:

• Learning to drive a cycle: This is a task that can be approached using supervised
learning (with labeled data) or unsupervised learning (finding patterns in data
without explicit labels).
 • Learning to predict stock prices: This is a task usually tackled using time-series
analysis, econometrics, or machine learning techniques, but it is not considered a
reinforcement learning task. In predicting stock prices, the model typically learns
patterns and relationships in historical stock price data to make predictions about
future prices.
• Learning to predict spam labels for emails: This is a supervised learning task.
The model can be trained on labeled data (emails labeled as spam or not spam)
to predict whether new, unseen emails are spam or not. It does not involve the
agent interacting with an environment to maximize rewards.

In summary, "Learning to play chess" is the reinforcement learning task from the given
options.

Question 7.

Let X� and Y� be a uniformly distributed random variable over the

interval [0,4][0,4] and [0,6][0,6] respectively. If X� and Y� are independent events, then compute the
probability, P(max(X,Y)>3)�(���(�,�)>3)

1/6

5/6

2/3

1/2

2/6

5/8
None of the above

Answer: The probability that max(X,Y)>3 can be calculated as follows:

P(max(X,Y)>3) = 1 - P(max(X,Y)<=3)
Since X and Y are uniformly distributed random variables over the interval [0,4] and [0,6]
respectively and are independent events, we can use the following formula:
P(max(X,Y)<=k) = P(X<=k) * P(Y<=k)
where k is a constant.
Therefore,
P(max(X,Y)<=3) = P(X<=3) * P(Y<=3)
P(X<=3) = 3/4 (since X is uniformly distributed over [0,4])
P(Y<=3) = 1/2 (since Y is uniformly distributed over [0,6])
So,
P(max(X,Y)<=3) = (3/4) * (1/2) = 3/8
Therefore,
P(max(X,Y)>3) = 1 - P(max(X,Y)<=3)
= 1 - 3/8
= 5/8
Therefore, the probability that max(X,Y)>3 is 5/8.

Question 8.
Find the mean of 0-1 loss for the given predictions:

11

00

1.51.5

0.5
Answer: : The mean of 0-1 loss for the given predictions can be calculated as follows:
Let y_i be the true label and y_i_hat be the predicted label.
0-1 loss is defined as:
L(y_i, y_i_hat) = 1 if y_i != y_i_hat and 0 otherwise.
Therefore, the 0-1 loss for the given predictions is:
L(1, 1) = 0
L(0, 0) = 0
L(1, 0) = 1
L(0, 1) = 1
The mean of 0-1 loss is defined as:
mean(L(y_i, y_i_hat)) = (1/n) * sum(L(y_i, y_i_hat))
where n is the number of predictions.
Therefore,
mean(L(y_i, y_i_hat)) = (1/4) * (L(1, 1) + L(0, 0) + L(1, 0) + L(0, 1))
= (1/4) * (0 + 0 + 1 + 1)
= (1/4) * 2
= 0.5
Therefore, the mean of 0-1 loss for the given predictions is 0.5.

Question 9
Which of the following statements are true? Check all that apply.

A model with more parameters is more prone to overfitting and typically has higher variance.
If a learning algorithm is suffering from high bias, only adding more training examples may not improve the
test error significantly.
When debugging learning algorithms, it is useful to plot a learning curve to understand if there is a high
bias or high variance problem.
If a neural network has much lower training error than test error, then adding more layers will help bring
the test error down because we can fit the test set better.

Answer: The true statements among the options are:

1. A model with more parameters is more prone to overfitting and typically

has higher variance: This statement is true. A model with more parameters has
higher capacity and can memorize the training data, leading to overfitting. It may
perform well on the training data but generalize poorly to new, unseen data,
resulting in higher variance.
2. If a learning algorithm is suffering from high bias, only adding more
training examples may not improve the test error significantly: This
statement is true. If the learning algorithm has high bias, it means it is not
 complex enough to capture the underlying patterns in the data. Simply adding
more training examples may not be sufficient to overcome this limitation.
3. When debugging learning algorithms, it is useful to plot a learning curve to
understand if there is a high bias or high variance problem: This statement is
true. Learning curves are useful tools to diagnose high bias or high variance
problems. By plotting the training and cross-validation error against the size of
the training set, we can gain insights into the model's performance and identify if
there are issues related to bias or variance.

The following statement is not true:

• If a neural network has much lower training error than test error, then
adding more layers will help bring the test error down because we can fit
the test set better: This statement is not necessarily true. If a neural network has
a significantly lower training error than the test error, it indicates overfitting, and
adding more layers may worsen the problem by increasing model complexity. It
is more appropriate to consider techniques like regularization or reducing the
network's size to improve generalization rather than blindly adding more layers.

Question 10.
Bias and variance are given by:

E[f^(x)]−f(x),E[(E[f^(x)]−f^(x))2]

E[f^(x)]−f(x),E[(E[f^(x)]−f^(x))]2

(E[f^(x)]−f(x))2,E[(E[f^(x)]−f^(x))2]

(E[f^(x)]−f(x))2,E[(E[f^(x)]−f^(x))]2

Answer: Bias
and variance are measures that help us understand the performance of a
machine learning model.

The correct representation for bias and variance is:

Bias: (E[f^(x)] - f(x))^2

Here, f(x) represents the true or target function we are trying to approximate, and f^(x)
represents the model's prediction or approximation of the true function. The bias
measures how much the predictions of the model differ from the true function on
average.

Variance: E[(E[f^(x)] - f^(x))^2]

The variance measures the variability or spread of the model's predictions around their
average (E[f^(x)]), which indicates how sensitive the model is to changes in the training
data. A higher variance suggests that the model is overfitting to the noise in the training
data.

So, the correct representation of bias and variance is:

• Bias: (E[f^(x)] - f(x))^2

• Variance: E[(E[f^(x)] - f^(x))^2]