Statistics for Data Science 2

Professor. Andrew Thangaraj

Electrical Engineering Department
Indian Institute of Technology, Madras
Lecture No. 1.2
Multiple Random Variables
(Refer Slide Time: 0:13)

Hello and welcome to this lecture. We are looking at the joint PMF of two random variables. We
are studying two random variables together in a probability space what are all the different types
of distributions that are of interest. So, we saw the joint PMF. Now, we are going to look at what
are called marginal PMFs when you look at multiple random variables. This word marginal will
show up a lot.

And it is very important to understand what marginal is, what marginalization is, so there is also
this word called marginalization, you will hear a lot when you study probability and statistics, it
is important to know where it comes from. So, let us first define marginal PMF. It is not actually
a major definition, but you will see it is easy to think of. So, I am going to start with two jointly
distributed random variables, let us say and have a joint PMF .

So, the PMF of the individual random variables, so you are given a joint PMF, let us say you do
not start with the marginal PMF, you start with the joint PMF, you do not start with the
individual random variables, you start with the joint distribution of two random variables,
somebody gives you the table. How do you go from there to thinking of as an individual
random variable?

So, is a random variable on its own, is a random variable on its own. Each of them have
their own PMFs, those kind of PMFs when you have joint distributions, those individual PMFs
are called marginal PMFs. That is just a notation or definition in some sense, very simple
definition, but the important things are these two equations, these are marginalization equations.
What is marginalization?

You have the joint PMF, how do you go to the marginal PMF from the joint PMFs. It turns out
this way is uniquely defined by these specific equations. Remember once again from the joint
PMF you have a clear equation, which will take you to 1 marginal PMF, nothing else is possible
and that is given by this. Actually it is very easy to prove this. Let us look at the first statement
. What is ? It is the probability that , is not it?

Now, what is, let us go down to the proof here and see what is If you look at the event
, one can write this event as and y equals y1, so let us say the range of is
to , some k entries are there. So, if you want to just think of , I can always write this
event as and = or and = so on till and =

This is a way for me to decompose this as a disjoint union of several events, so this is
something we have done several times before. Now, if you, now once you do this
now these are all odds but even though they are odds they are all disjoint, is not it, takes
different values, so they are all disjoint, so when you write , its = +
= so on till = and that is what is written here.

So, notice how this goes to this, exact same thing, sum over all the values in in the range of
. So, you take, fix the value of , the value of is fixed to throughout and then you
take all possible values for and simply add the joint PMF at all possible values for , with the
same value of . So, that process is called marginalization. So, here in the first equation you are
marginalizing .

So, your marginalizing out of the summation, so you are only left with that is what happens.
Now very similarly you can do , the same proof will go through, but in this case you have
to marginalize out , so sum over all prime and the joint PMF, so you keep fixed
at t go through all possible values for and add up the joint PMF you will get the marginal
PMF.

Now, the marginal PMF is so uniquely defined once you define the joint PMF, it is simply a
PMF, it has its own range and you can calculate from the joint PMF through this marginalization
process. So, let us see a few examples to see how marginalization works.

(Refer Slide Time: 4:56)

It is easiest to think of the table and do the marginalization with respect to the table, so in the toss
of fair coin toys example that we saw just a little while ago, we had this table for the joint PMF
, , , . If you want to do the marginal PMF of you simply add over the columns of joint

PMF table. So, if you want to do , is 0, 1, I want to do .

So, this is , this is ., so you simply add along the columns, so you add these two you
get this guy, you add these two you get this, this is how you marginalize and then likewise you
add along the row you will get , so is , is like that. So, once you write down

a table, joint PMF is written as a table. How do you marginalize?

You just sum over either the rows or the columns, the values that vary as you go over the rows
are basically , is not it? If you write there, so comes from , so you will get the marginal
of , the marginal PMF as you add up everything in the row. If you add up everything in the
column you will get the marginal PMF of , whatever does not change, whatever remains the
same you get the marginal PMF of that.

So, simple, is not it, just write down the table, add up along the rows, add up along the columns,
you get the marginal PMF. So, once again remember given the joint PMF the marginal is unique,
there is nothing you can do to change the marginal PMF. You will just get 1 value for the
marginal PMFs, given the joint PMF.

(Refer Slide Time: 6:43)

So, let us take a slightly different example. So, let me just show this to you. You have got here,
so first anybody gives you a table you can verify that this is a valid PMF; verify it is a valid
PMF. How do I verify that? You can see every entry is between 0 and 1; that is easy enough and
you can add all these guys, is equal to 1. So, it is a valid PMF. No problem. Once it is a valid
PMF it is just a question of adding along the rows, adding along the columns.

You are going to get here 0.40, 0.60, you are going to get your 0.30, 0.70, so you got your PMF
for here, you got your PMF for here, remember this has to be a valid PMF, this has to be a
valid PMF all of that, you can check that this is a valid joint PMF, you can check that this guy is
a valid PMF 0.4 + 0.61. You can check that is a valid PMF 0.3 + 0.7 is 1 everything
works out very cleanly; you get marginal PMFs so easily here. So, you see, I mean, given any
table like this you can also add it up and find out it is no big deal, it is very easy.
(Refer Slide Time: 8:07)

I want to quickly point out that the same marginal PMF can result from different joint PMFs; in
fact, you can have any number of joint PMFs giving you the same marginal PMF. So, this is a
mistake that most commonly a lot of people make. What lot of people will assume is given the
marginal PMF they will simply assume that the joint PMF is the product of the marginal PMFs
and you will get some value for the joint PMF.

That is a valid answer, given the marginal PMF that is one way to come up with the joint PMF,
but that is not the only joint PMF which will give you the marginal. So, I have shown you here
case one and case two, in case one you see the , , , , distribution is there and you got a

marginal , , , very easy. Here look at case two the joint PMF is something else.

I put a variable X here between 0 and , maybe you want to take that variable to be, I do not

know, you take it as, I mean, do not take it as take it as something else, like or something,

like maybe take it as 0.1. So, you have your 0.1, here you have 0.5 - 0.1 is just 0.4. Here again
you have 0.4 and here again you have 0.1. So, clearly this is different from this joint PMF, this is
all 0.25 here, you have 0.1, 0.4, 0.4, 0.1.

But you find the marginals you will get the same margin, , , , so notice what has happened

here, so remember this rule, this is very-very important, you can go from joint PMF to marginal
in a unique way, you cannot go from marginal to joint PMF in a unique way. Usually it never
really happens. I mean, you have to really cook up a very strange situation, very simple trivial
sort of situation, only then you will go to the unique joint PMF.

Otherwise, always given the marginal alone, there can be any number, usually an infinite number
of joint PMFs are possible giving you the same marginal PMF, so this is something important to
understand and know that one way is definitely unique, the other way is not unique at all, you
can get any number of joint PMFs, if you only want a certain marginal PMF, so keep this in
mind.

I know already this is a slightly different idea to keep in mind, but just by the way in which we
added, you can see that it can all work out and still you can have. So, notice this is a valid PMF,
for any between 0 and this is a valid PMF, so 0, you add up, all this you will get which

is 1, so it is all interesting cases you can have.

So, in fact, you can push it to the extreme, you can say is 0, if you set is 0 you will get 0, 0,
, that also has the marginal of , , , marginally it will all look the same, but joint look

at how different it is, it is , here, 0, 0 here, while you have , , , , very different sort of joint

PMFs that give you the same marginal, so this is also important in practice. So, whether you
have something tending towards the independent case which is , , , or something else is

something you have to look at very carefully.

(Refer Slide Time: 11:23)

So, let us move on, so let us go back to this random two digit number example, you remember
the join PMF that we had that is , occurring alternatively, so here again you can add to get

and and you see you get what is expected, you get , , , and , . We knew

this before, we knew that is uniform, 0, 1 to 9, and then is uniform, 0, 1, to 3.

We knew these two before and we knew that and can sort of influence each other, it is not
like you, so notice what is happening here, this, so if you look at probability that 0, 0, I
wanted you to think about this, this is basically what, this is probability that 0 and 0

and this will work out to . Now, what is probability that 0 times probability that 0.

So, probability that 0 is , is not it? Probability that 0 is , is not it, so you get . So,

these two are not equal, so clearly 0 and 0 are two different events that we have
defined using the random variable and and they are not independent. You can see that the
product of the two probabilities is not the same as the probability of the intersection of the two
events.

So, product is not the same, so these are not independent. So, notice how from the joint PMF you
can find the marginals and you can determine independence of events defined using the random
variables. So, this is an important process, so quite often you may define a joint PMF and then
say is this event independent of that event, you can calculate that from the joint PMF, find the
marginals find the probabilities, which they defined with and .
So, basically find the individual probabilities of the two events and find the intersection of the
two events, see if the product is satisfied. So, these kinds of things you can do with the joint
PMF. So, the joint PMF is a powerful tool, it gives you all the data that you need to work out any
problem with the two random variables.