Probability

Chapter 1

Combinatorics

(Version 4, August 30, 2009)

This file contains the first three chapters (plus some appendices) of a potential book on Probabilityand Statistics. It does not assume knowledge of calculus. The first three chapters are titled “Com-binatorics,” “Probability,” and “Distributions.” And Appendix B gives a nice little introduction tothe natural logarithm,

e

. Future chapters on statistics will be added in the summer of 2010.

Combinatorics is the study of how to count things. By “things” we mean the variouscombinations, permutations, subgroups, etc., that can be formed from a given set of objectsor events. For example, how many different committees of three people can be chosen fromfive people? How many different full-house hands are there in poker? How many differentoutcomes are possible if you flip a coin four times? Knowing how to count such things iscritical for an understanding of probability, because when calculating the probability of agiven event, we invariably need to count the number of ways that this event can happen.The outline of this chapter is as follows. In Section 1.1 we introduce the concept of

factorials

, which will be ubiquitous in our study of probability. In Section 1.2 we learn howto count the number of possible permutations (that is, the number of possible orderings)of a set of objects. In Section 1.3 we learn how to count the number of possible subgroupsthat can be formed from a set of objects. We consider both the case where the order of the objects matters, and the case where it doesn’t matter. For example, the poker questionposed above is one where the order of the objects (the cards) doesn’t matter. In Section 1.4we learn how to count the number of possible outcomes of a repeated experiment, whereeach repetition has an identical set of possible results. Examples include rolling dice orﬂipping coins. Finally, in Section 1.5 we look at the coin-ﬂipping example in more detail,and we see how it relates to a set of numbers called the

binomial coeﬃcients

.Having learned how to count all these things, we’ll then see in Chapter 2 how the resultscan be used to calculate probabilities. It turns out that it’s generally a trivial step to obtaina probability once you’ve counted the relevant things, so the bulk of the work we’ll need todo will be in the present chapter.

1.1 Factorials

Before getting into the discussion of actual combinatorics, we’ll ﬁrst need to look at a certainquantity that comes up again and again. This quantity is called the

factorial

. We’ll seethroughout this chapter that when dealing with a situation that involves an integer

N

,we often need to consider the product of the ﬁrst

N

integers. This product is called “

N

1

2

CHAPTER 1. COMBINATORICS

factorial,” and it is denoted by the shorthand notation, “N!”.

1

For the ﬁrst few integers, wehave:1! = 12! = 1

·

2 = 23! = 1

·

2

·

3 = 64! = 1

·

2

·

3

·

4 = 245! = 1

·

2

·

3

·

4

·

5 = 1206! = 1

·

2

·

3

·

4

·

5

·

6 = 720 (1.1)As

N

increases,

N

! gets very big very fast. For example, 10! = 3

,

628

,

800, and 20!

≈

2

.

43

·

10

18

. In Chapter 3 we’ll make good use of an approximate formula for

N

!, called

Stirling’s formula

. This formula will make it clear what we mean by the statement, “

N

!gets very big very fast.”We should add that 0! is deﬁned to be 1. Of course, 0! doesn’t make much sense, becausewhen we talk about the product of the ﬁrst

N

integers, it is understood that we start with1. Since 0 is below this starting point, it is unclear what 0! actually means. However, there’sno need to think too hard about trying to make sense out of it, because as we’ll see below,if we simply deﬁne 0! to be 1, then a number of formulas turn out to be very nice.Having deﬁned

N

!, we can now start counting things. There are two main things we’llneed to know how to count, and the results for both of these involve

N

!. These two things are(1) the

permutations

(the orderings) of

N

objects , and (2) the number of ways of choosingsubgroups from

N

objects, for example, the number of diﬀerent committees of three peoplethat can be chosen from ﬁve people. Let’s look at each of these in turn.

1.2 Permutations

A

permutation

of a set of objects is a way of ordering them. For example, if we have threepeople, Alice, Bob, and Carol, then one permutation of them is Alice, Bob, Carol. Anotherpermutation is Carol, Alice, Bob. And another is Bob, Alice, Carol. The goal of this sectionis to learn how to count the number of possible permutations. We’ll do this by starting off with the very simple case where we have just one object. Then we’ll consider two objects,then three, and so on, until we see a pattern. As we’ll find throughout this book, this isinvariably a good strategy when trying to figure something out: Start with small numbers,and then gradually increase until you see a pattern.

One object

If we have only one object, then there is clearly only one way to “order” it. There is noordering to be done. A list of one object simply consists of that one object, and that’s that.If we use the notation where

P

N

stands for the number of permutations of

N

objects, thenwe have

P

1

= 1.

Two objects

With two objects, things aren’t completely trivial like they were in the one-object case, butthey’re still very simple. If we label our two objects as “1” and “2,” then we can order themin two ways:

1

I’m not sure why someone long ago picked the exclamation point for this notation. But just rememberthat it has nothing to do with the more common grammatical use of the exclamation point for emphasis.So try not to get too excited when you see “N!”!

1.2. PERMUTATIONS

312 or 21So we have

P

2

= 2. At this point, you might be thinking that this result, along with theprevious

P

1

= 1 result, implies that

P

N

=

N

for any number

N

. This would imply thatthere should be three diﬀerent ways to order three objects. Well, not so fast

...

Three objects

Things get more interesting with three objects. If we call them “1,” “2,” and “3,” then wecan list out the possible orderings. (If you haven’t already looked at the table below, youshould cover it up with your hand and try to list out all the permutations yourself. We’lleven add on this extra sentence here to make this parenthetical remark a little longer, sothat you don’t have any excuse for saying that you already looked at it!) The permutationsare:123 213 312132 231 321

Table 1.1

So we have

P

3

= 6. Note that we’ve grouped these six permutations into three subgroups(the three columns), according to which number comes ﬁrst. It isn’t necessary to groupthem this way, but we’ll see below that this method of organization has deﬁnite advantages.It will simplify how we think about the case where the number of objects is a general number

N

.

Remark:

There’s no need to use the numbers 1,2,3 to represent the three objects. You can usewhatever symbols you want. For example, the letters A,B,C work ﬁne, as do the letters H,Q,Z.You can even use symbols like

⊗

,

♠

,

♥

. Or you can mix things up with



,W,7 if you want tobe unconventional. The point is that the numbers/letters/symbols/whatever simply stand forthree diﬀerent things, and they need not have any meaningful properties except for their diﬀerentappearances when you write them down on the paper.However, there is certainly something simple about the numbers 1,2,3,

...

, or the letters A,B,C,

...

,so we’ll generally work with these. In any event, it’s invariably a good idea to be as economical aspossible and not write down the full names, such as Alice, Bob, and Carol. Of course, with thesethree particular names, there’s some logic in going with A,B,C.

♣

Four objects

The pattern so far is

P

1

= 1,

P

2

= 2, and

P

3

= 6. Although you might be able to guess thegeneral rule from these three results, it will be easier to see the pattern if we look at thenext case with four objects. Taking a cue from the above list of six permutations of threeobjects, let’s organize the permutations of four object according to which number starts thelist. (Again, you should cover up the following table with your hand and try to list out allthe permutations yourself.) We end up with:1234 2134 3124 41231243 2143 3142 41321324 2314 3214 42131342 2341 3241 42311423 2413 3412 43121432 2431 3421 4321

Table 1.2