1.3.4 Discrete Probability Models
Here, we will distinguish between two different types of sample spaces, discrete and continuous. We will discuss the difference more in detail later on, when we discuss random variables. The basic idea is that in discrete probability models we can compute the probability of events by adding all the corresponding outcomes, while in continuous probability models we need to use integration instead of summation.
Consider a sample space $S$. If $S$ is a countable set, this refers to a discrete probability model. In this case, since $S$ is countable, we can list all the elements in $S$: $$S=\{s_1, s_2, s_3,\cdots\}.$$ If $A \subset S$ is an event, then $A$ is also countable, and by the third axiom of probability we can write $$P(A)=P(\bigcup_{s_j \in A} \{s_j\})=\sum_{s_j \in A} P(s_j).$$ Thus in a countable sample space, to find probability of an event, all we need to do is sum the probability of individual elements in that set.
Example
I play a gambling game in which I will win $k-2$ dollars with probability $\frac{1}{2^k}$ for any $k \in \mathbb{N}$, that is,
- with probability $\frac{1}{2}$, I lose $1$ dollar;
- with probability $\frac{1}{4}$, I win $0$ dollar;
- with probability $\frac{1}{8}$, I win $1$ dollar;
- with probability $\frac{1}{16}$, I win $2$ dollars;
- with probability $\frac{1}{32}$, I win $3$ dollars;
- $\cdots$
What is the probability that I win more than or equal to $1$ dollar and less than $4$ dollars? What is the probability that I win more than $2$ dollars?
- Solution
-
In this problem, the random experiment is the gambling game and the outcomes are the amount
in dollars that I win (lose). Thus we may write
$$S=\{-1,0,1,2,3,4,5,\cdots\}.$$
As we see this is an infinite but countable set. The problem also states that
$$P(k)=P(\{k\})=\frac{1}{2^{k+2}} \textrm{ for $k \in S$}.$$
First, let's check that this is a valid probability measure. To do so, we should check if all
probabilities add up to one, i.e., $P(S)=1$. We have
$P(S)$ $ =\sum_{k=-1}^{\infty} P(k)$ $= \sum_{k=-1}^{\infty} \frac{1}{2^{k+2}}$ $= \frac{1}{2}+\frac{1}{4}+\frac{1}{8}+\cdots \hspace{20pt}$ $\textrm{(geometric sum)}$ $=1$.
Now let's solve the problem. Let's define $A$ as the event that I win more than or equal to $1$ dollar and less than $4$ dollars, and $B$ as the event that I win more than $2$ dollars. Thus, $$A=\{1,2,3\}, B=\{3,4,5,\cdots\}.$$ Then
$P(A)$ $=P(1)+P(2)+P(3)$ $=\frac{1}{8}+\frac{1}{16}+\frac{1}{32}$ $=\frac{7}{32}$ $\approx 0.219$
Similarly,
$P(B)$ $=P(3)+P(4)+P(5)+P(6)+\cdots$ $=\frac{1}{32}+\frac{1}{64}+\frac{1}{128}+\frac{1}{256}+\cdots \hspace{20pt}$ $\textrm{(geometric sum)}$ $=\frac{1}{16}$ $= 0.0625$
Note that another way to find $P(B)$ is to write
$P(B)$ $=1-P(B^c)$ $=1-P(\{-1,0,1,2\})$ $=1-\bigg(P(-1)+P(0)+P(1)+P(2)\bigg)$ $=1-\left(\frac{1}{2}+\frac{1}{4}+\frac{1}{8}+\frac{1}{16}\right)$ $=1-\frac{15}{16}$ $=\frac{1}{16}$ $= 0.0625$
-
In this problem, the random experiment is the gambling game and the outcomes are the amount
in dollars that I win (lose). Thus we may write
$$S=\{-1,0,1,2,3,4,5,\cdots\}.$$
As we see this is an infinite but countable set. The problem also states that
$$P(k)=P(\{k\})=\frac{1}{2^{k+2}} \textrm{ for $k \in S$}.$$
First, let's check that this is a valid probability measure. To do so, we should check if all
probabilities add up to one, i.e., $P(S)=1$. We have
Finite Sample Spaces with Equally Likely Outcomes:
An important special case of discrete probability models is when we have a finite sample space $S$, where each outcome is equally likely, i.e., $$S=\{s_1,s_2,\cdots,s_N\}, \textrm{ where } P(s_i)=P(s_j) \textrm{ for all }i,j \in \{1,2,\cdots,N\}.$$ Rolling a fair die is an instance of such a probability model. Since all outcomes are equally likely, we must have $$P(s_i)=\frac{1}{N}, \textrm{ for all }i \in \{1,2,\cdots,N\}.$$ In such a model, if $A$ is any event with cardinality $|A|=M$, we can write $$P(A)=\sum_{s_j \in A} P(s_j)=\sum_{s_j \in A} \frac{1}{N}=\frac{M}{N}=\frac{|A|} {|S|}.$$ Thus, finding probability of $A$ reduces to a counting problem in which we need to count how many elements are in $A$ and $S$.
Example
I roll a fair die twice and obtain two numbers: $X_1=$ result of the first roll, and $X_2=$ result of the second roll. Write down the sample space $S$, and assuming that all outcomes are equally likely (because the die is fair), find the probability of the event $A$ defined as the event that $X_1+X_2=8$.
- Solution
-
The sample space $S$ can be written as
$S=\{$ $(1,1),(1,2),(1,3),(1,4),(1,5),(1,6),$ $(2,1),(2,2),(2,3),(2,4),(2,5),(2,6),$ $(3,1),(3,2),(3,3),(3,4),(3,5),(3,6),$ $(4,1),(4,2),(4,3),(4,4),(4,5),(4,6),$ $(5,1),(5,2),(5,3),(5,4),(5,5),(5,6),$ $(6,1),(6,2),(6,3),(6,4),(6,5),(6,6)\}$.
As we see there are $|S|=36$ elements in $S$. To find probability of $A$, all we need to do is find $M=|A|$. In particular, $A$ is defined as
$A$ $=\{(X_1,X_2) | X_1+X_2=8, X_1,X_2 \in \{1,2,\cdots,6\}\}$ $=\{(2,6),(3,5),(4,4),(5,3),(6,2)\}$.
Thus, $|A|=5$, which means that $$P(A)=\frac{|A|}{|S|}=\frac{5}{36}.$$ A very common mistake is not distinguishing between, say $(2,6)$ and $(6,2)$. It is important to note that these are two different outcomes: $(2,6)$ means that the first roll is a $2$ and the second roll is a $6$, while $(6,2)$ means that the first roll is a $6$ and the second roll is a $2$. Note that it is very common to write $P(X_1+X_2=8)$ when referring to $P(A)$ as defined above. In fact, $X_1$ and $X_2$ are examples of random variables that will be discussed in detail later on.
-
The sample space $S$ can be written as
In a finite sample space $S$, where all outcomes are equally likely, the probability of any event $A$ can be found by $$P(A)=\frac{|A|}{|S|}.$$
The formula $P(A)=\frac{|A|} {|S|}$ suggests that it is important to be able to count elements in sets. If sets are small, this is an easy task; however, if the sets are large and defined implicitly, this could be a difficult job. That is why we discuss counting methods later on.
