---

We have discussed conditional probability before, and you have already seen some problems regarding random variables and conditional probability. Here, we will discuss conditioning for random variables more in detail and introduce the conditional PMF, conditional CDF, and conditional expectation. We would like to emphasize that there is only one main formula regarding conditional probability which is \begin{align}\label{} \nonumber P(A|B)=\frac{P(A \cap B)}{P(B)}, \textrm{ when } P(B)>0. \end{align} Any other formula regarding conditional probability can be derived from the above formula. Specifically, if you have two random variables $X$ and $Y$, you can write \begin{align}\label{} \nonumber P(X \in C|Y \in D)=\frac{P(X \in C, Y \in D)}{P(Y \in D)}, \textrm{ where } C, D \subset \mathbb{R}. \end{align}

Conditional PMF and CDF:

Remember that the PMF is by definition a probability measure, i.e., it is $P(X=x_k)$. Thus, we can talk about the conditional PMF. Specifically, the conditional PMF of $X$ given event $A$, is defined as \begin{align}%\label{} \nonumber P_{X|A}(x_i)&=P(X=x_i|A) \\ \nonumber &=\frac{P(X=x_i \textrm{ and } A)}{P(A)}. \end{align}

---

Example
I roll a fair die. Let $X$ be the observed number. Find the conditional PMF of $X$ given that we know the observed number was less than $5$.

---

Conditional PMF of $X$ Given $Y$:

In some problems, we have observed the value of a random variable $Y$, and we need to update the PMF of another random variable $X$ whose value has not yet been observed. In these problems, we use the conditional PMF of $X$ given $Y$. The conditional PMF of $X$ given $Y$ is defined as \begin{align}%\label{} \nonumber P_{X|Y}(x_i|y_j)&=P(X=x_i|Y=y_j) \\ \nonumber &=\frac{P(X=x_i,Y=y_j)}{P(Y=y_j)}\\ \nonumber &=\frac{P_{XY}(x_i,y_j)}{P_Y(y_j)}. \end{align} Similarly, we can define the conditional probability of $Y$ given $X$: \begin{align}%\label{} \nonumber P_{Y|X}(y_j|x_i)&=P(Y=y_j|X=x_i) \\ \nonumber &=\frac{P_{XY}(x_i,y_j)}{P_X(x_i)}. \end{align}

Independent Random Variables:

We have defined independent random variables previously. Now that we have seen joint PMFs and CDFs, we can restate the independence definition.

Figure 5.4: Grid for example 5.4

So, if $X$ and $Y$ are independent, we have \begin{align}%\label{} \nonumber P_{X|Y}(x_i|y_j)&=P(X=x_i|Y=y_j)\\ \nonumber &=\frac{P_{XY}(x_i,y_j)}{P_Y(y_j)}\\ \nonumber &=\frac{P_X(x_i) P_Y(y_j)}{P_Y(y_j)}\\ \nonumber &=P_X(x_i). \end{align} As we expect, for independent random variables, the conditional PMF is equal to the marginal PMF. In other words, knowing the value of $Y$ does not provide any information about $X$.

---

Example
Consider the set of points in the grid shown in Figure 5.4. These are the points in set $G$ defined as \begin{align}%\label{} \nonumber G=\{(x,y) | x,y \in \mathbb{Z}, |x|+|y|\leq 2\}. \end{align} Suppose that we pick a point $(X,Y)$ from this grid completely at random. Thus, each point has a probability of $\frac{1}{13}$ of being chosen.

1. Find the joint and marginal PMFs of $X$ and $Y$.
2. Find the conditional PMF of $X$ given $Y=1$.
3. Are $X$ and $Y$ independent?

---

Conditional Expectation:

Given that we know event $A$ has occurred, we can compute the conditional expectation of a random variable $X$, $E[X|A]$. Conditional expectation is similar to ordinary expectation. The only difference is that we replace the PMF by the conditional PMF. Specifically, we have \begin{align}%\label{} \nonumber E[X|A]=\sum_{x_i \in R_{X}} x_i P_{X|A}(x_i). \end{align} Similarly, given that we have observed the value of random variable $Y$, we can compute the conditional expectation of $X$. Specifically, the conditional expectation of $ X$ given that $Y=y$ is \begin{align}%\label{} \nonumber E[X|Y=y]=\sum_{x_i \in R_{X}} x_i P_{X|Y}(x_i|y). \end{align}

---

Example Let $X$ and $Y$ be the same as in Example 5.4.

1. Find $E[X|Y=1|$.
2. Find $E[X| -1<Y<2]$.
3. Find $E[|X| | -1<Y<2]$.

---

Conditional expectation has some interesting properties that are used commonly in practice. Thus, we will revisit conditional expectation in Section 5.1.5, where we discuss properties of conditional expectation, conditional variance, and their applications.

Law of Total Probability:

Remember the law of total probability: If $B_1, B_2, B_3,...$ is a partition of the sample space $S$, then for any event $A$ we have \begin{align}\label{} \nonumber P(A)=\sum_{i} P(A \cap B_i)=\sum_{i} P(A | B_i) P(B_i). \end{align} If $Y$ is a discrete random variable with range $R_Y=\{y_1,y_2,...\}$, then the events $\{Y=y_1\}$, $\{Y=y_2\}$, $\{Y=y_3\}$, $\cdots$ form a partition of the sample space. Thus, we can use the law of total probability. In fact we have already used the law of total probability to find the marginal PMFs: \begin{align} \nonumber P_X(x)=\sum_{y_j \in R_Y} P_{XY}(x,y_j)=\sum_{y_j \in R_Y} P_{X|Y}(x|y_j)P_Y(y_j). \end{align} We can write this more generally as \begin{align} \nonumber P(X \in A)=\sum_{y_j \in R_Y} P(X \in A|Y=y_j)P_Y(y_j), \hspace{10pt} \textrm{for any set $A$}. \end{align} We can write a similar formula for expectation as well. Indeed, if $B_1, B_2, B_3,...$ is a partition of the sample space $S$, then \begin{align}\label{} \nonumber EX=\sum_{i} E[X|B_i]P(B_i). \end{align} To see this, just write the definition of $E[X|B_i]$ and apply the law of total probability. The above equation is sometimes called the law of total expectation [2].

---

Example
Let $X \sim Geometric(p)$. Find $EX$ by conditioning on the result of the first "coin toss."

---

Example
Suppose that the number of customers visiting a fast food restaurant in a given day is $N \sim Poisson(\lambda)$. Assume that each customer purchases a drink with probability $p$, independently from other customers and independently from the value of $N$. Let $X$ be the number of customers who purchase drinks. Find $EX$.