5.2.3 Conditioning and Independence
If X is a continuous random variable, and A is the event that a<X<b (where possibly b=∞ or a=−∞), then
FX|A(x)={1x>bFX(x)−FX(a)FX(b)−FX(a)a≤x<b0x<a
fX|A(x)={fX(x)P(A)a≤x<b0otherwise
The conditional expectation and variance are defined by replacing the PDF by conditional PDF in the definitions of expectation and variance. In general, for a random variable X and an event A, we have the following:
E[X|A]=∫∞−∞xfX|A(x)dx,E[g(X)|A]=∫∞−∞g(x)fX|A(x)dx,Var(X|A)=E[X2|A]−(E[X|A])2
Example
Let X∼Exponential(1).
- Find the conditional PDF and CDF of X given X>1.
- Find E[X|X>1].
- Find Var(X|X>1).
- Solution
-
- Let A be the event that X>1. Then P(A)=∫∞1e−xdx=1e. Thus, fX|X>1(x)={e−x+1x>10otherwise For x>1, we have FX|A(A)=FX(x)−FX(1)P(A)=1−e−x+1. Thus, FX|A(x)={1−e−x+1x>10otherwise
- We have E[X|X>1]=∫∞1xfX|X>1(x)dx=∫∞1xe−x+1dx=e∫∞1xe−xdx=e[−e−x−xe−x]∞1=e2e=2.
- We have E[X2|X>1]=∫∞1x2fX|X>1(x)dx=∫∞1x2e−x+1dx=e∫∞1x2e−xdx=e[−2e−x−2xe−x−x2e−x]∞1=e5e=5. Thus, Var(X|X>1)=E[X2|X>1]−(E[X|X>1])2=5−4=1.
-
Conditioning by Another Random Variable:
If X and Y are two jointly continuous random variables, and we obtain some information regarding Y, we should update the PDF and CDF of X based on the new information. In particular, if we get to observe the value of the random variable Y, then how do we need to update the PDF and CDF of X? Remember for the discrete case, the conditional PMF of X given Y=y is given by
PX|Y(xi|yj)=PXY(xi,yj)PY(yj). Now, if X and Y are jointly continuous, the conditional PDF of X given Y is given by fX|Y(x|y)=fXY(x,y)fY(y). This means that if we get to observe Y=y, then we need to use the above conditional density for the random variable X. To get an intuition about the formula, note that by definition, for small Δx and Δy we should have fX|Y(x|y)≈P(x≤X≤x+Δx|y≤Y≤y+Δy)Δx(definition of PDF)=P(x≤X≤x+Δx,y≤Y≤y+Δy)P(y≤Y≤y+Δy)Δx≈fXY(x,y)ΔxΔyfY(y)ΔyΔx=fXY(x,y)fY(y). Similarly, we can write the conditional PDF of Y, given X=x, as fY|X(y|x)=fXY(x,y)fX(x).
For two jointly continuous random variables X and Y, we can define the following conditional concepts:
- The conditional PDF of X given Y=y: fX|Y(x|y)=fXY(x,y)fY(y)
- The conditional probability that X∈A given Y=y: P(X∈A|Y=y)=∫AfX|Y(x|y)dx
- The conditional CDF of X given Y=y: FX|Y(x|y)=P(X≤x|Y=y)=∫x−∞fX|Y(x|y)dx
Example Let X and Y be two jointly continuous random variables with joint PDF fXY(x,y)={x24+y24+xy60≤x≤1,0≤y≤20otherwise For 0≤y≤2, find
- the conditional PDF of X given Y=y;
- P(X<12|Y=y).
- Solution
-
- Let us first find the marginal PDF of Y. We have fY(y)=∫10x24+y24+xy6dx=3y2+y+112,for 0≤y≤2. Thus, for 0≤y≤2, we obtain fX|Y(x|y)=fXY(x,y)fY(y)=3x2+3y2+2xy3y2+y+1,for 0≤x≤1. Thus, for 0≤y≤2, we have fX|Y(x|y)={3x2+3y2+2xy3y2+y+10≤x≤10otherwise
- We have P(X<12|Y=y)=∫1203x2+3y2+2xy3y2+y+1dx=13y2+y+1[x3+yx2+3y2x]120=32y2+y4+183y2+y+1. Note that, as we expect, P(X<12|Y=y) depends on y.
-
Conditional expectation and variance are similarly defined. Given Y=y, we need to replace fX(x) by fX|Y(x|y) in the formulas for expectation:
For two jointly continuous random variables X and Y, we have:
- Expected value of X given Y=y: E[X|Y=y]=∫∞−∞xfX|Y(x|y)dx
- Conditional LOTUS: E[g(X)|Y=y]=∫∞−∞g(x)fX|Y(x|y)dx
- Conditional variance of X given Y=y: Var(X|Y=y)=E[X2|Y=y]−(E[X|Y=y])2
Example
Let X and Y be as in Example 5.21. Find E[X|Y=1] and Var(X|Y=1).
- Solution
- E[X|Y=1]=∫∞−∞xfX|Y(x|1)dx=∫10x3x2+3y2+2xy3y2+y+1|y=1dx=∫10x3x2+3+2x3+1+1dx(y=1)=15∫103x3+2x2+3xdx=712, E[X2|Y=1]=∫∞−∞x2fX|Y(x|1)dx=15∫103x4+2x3+3x2dx=2150. So we have Var(X|Y=1)=E[X2|Y=1]−(E[X|Y=1])2=2150−(712)2=2873600.
Independent Random Variables:
When two jointly continuous random variables are independent, we must have fX|Y(x|y)=fX(x). That is, knowing the value of Y does not change the PDF of X. Since fX|Y(x|y)=fXY(x,y)fY(y), we conclude that for two independent continuous random variables we must have fXY(x,y)=fX(x)fY(y).
Two continuous random variables X and Y are independent if
fXY(x,y)=fX(x)fY(y), for all x,y.
Equivalently, X and Y are independent if
FXY(x,y)=FX(x)FY(y), for all x,y.
If X and Y are independent, we have
E[XY]=EXEY,E[g(X)h(Y)]=E[g(X)]E[h(Y)].
Suppose that we are given the joint PDF fXY(x,y) of two random variables X and Y. If we can write fXY(x,y)=f1(x)f2(y), then X and Y are independent.
Example
Determine whether X and Y are independent:
- fXY(x,y)={2e−x−2yx,y>00otherwise
- fXY(x,y)={8xy0<x<y<10otherwise
- Solution
- We can write fXY(x,y)=[e−xu(x)][2e−2yu(y)], where u(x) is the unit step function: u(x)={1x≥00otherwise Thus, we conclude that X and Y are independent.
- For this case, it does not seem that we can write fXY(x,y) as a product of some f1(x) and f2(y). Note that the given region 0<x<y<1 enforces that x<y. That is, we always have X<Y. Thus, we conclude that X and Y are not independent. To show this, we can obtain the marginal PDFs of X and Y and show that fXY(x,y)≠fX(x)fY(y), for some x,y. We have, for 0≤x≤1, fX(x)=∫1x8xydy=4x(1−x2). Thus, fX(x)={4x(1−x2)0<x<10otherwise Similarly, we obtain fY(y)={4y30<y<10otherwise As we see, fXY(x,y)≠fX(x)fY(y), thus X and Y are NOT independent.
Example
Consider the unit disc D={(x,y)|x2+y2≤1}. Suppose that we choose a point (X,Y) uniformly at random in D. That is, the joint PDF of X and Y is given by fXY(x,y)={c(x,y)∈D0otherwise
- Find the constant c.
- Find the marginal PDFs fX(x) and fY(y).
- Find the conditional PDF of X given Y=y, where −1≤y≤1.
- Are X and Y independent?
- Solution
-
- We have 1=∫∞−∞∫∞−∞fXY(x,y)dxdy=∬ Thus, c=\frac{1}{\pi}.
- For -1 \leq x \leq 1, we have \begin{align}%\label{} \nonumber f_X(x)&=\int_{-\infty}^{\infty} f_{XY}(x,y)dy\\ \nonumber &=\int_{-\sqrt{1-x^2}}^{\sqrt{1-x^2}} \frac{1}{\pi}\hspace{5pt} dy\\ \nonumber &=\frac{2}{\pi}\sqrt{1-x^2}. \end{align} Thus, \begin{equation} \nonumber f_X(x) = \left\{ \begin{array}{l l} \frac{2}{\pi}\sqrt{1-x^2} & \quad -1 \leq x \leq 1 \\ & \quad \\ 0 & \quad \text{otherwise} \end{array} \right. \end{equation} Similarly, \begin{equation} \nonumber f_Y(y) = \left\{ \begin{array}{l l} \frac{2}{\pi}\sqrt{1-y^2} & \quad -1 \leq y \leq 1 \\ & \quad \\ 0 & \quad \text{otherwise} \end{array} \right. \end{equation}
- We have \begin{align}%\label{} \nonumber f_{X|Y}(x|y)&=\frac{f_{XY}(x,y)}{f_Y(y)}\\ \nonumber &=\left\{ \begin{array}{l l} \frac{1}{2\sqrt{1-y^2}} & \quad -\sqrt{1-y^2} \leq x \leq \sqrt{1-y^2} \\ & \quad \\ 0 & \quad \text{otherwise} \end{array} \right. \end{align} Note that the above equation indicates that, given Y=y, X is uniformly distributed on [-\sqrt{1-y^2},\sqrt{1-y^2}]. We write \begin{align}%\label{} \nonumber X|Y=y \hspace{5pt} \sim \hspace{5pt} Uniform(-\sqrt{1-y^2},\sqrt{1-y^2}). \end{align}
- Are X and Y independent? No, because f_{XY}(x,y)\neq f_X(x) f_Y(y).
-
Law of Total Probability:
Now, we'll discuss the law of total probability for continuous random variables. This is completely analogous to the discrete case. In particular, the law of total probability, the law of total expectation (law of iterated expectations), and the law of total variance can be stated as follows:Law of Total Probability:
\begin{align}\label{eq:LOTP-cont} P(A)=\int_{-\infty}^{\infty}P(A|X=x)f_X(x) \hspace{5pt} dx \hspace{20pt} (5.16) \end{align}Law of Total Expectation:
\begin{align}\label{eq:LOTE-cont} \nonumber E[Y]&=\int_{-\infty}^{\infty}E[Y|X=x]f_X(x) \hspace{5pt} dx \hspace{20pt} (5.17)\\ &=E[E[Y|X]] \end{align}Law of Total Variance:
\begin{align}\label{eq:LOTV-cont} \textrm{Var}(Y)=E[\textrm{Var}(Y|X)]+\textrm{Var}(E[Y|X]) \hspace{20pt} (5.18) \end{align}Example
Let X and Y be two independent Uniform(0,1) random variables. Find P(X^3+Y>1).
- Solution
- Using the law of total probability (Equation 5.16), we can write \begin{align}%\label{} \nonumber P(X^3+Y>1)&= \int_{-\infty}^{\infty}P(X^3+Y>1|X=x)f_X(x) \hspace{5pt} dx\\ \nonumber &=\int_{0}^{1}P(x^3+Y>1|X=x) \hspace{5pt} dx\\ \nonumber &=\int_{0}^{1}P(Y>1-x^3) \hspace{5pt} dx &\textrm{(since $X$ and $Y$ are independent)} \\ \nonumber &=\int_{0}^{1}x^3 \hspace{5pt} dx &\textrm{(since $Y \sim Uniform(0,1)$) }\\ \nonumber &=\frac{1}{4}. \end{align}
Example
Suppose X \sim Uniform(1,2) and given X=x, Y is an exponential random variable with parameter \lambda=x, so we can write \begin{align}%\label{} \nonumber Y|X=x \hspace{10pt} \sim \hspace{10pt} Exponential(x). \end{align} We sometimes write this as \begin{align}%\label{} \nonumber Y|X \hspace{10pt} \sim \hspace{10pt} Exponential(X). \end{align}
- Find EY.
- Find Var(Y).
- Solution
-
- We use the law of total expectation (Equation 5.17) to find EY. Remember that if Y \sim Exponential(\lambda), then EY=\frac{1}{\lambda}. Thus we conclude \begin{align} \nonumber E[Y|X=x]=\frac{1}{x}. \end{align} Using the law of total expectation, we have \begin{align} \nonumber EY&=\int_{-\infty}^{\infty} E[Y|X=x] f_X(x)dx\\ \nonumber &=\int_{1}^{2} E[Y|X=x] \cdot 1 dx\\ \nonumber &=\int_{1}^{2}\frac{1}{x}dx\\ \nonumber &= \ln 2. \end{align} Another way to write the above calculation is \begin{align} \nonumber EY&=E[E[Y|X]] &(\textrm{law of total expectation})\\ \nonumber &=E\left[\frac{1}{X}\right] &(\textrm{since } E[Y|X]=\frac{1}{X})\\ \nonumber &=\int_{1}^{2}\frac{1}{x}dx\\ \nonumber &= \ln 2. \end{align}
- To find Var(Y), we can write \begin{align} \nonumber Var(Y)&=E[Y^2]-(E[Y])^2\\ &=E[Y^2]-(\ln 2)^2\\ &=E\big[E[Y^2|X]\big]-(\ln 2)^2 & (\textrm{law of total expectation})\\ &=E\left[\frac{2}{X^2}\right]-(\ln 2)^2 & \big(\textrm{since } Y|X \sim Exponential(X)\big)\\ &=\int_{1}^{2}\frac{2}{x^2}dx-(\ln 2)^2 \\ &=1-(\ln 2)^2. \end{align} Another way to find Var(Y) is to apply the law of total variance: \begin{align} \textrm{Var}(Y)=E[\textrm{Var}(Y|X)]+\textrm{Var}(E[Y|X]). \end{align} Since Y|X \sim Exponential(X), we conclude \begin{align} &E[Y|X]=\frac{1}{X},\\ &Var(Y|X)=\frac{1}{X^2}. \end{align} Therefore \begin{align} \textrm{Var}(Y)&=E\left[\frac{1}{X^2}\right]+\textrm{Var}\left(\frac{1}{X}\right)\\ &=E\left[\frac{1}{X^2}\right]+E\left[\frac{1}{X^2}\right]-\left(E\left[\frac{1}{X}\right]\right)^2\\ &=E\left[\frac{2}{X^2}\right]-(\ln 2)^2\\ &=1-(\ln 2)^2. \end{align}
-