6.2.4 Cauchy-Schwarz Inequality
You might have seen the Cauchy-Schwarz inequality in your linear algebra course. The same inequality is valid for random variables. Let us state and prove the Cauchy-Schwarz inequality for random variables.
Cauchy-Schwarz Inequality
For any two random variables $X$ and $Y$, we have \begin{align}%\label{} |EXY| \leq \sqrt{E[X^2] E[Y^2]}, \end{align} where equality holds if and only if $X=\alpha Y$, for some constant $\alpha \in \mathbb{R}$.
You can prove the Cauchy-Schwarz inequality with the same methods that we used to prove $|\rho(X,Y)| \leq 1$ in Section 5.3.1. Here we provide another proof. Define the random variable $W=(X-\alpha Y)^2$. Clearly, $W$ is a nonnegative random variable for any value of $\alpha \in \mathbb{R}$. Thus, we obtain
| $ 0 \leq EW$ | $=E(X-\alpha Y)^2$ |
| $=E[X^2-2\alpha XY+ \alpha^2 Y^2]$ | |
| $=E[X^2]-2\alpha E[XY]+ \alpha^2 E[Y^2]$. |
So, if we let $f(\alpha)=E[X^2]-2\alpha E[XY]+ \alpha^2 E[Y^2]$, then we know that $f(\alpha) \geq 0$, for all $\alpha \in \mathbb{R}$. Moreover, if $f(\alpha)=0$ for some $\alpha$, then we have $EW=E(X-\alpha Y)^2=0$, which essentially means $X= \alpha Y$ with probability one. To prove the Cauchy-Schwarz inequality, choose $\alpha=\frac{EXY}{EY^2}$. We obtain
| $0$ | $\leq E[X^2]-2\alpha E[XY]+ \alpha^2 E[Y^2]$ |
| $=E[X^2]-2\frac{EXY}{EY^2} E[XY]+ \frac{(EXY)^2}{(EY^2)^2} E[Y^2]$ | |
| $=E[X^2]-\frac{(E[XY])^2}{EY^2}.$ |
Thus, we conclude \begin{align}%\label{} (E[XY])^2 \leq E[X^2] E[Y^2], \end{align} which implies \begin{align}%\label{} |EXY| \leq \sqrt{E[X^2] E[Y^2]}. \end{align} Also, if $|EXY| = \sqrt{E[X^2] E[Y^2]}$, we conclude that $f(\frac{EXY}{EY^2})=0$, which implies $X= \frac{EXY}{EY^2} Y$ with probability one.
Example
Using the Cauchy-Schwarz inequality, show that for any two random variables $X$ and $Y$ \begin{align}%\label{} |\rho(X,Y)| \leq 1. \end{align} Also, $|\rho(X,Y)| = 1$ if and only if $Y=aX+b$ for some constants $a,b \in \mathbb{R}$.
- Solution
- Let \begin{align}%\label{} U=\frac{X-EX}{\sigma_X}, &\qquad V=\frac{Y-EY}{\sigma_Y}. \end{align} Then $EU=EV=0$, and $Var(U)=Var(V)=1$. Using the Cauchy-Schwarz inequality for $U$ and $V$, we obtain \begin{align}%\label{} |EUV| \leq \sqrt{E[U^2] E[V^2]}=1. \end{align} But note that $EUV=\rho(X,Y)$, thus we conclude \begin{align}%\label{} |\rho(X,Y)| \leq 1, \end{align} where equality holds if and only if $V=\alpha U$ for some constant $\alpha \in \mathbb{R}$. That is \begin{align}%\label{} \frac{Y-EY}{\sigma_Y}=\alpha \frac{X-EX}{\sigma_X}, \end{align} which implies \begin{align}%\label{} Y=\frac{\alpha \sigma_Y}{\sigma_X} X+ (EY-\frac{\alpha \sigma_Y}{\sigma_X} EX). \end{align} In the Solved Problems section, we provide a generalization of the Cauchy-Schwrarz inequality, called Hölder's inequality.
