---

The above discussion suggests the sample mean, $\overline{X}$, is often a reasonable point estimator for the mean. Now, suppose that we would like to estimate the variance of a distribution $\sigma^2$. Assuming $0<\sigma^2<\infty$, by definition \begin{align}%\label{} \sigma^2=E[(X-\mu)^2]. \end{align} Thus, the variance itself is the mean of the random variable $Y=(X-\mu)^2$. This suggests the following estimator for the variance \begin{align}%\label{} \hat{\sigma}^2=\frac{1}{n} \sum_{k=1}^n (X_k-\mu)^2. \end{align} By linearity of expectation, $\hat{\sigma}^2$ is an unbiased estimator of $\sigma^2$. Also, by the weak law of large numbers, $\hat{\sigma}^2$ is also a consistent estimator of $\sigma^2$. However, in practice we often do not know the value of $\mu$. Thus, we may replace $\mu$ by our estimate of the $\mu$, the sample mean, to obtain the following estimator for $\sigma^2$: \begin{align}%\label{} {\overline{S}}^2=\frac{1}{n} \sum_{k=1}^n (X_k-\overline{X})^2. \end{align} Using a little algebra, you can show that \begin{align}%\label{} {\overline{S}}^2=\frac{1}{n} \left(\sum_{k=1}^n X^2_k-n\overline{X}^2\right). \end{align}

---

Example

Let $X_1$, $X_2$, $X_3$, $...$, $X_n$ be a random sample with mean $EX_i=\mu$, and variance $\mathrm{Var}(X_i)=\sigma^2$. Suppose that we use

\begin{align}%\label{} {\overline{S}}^2=\frac{1}{n} \sum_{k=1}^n (X_k-\overline{X})^2=\frac{1}{n} \left(\sum_{k=1}^n X^2_k-n\overline{X}^2\right) \end{align} to estimate $\sigma^2$. Find the bias of this estimator \begin{align}%\label{} B({\overline{S}}^2)=E[{\overline{S}}^2]-\sigma^2. \end{align}

---

We conclude that $\overline{S}^2$ is a biased estimator of the variance. Nevertheless, note that if $n$ is relatively large, the bias is very small. Since $E[{\overline{S}}^2]=\frac{n-1}{n}\sigma^2$, we can obtain an unbiased estimator of $\sigma^2$ by multiplying $\overline{S}^2$ by $\frac{n}{n-1}$. Thus, we define \begin{align}%\label{} {S}^2=\frac{1}{n-1} \sum_{k=1}^n (X_k-\overline{X})^2=\frac{1}{n-1} \left(\sum_{k=1}^n X^2_k-n\overline{X}^2\right). \end{align} By the above discussion, $S^2$ is an unbiased estimator of the variance. We call it the sample variance. We should note that if $n$ is large, the difference between $S^2$ and $\overline{S}^2$ is very small. We also define the sample standard deviation as \begin{align}%\label{} S&= \sqrt{S^2}. \end{align} Although the sample standard deviation is usually used as an estimator for the standard deviation, it is a biased estimator. To see this, note that $S$ is random, so $\mathrm{Var}(S)>0$. Thus, \begin{align}%\label{} 0<\mathrm{Var}(S)&= ES^2-(ES)^2\\ &=\sigma^2-(ES)^2. \end{align} Therefore, $ES<\sigma$, which means that $S$ is a biased estimator of $\sigma$.

---

You can use the mean command in MATLAB to compute the sample mean for a given sample. More specifically, for a given vector $x=$$[$$x_1$, $x_2$, $\cdots$, $x_n$ $]$, mean(x) returns the sample average \begin{align}%\label{} \frac{x_1+x_2+\cdots+x_n}{n}. \end{align} Also, the functions var and std can be used to compute the sample variance and the sample standard deviation respectively.

---

Example

Let $T$ be the time that is needed for a specific task in a factory to be completed. In order to estimate the mean and variance of $T$, we observe a random sample $T_1$,$T_2$,$\cdots$,$T_6$. Thus, $T_i$'s are i.i.d. and have the same distribution as $T$. We obtain the following values (in minutes):

\begin{equation}%\label{} 18, 21, 17, 16, 24, 20. \end{equation} Find the values of the sample mean, the sample variance, and the sample standard deviation for the observed sample.

---

You can use the following MATLAB code to compute the above values:

---