Asymptotic Properties of MLEs

8.2.4 Asymptotic Properties of MLEs

We end this section by mentioning that MLEs have some nice asymptotic properties. By asymptotic properties we mean properties that are true when the sample size becomes large. Here, we state these properties without proofs.

Let $X_1$, $X_2$, $X_3$, $...$, $X_n$ be a random sample from a distribution with a parameter $\theta$. Let $\hat{\Theta}_{ML}$ denote the maximum likelihood estimator (MLE) of $\theta$. Then, under some mild regularity conditions,

$\hat{\Theta}_{ML}$ is asymptotically consistent, i.e., \begin{align} \lim_{n \rightarrow \infty} P(|\hat{\Theta}_{ML}-\theta|>\epsilon)=0. \end{align}
$\hat{\Theta}_{ML}$ is asymptotically unbiased, i.e., \begin{align} \lim_{n \rightarrow \infty} E[\hat{\Theta}_{ML}]=\theta. \end{align}
As $n$ becomes large, $\hat{\Theta}_{ML}$ is approximately a normal random variable. More precisely, the random variable \begin{align} \frac{\hat{\Theta}_{ML}-\theta}{\sqrt{\mathrm{Var}(\hat{\Theta}_{ML})}} \end{align} converges in distribution to $N(0,1)$.

← previous

The print version of the book is available on Amazon.

Practical uncertainty: Useful Ideas in Decision-Making, Risk, Randomness, & AI