7.2.6 Convergence in Mean
One way of interpreting the convergence of a sequence $X_n$ to $X$ is to say that the ''distance'' between $X$ and $X_n$ is getting smaller and smaller. For example, if we define the distance between $X_n$ and $X$ as $P\big(|X_n-X| \geq \epsilon \big)$, we have convergence in probability. One way to define the distance between $X_n$ and $X$ is
\begin{align}%\label{eq:union-bound} E\left(|X_n-X|^{\large r}\right), \end{align}where $r \geq 1$ is a fixed number. This refers to convergence in mean. (Note: for convergence in mean, it is usually required that $E|X^{\large r}_n|<\infty$.) The most common choice is $r=2$, in which case it is called the mean-square convergence. (Note: Some authors refer to the case $r=1$ as convergence in mean.)
Convergence in Mean
Let $r \geq 1$ be a fixed number. A sequence of random variables $X_1$, $X_2$, $X_3$, $\cdots$ converges in the $r$th mean or in the $L^{\large r}$ norm to a random variable $X$, shown by $ X_n \ \xrightarrow{L^{\large r}}\ X$, if \begin{align}%\label{eq:union-bound} \lim_{n \rightarrow \infty} E\left(|X_n-X|^{\large r}\right)=0. \end{align} If $r=2$, it is called the mean-square convergence, and it is shown by $ X_n \ \xrightarrow{m.s.}\ X$.Example
Let $X_n \sim Uniform\left(0, \frac{1}{n}\right)$. Show that $ X_n \ \xrightarrow{L^{\large r}}\ 0$, for any $r \geq 1$.
- Solution
- The PDF of $X_n$ is given by \begin{equation} \nonumber f_{X_n}(x) = \left\{ \begin{array}{l l} n & \quad 0 \leq x \leq \frac{1}{n} \\ & \quad \\ 0 & \quad \text{otherwise} \end{array} \right. \end{equation} We have \begin{align}%\label{eq:union-bound} E\left(|X_n-0|^{\large r}\right)&=\int_{0}^{\frac{1}{n}} x^{\large r} n \hspace{10pt} dx\\ &= \frac{1}{(r+1) n^{\large r}} \rightarrow 0, \qquad \textrm{ for all }r\geq 1. \end{align}
Theorem
Let $1 \leq r \leq s$. If $ X_n \ \xrightarrow{L^{\large s}}\ X$, then $ X_n \ \xrightarrow{L^{\large r}}\ X$.
- Proof
- We can use Hölder's inequality, which was proved in Section 6.2.6. Hölder's Inequality states that \begin{align}%\label{} E |XY| \leq \big(E|X|^p\big)^{{\large \frac{1}{p}}} \big(E|Y|^q\big)^{{\large \frac{1}{q}}}, \end{align} where $1 < p$ , $q < \infty$ and ${\large \frac{1}{p}}+{\large \frac{1}{q}}= 1$. In Hölder's inequality, choose \begin{align}%\label{} X&=|X_n-X|^{\large r}, \\ Y&=1,\\ p&=\frac{s}{r}>1. \end{align} We obtain \begin{align}%\label{} E |X_n-X|^{\large r} \leq \big(E|X_n-X|^{\large s}\big)^{{\large \frac{1}{p}}}. \end{align} Now, by assumption $ X_n \ \xrightarrow{L^{\large s}}\ X$, which means \begin{align}%\label{eq:union-bound} \lim_{n \rightarrow \infty} E\left(|X_n-X|^{\large s}\right)=0. \end{align} We conclude \begin{align}%\label{eq:union-bound} \lim_{n \rightarrow \infty} E\left(|X_n-X|^{\large r}\right) & \leq \lim_{n \rightarrow \infty} \big(E|X_n-X|^{\large s}\big)^{\frac{1}{p}}\\ &= 0. \end{align} Therefore, $ X_n \ \xrightarrow{L^{\large r}}\ X$.
As we mentioned before, convergence in mean is stronger than convergence in probability. We can prove this using Markov's inequality.
Theorem
If $ X_n \ \xrightarrow{L^{\large r}}\ X$ for some $r\geq 1$, then $ X_n \ \xrightarrow{p}\ X$.
- Proof
- For any $\epsilon>0$, we have \begin{align}%\label{} P\big(|X_n-X| \geq \epsilon \big)&= P\big(|X_n-X|^{\large r} \geq \epsilon^{\large r} \big) & \textrm{ (since $r \geq 1$)}\\ & \leq \frac{E|X_n-X|^{\large r}}{\epsilon^{\large r}} &\textrm{ (by Markov's inequality)}. \end{align} Since by assumption $\lim \limits_{n \rightarrow \infty} E\left(|X_n-X|^{\large r}\right)=0$, we conclude \begin{align}%\label{eq:union-bound} \lim_{n \rightarrow \infty} P\big(|X_n-X| \geq \epsilon \big)=0, \qquad \textrm{ for all }\epsilon>0. \end{align}
The converse of Theorem 7.4 is not true in general. That is, there are sequences that converge in probability but not in mean. Let us look at an example.
Example
Consider a sequence $\{X_n, n=1,2,3, \cdots \}$ such that
\begin{equation} \nonumber X_n = \left\{ \begin{array}{l l} n^2 & \qquad \textrm{with probability } \frac{1}{n} \\ & \qquad \\ 0 & \qquad \textrm{with probability } 1-\frac{1}{n} \end{array} \right. \end{equation} Show that- $X_n \ \xrightarrow{p}\ 0$.
- $X_n$ does not converge in the $r$th mean for any $r \geq 1$.
- Solution
-
- To show $X_n \ \xrightarrow{p}\ 0$, we can write, for any $\epsilon>0$ \begin{align}%\label{eq:union-bound} \lim_{n \rightarrow \infty} P\big(|X_n| \geq \epsilon \big)&=\lim_{n \rightarrow \infty} P(X_n=n^2)\\ &= \lim_{n \rightarrow \infty} \frac{1}{n}\\ &=0. \end{align} We conclude that $X_n \ \xrightarrow{p}\ 0$.
- For any $r \geq 1$, we can write \begin{align}%\label{eq:union-bound} \lim_{n \rightarrow \infty} E\left(|X_n|^{\large r}\right)&=\lim_{n \rightarrow \infty} \left( n^{2r} \cdot \frac{1}{n} +0 \cdot \left(1-\frac{1}{n}\right) \right)\\ &=\lim_{n \rightarrow \infty} n^{2r-1}\\ \ &=\infty \qquad (\textrm{since $r \geq 1$}). \end{align} Therefore, $X_n$ does not converge in the $r$th mean for any $r \geq 1$. In particular, it is interesting to note that, although $X_n \ \xrightarrow{p}\ 0$, the expected value of $X_n$ does not converge to $0$.
-
