9.1.3 Comparison to ML Estimation
Example
Suppose that the signal $X \sim N(0,\sigma^2_X)$ is transmitted over a communication channel. Assume that the received signal is given by \begin{align} Y=X+W, \end{align} where $W \sim N(0,\sigma_W^2)$ is independent of $X$.
- Find the ML estimate of $X$, given $Y=y$ is observed.
- Find the MAP estimate of $X$, given $Y=y$ is observed.
- Solution
-
Here, we have
\begin{equation}
\nonumber f_X(x) = \frac{1}{\sqrt{2 \pi}\sigma_X} e^{-\frac{x^2}{2\sigma^2_X}}.
\end{equation}
We also have, $Y | X=x \quad \sim \quad N(x, \sigma_W^2)$, so
\begin{equation}
\nonumber f_{Y|X}(y|x) = \frac{1}{\sqrt{2 \pi}\sigma_W} e^{-\frac{(y-x)^2}{2\sigma_W^2}}.
\end{equation}
- The ML estimate of $X$, given $Y=y$, is the value of $x$ that maximizes \begin{equation} \nonumber f_{Y|X}(y|x) = \frac{1}{\sqrt{2 \pi}\sigma_W} e^{-\frac{(y-x)^2}{2\sigma_W^2}}. \end{equation} To maximize the above function, we should minimize $(y-x)^2$. Therefore, we conclude \begin{align} \hat{x}_{ML}=y. \end{align}
- The MAP estimate of $X$, given $Y=y$, is the value of $x$ that maximizes \begin{equation} f_{Y|X}(y|x)f_X(x) = c \exp \left\{-\left[\frac{(y-x)^2}{2\sigma_W^2}+\frac{x^2}{2\sigma^2_X}\right]\right\}, \end{equation} where $c$ is a constant. To maximize the above function, we should minimize \begin{equation} \frac{(y-x)^2}{2\sigma_W^2}+\frac{x^2}{2\sigma^2_X}. \end{equation} By differentiation, we obtain the MAP estimate of $x$ as \begin{align} \hat{x}_{MAP}=\frac{\sigma^2_X}{\sigma^2_X+\sigma^2_W} y. \end{align}
-
Here, we have
\begin{equation}
\nonumber f_X(x) = \frac{1}{\sqrt{2 \pi}\sigma_X} e^{-\frac{x^2}{2\sigma^2_X}}.
\end{equation}
We also have, $Y | X=x \quad \sim \quad N(x, \sigma_W^2)$, so
\begin{equation}
\nonumber f_{Y|X}(y|x) = \frac{1}{\sqrt{2 \pi}\sigma_W} e^{-\frac{(y-x)^2}{2\sigma_W^2}}.
\end{equation}
