9.1.2 Maximum A Posteriori (MAP) Estimation
The posterior distribution, $f_{X|Y}(x|y)$ (or $P_{X|Y}(x|y)$), contains all the knowledge about the unknown quantity $X$. Therefore, we can use the posterior distribution to find point or interval estimates of $X$. One way to obtain a point estimate is to choose the value of $x$ that maximizes the posterior PDF (or PMF). This is called the maximum a posteriori (MAP) estimation.
The MAP estimate of the random variable $X$, given that we have observed $Y=y$, is given by the value of $x$ that maximizes \begin{align} &f_{X|Y}(x|y) \textrm{ if $X$ is a continuous random variable, }\\ &P_{X|Y}(x|y) \textrm{ if $X$ is a discrete random variable. } \end{align} The MAP estimate is shown by $\hat{x}_{MAP}$.
To find the MAP estimate of $X$ given that we have observed $Y=y$, we find the value of $x$ that maximizes
\begin{align}
f_{Y|X}(y|x)f_{X}(x).
\end{align}
If either $X$ or $Y$ is discrete, we replace its PDF in the above expression by the corresponding PMF.
Example
Let $X$ be a continuous random variable with the following PDF: \begin{align} \nonumber f_X(x) = \left\{ \begin{array}{l l} 2x & \quad \textrm{if }0 \leq x \leq 1 \\ & \quad \\ 0 & \quad \text{otherwise} \end{array} \right. \end{align} Also, suppose that \begin{align} Y \; | \; X=x \quad \sim \quad Geometric(x). \end{align} Find the MAP estimate of $X$ given $Y=3$.
- Solution
- We know that $ Y \; | \; X=x \quad \sim \quad Geometric(x)$, so \begin{align} P_{Y|X}(y|x)=x (1-x)^{y-1}, \quad \textrm{ for }y=1,2,\cdots. \end{align} Therefore, \begin{align} P_{Y|X}(3|x)=x (1-x)^2. \end{align} We need to find the value of $x \in [0,1]$ that maximizes \begin{align} P_{Y|X}(y|x)f_{X}(x)&=x (1-x)^2 \cdot 2x\\ &=2x^2(1-x)^2. \end{align} We can find the maximizing value by differentiation. We obtain \begin{align} \frac{\textrm{d}}{\textrm{d}x} \bigg[x^2(1-x)^2 \bigg]=2x(1-x)^2-2(1-x)x^2=0. \end{align} Solving for $x$ (and checking for maximization criteria), we obtain the MAP estimate as \begin{align} \hat{x}_{MAP}=\frac{1}{2}. \end{align}
