9.1.2 Maximum A Posteriori (MAP) Estimation

The posterior distribution, $f_{X|Y}(x|y)$ (or $P_{X|Y}(x|y)$), contains all the knowledge about the unknown quantity $X$. Therefore, we can use the posterior distribution to find point or interval estimates of $X$. One way to obtain a point estimate is to choose the value of $x$ that maximizes the posterior PDF (or PMF). This is called the maximum a posteriori (MAP) estimation.

MAP
Figure 9.3 - The maximum a posteriori (MAP) estimate of $X$ given $Y=y$ is the value of $x$ that maximizes the posterior PDF or PMF. The MAP estimate of $X$ is usually shown by $\hat{x}_{MAP}$.
Maximum A Posteriori (MAP) Estimation

The MAP estimate of the random variable $X$, given that we have observed $Y=y$, is given by the value of $x$ that maximizes \begin{align} &f_{X|Y}(x|y) \textrm{ if $X$ is a continuous random variable, }\\ &P_{X|Y}(x|y) \textrm{ if $X$ is a discrete random variable. } \end{align} The MAP estimate is shown by $\hat{x}_{MAP}$.
To find the MAP estimate, we need to find the value of $x$ that maximizes \begin{align} f_{X|Y}(x|y)=\frac{f_{Y|X}(y|x)f_{X}(x)}{f_{Y}(y)}. \end{align} Note that $f_{Y}(y)$ does not depend on the value of $x$. Therefore, we can equivalently find the value of $x$ that maximizes \begin{align} f_{Y|X}(y|x)f_{X}(x). \end{align} This can simplify finding the MAP estimate significantly, because finding $f_{Y}(y)$ might be complicated. More specifically, finding $f_{Y}(y)$ usually is done using the law of total probability, which involves integration or summation, such as the one in Example 9.3.
To find the MAP estimate of $X$ given that we have observed $Y=y$, we find the value of $x$ that maximizes \begin{align} f_{Y|X}(y|x)f_{X}(x). \end{align} If either $X$ or $Y$ is discrete, we replace its PDF in the above expression by the corresponding PMF.


Example
Let $X$ be a continuous random variable with the following PDF: \begin{align} \nonumber f_X(x) = \left\{ \begin{array}{l l} 2x & \quad \textrm{if }0 \leq x \leq 1 \\ & \quad \\ 0 & \quad \text{otherwise} \end{array} \right. \end{align} Also, suppose that \begin{align} Y \; | \; X=x \quad \sim \quad Geometric(x). \end{align} Find the MAP estimate of $X$ given $Y=3$.
  • Solution
    • We know that $ Y \; | \; X=x \quad \sim \quad Geometric(x)$, so \begin{align} P_{Y|X}(y|x)=x (1-x)^{y-1}, \quad \textrm{ for }y=1,2,\cdots. \end{align} Therefore, \begin{align} P_{Y|X}(3|x)=x (1-x)^2. \end{align} We need to find the value of $x \in [0,1]$ that maximizes \begin{align} P_{Y|X}(y|x)f_{X}(x)&=x (1-x)^2 \cdot 2x\\ &=2x^2(1-x)^2. \end{align} We can find the maximizing value by differentiation. We obtain \begin{align} \frac{\textrm{d}}{\textrm{d}x} \bigg[x^2(1-x)^2 \bigg]=2x(1-x)^2-2(1-x)x^2=0. \end{align} Solving for $x$ (and checking for maximization criteria), we obtain the MAP estimate as \begin{align} \hat{x}_{MAP}=\frac{1}{2}. \end{align}






The print version of the book is available through Amazon here.

Book Cover