Beta, Dirichlet, and von Mises-Fisher

Chapter 13: Probability and Random Variables. In the chapter mind map this icon labels Beta,Dirichlet,vMF. The discussion below is excerpted and lightly edited from § Beta Distribution in Mathematics for AI and Machine Learning.

A random variable $X$ follows a Beta distribution with shape parameters $\alpha > 0$ and $\beta > 0$, denoted as $X \sim \mathrm{Beta}(\alpha, \beta)$, if its probability density function is:

where $\Gamma(\cdot)$ is the gamma function. The normalization constant $\frac{\Gamma(\alpha + \beta)}{\Gamma(\alpha)\Gamma(\beta)}$ ensures that the PDF integrates to 1 over $[0, 1]$.

Because it lives on $[0, 1]$, the Beta distribution is the natural model for an unknown probability or proportion, and it is the conjugate prior of the Bernoulli distribution. With prior $p \sim \mathrm{Beta}(\alpha, \beta)$ and $s$ successes in $n$ independent Bernoulli trials, the likelihood is proportional to $p^{s}(1-p)^{n-s}$, so the posterior is again a Beta distribution:

Updating is just counting: $\alpha$ and $\beta$ act as pseudo-counts of successes and failures seen before the data.

Thompson sampling uses this posterior to choose among $K$ options with unknown success rates $p_0, \ldots, p_{K-1}$ (the arms of a multi-armed bandit). Keep a posterior $\mathrm{Beta}(\alpha_k, \beta_k)$ for each arm; in every round draw one sample $\tilde p_k$ from each posterior, play the arm with the largest $\tilde p_k$, and add the observed success or failure to that arm's counts. An arm with few trials has a wide posterior and is sometimes drawn high, so it still gets tried; an arm measured many times and found poor is almost never chosen. Exploration therefore shrinks automatically as the posteriors narrow, without a tuning parameter.