Bayes Formula

Let $\{ B_1,...B_n\}$ be a partition of our sample space $S$ . Let us say for our partition we have the probability of each element of the partition $B_i$ , as well as the conditional probabilities of $P(A | B_i)$ for some $A \subset S$ with $P(A) > 0$ . Can we calculate $P(B_i | A)$ ? Put another way if we have the probabilities of an event $A$ given each element of the partition, can we calculate the probability of an element of the partition occurring given that $A$ has occurred?

We remember that $P(A | B) = \frac{P(A \cap B)}{P(B)}$ . So we look at what we have to find,

$P(B_i | A) = \frac{P(B_i \cap A}{ P(A)}$

we can rewrite the right-hand side and use the multiplication law to get

$\frac{P(B_i \cap A}{ P(A)} = \frac{P(A | B_i)P(B_i)}{P(A)}$

now, we don’t have $P(A)$ , but we do have $P(A | B_i)$ for each $i$ and we have $P(B_i)$ for each $i$ , so using the law of total probability, we arrive at

$P(B_i | A) = \frac{P(A | B_i)P(B_i)}{\sum_{k = 1}^{n}P(A | B_k)P(B_k)}$

This result is known as Bayes’ Theorem and is stated below,

Theorem: [Bayes’ Theorem] Let $\{ B_1,...B_n\}$ be a partition of our sample space $S$ . If for $i = 1,2,...,n; P(B_i) > 0$ , then for any event $A \subset S$ with $P(A) > 0$ ,

$\begin{align*} P(B_k | A) &= \frac{P(A | B_k)P(B_k)}{P(A|B_1)P(B_1) + ... + P(A | B_n)P(B_n)} \end{align*}$