Bayes' Theorem

In probability theory and statistics, Bayes' Theorem describes the probability of an event considering the prior information. It is also known as Bayes' Rule or Bayes' Law. It is particularly used in Bayesian Inference, a particular approach to statistical inference. The theorem was coined by Reverend Thomas Bayes, the first person to use conditional probability to calculate the limits of an unknown parameter of interest.

Theorem:

Suppose S is a sample space. And let B₁, B₂, …, B_n be n disjoint non-empty subsets (or events) of S such that S = B₁ ꓴ B₂ ꓴ … ꓴ B_n such that P(B_i) lying between 0 and 1 and i = 1, 2, …, n.

Fig. 1: events description

Suppose A is an event, as shown in the figure above (Fig. 1), then

But,

Therefore,

Interpretation:

The interpretation of Bayes' theorem depends solely on the approach of the interpretation of probabilities.

1. In the Bayesian approach, probability measures the degree of belief. For event A and evidence B, P(A) is the prior i.e. initial degree of belief in A. And P(A|B) is the posterior, i.e. the degree of belief accounting for B. Then P(B|A)/P(B) is the support B provides for A.
2. In the Frequentist approach, probability measures the proportion of the event of interest. P(A) is the proportion of event A, and P(B) that of event B. P(B|A) is the proportion of event B out of event A, and P(A|B) is the proportion of those with A out of those with B.