Difference between P-value and Bayes Factor

Before we directly dive into the difference between P-value and Bayes factor. It is essential to know the difference between the Frequentist approach statistics and Bayesian approach statistics.

Frequentist vs Bayesian

Statistics is broadly discussed in two different approaches. They are the Frequentist approach and the Bayesian approach. Although the Bayesian approach is more realistic in providing inferences and comparing the hypotheses for the given data we analyse, these two approaches have their unique characteristics. The primary difference between the Frequentist and Bayesian approaches lies in the way they used probability. The frequentist approach provides statistical inference based on the classical p-value, the significance level, the power and the confidence interval. They used probability only to model certain specific processes described by the sampling procedure (or described by the sample). Thus, allowing the data to carry some amount of uncertainty. While adopting a Frequentist approach, there is always a worry in mind that the "correct model" is specified or a null model is rejected. On the contrary, in the Bayesian approach, we use the probability in a more diverse way to model the sampling processes as well as the other related uncertainties. We focus on whether or not the parameters and model are sensible for that particular dataset by providing credible intervals instead of confidence intervals. With this approach, we don't need to worry about setting up of null hypothesis; instead, we are given the power to make a direct probability statement about the parameter of interest. In the Bayesian approach, we make direct probability statements about the parameters using the observed sample. In contrast, the p-value is calculated on the assumption of drawing a hypothetical infinite number of samples (i.e. sampling distribution) that we never really observe. Finally, the Bayesian approach is also known to deal better with small samples. The Bayesian approach also incorporates prior information (about previous findings and theory) into the estimation, which sometimes can prove to be highly useful.

P-value Vs Bayes factor

P-value:

Fisher first developed it in a well-planned limited agricultural experimental setup. It is an intuitively appealing measure against the null-hypothesis, but many-a-times it is not perceived correctly. Some of the critical mistakes we commit while using the p-value are:

1. Interpreting it as the probability that H₀ is (not) true while it measures only the extremeness of the observed result under H₀.
2. It doesn't express the probability that the observed result occurred under H₀, but is rather the probability of observing a more extreme result under H₀. This implies that it is based not only on the observed result but also on fictive (never observed) data. For example, if we want to test the significance of the regression estimates (beta coefficients), we look for the p-value from the t-distribution, which is hypothetical. 
3. It is not an absolute measure. A small value does not necessarily mean there is a significant difference between two or more characteristics of interests (variables).
4. It does not take into account the size of the study. (Royall R. 1997)

It would be rather more informative to use a 95% Confidence Interval in place of the p-value as it is considered to provide more insights relevant to the obtained result.

Bayes factor:

It is the outcome of one of the major contributions of Jeffreys to Bayesian statistics. It measures the change from prior to posterior odds favouring the null hypothesis. And it is the Bayesian equivalent of the likelihood ratio test.

If y represents the observed data and H₀ represents the null hypothesis to be tested. Then according to Bayes theorem,

Here H₁ is the alternative hypothesis.
Similarly,

Thus,

The term, 

is known as the Bayes factor (BF). Its value ranged from 0 to infinity. According to Jeffreys, classification of BF favouring H₀ against H₁ is given as:

1. "decisive" if BF > 100
2. "very strong" if 32 < BF ≤ 100
3. "strong" if 10 < BF ≤ 32
4. "substantial" if 3.2 < BF ≤ 10
5. "not worth" if 1 < BF ≤ 3.2

Note: This blog highlights the key conceptual difference between the p-value and the Bayes factor. For more information please go through the material provided in the reading list.

Reading List:

1. Assaf, A. G., & Tsionas, M. (2018). Bayes factors vs p-values. Tourism Management, 67, 17-31.
2. Held, L., & Ott, M. (2018). On p-values and Bayes factors.
3. Lesaffre, E., & Lawson, A. B. (2012). Bayesian biostatistics. John Wiley & Sons.
4. Royall, R. (1997). Statistical evidence: a likelihood paradigm. Routledge.