Hostname: page-component-848d4c4894-nr4z6 Total loading time: 0 Render date: 2024-06-07T15:06:29.725Z Has data issue: false hasContentIssue false
  • English
  • Français

On the Jeffreys-Lindley Paradox

Published online by Cambridge University Press:  01 January 2022

Christian P. Robert
Get access Rights & Permissions [Opens in a new window]

Abstract

This article discusses the dual interpretation of the Jeffreys-Lindley paradox associated with Bayesian posterior probabilities and Bayes factors, both as a differentiation between frequentist and Bayesian statistics and as a pointer to the difficulty of using improper priors while testing. I stress the considerable impact of this paradox on the foundations of both classical and Bayesian statistics. While assessing existing resolutions of the paradox, I focus on a critical viewpoint of the paradox discussed by Spanos in Philosophy of Science.

Type
Research Article
Information
Philosophy of Science , Volume 81 , Issue 2 , April 2014 , pp. 216 - 232
Copyright
Copyright © The Philosophy of Science Association

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

†

Research was partly supported by the Agence Nationale de la Recherche (ANR, 212, rue de Bercy, 75012 Paris) through the 2012–15 grant ANR-11-BS01-0010 “Calibration” and by an Institut Universitaire de France chair. The author is most sincerely grateful to the editorial team of Philosophy of Science for its comments and support toward improving the exposition in the article.

References

Berger, J. 1985. Statistical Decision Theory and Bayesian Analysis. New York: Springer.CrossRefGoogle Scholar
Berger, J. 2003. “Could Fisher, Jeffreys and Neyman Have Agreed on Testing?” Statistical Science 18:1–32.CrossRefGoogle Scholar
Berger, J., Pericchi, L., and Varshavsky, J.. 1998. “Bayes Factors and Marginal Distributions in Invariant Situations.” Sankhya A 60:307–21.Google Scholar
Berger, J., and Sellke, T.. 1987. “Testing a Point-Null Hypothesis: The Irreconcilability of Significance Levels and Evidence.” Journal of the American Statistical Association 82:112–22.Google Scholar
Berger, J., and Wolpert, R.. 1988. The Likelihood Principle. 2nd ed. IMS Lecture Notes—Monograph Series 9. Hayward, CA: International Management Systems.Google Scholar
Birnbaum, A. 1962. “On the Foundations of Statistical Inference.” Journal of the American Statistical Association 57:269–306.CrossRefGoogle Scholar
Camus, A. 1942. Le mythe de sisyphe. Paris: Gallimard.Google Scholar
Celeux, G., Anbari, M. E., Marin, J., and Robert, C.. 2012. “Regularization in Regression: Comparing Bayesian and Frequentist Methods in a Poorly Informative Situation.” Bayesian Analysis 7:477–502.CrossRefGoogle Scholar
Dawid, A., Stone, N., and Zidek, J.. 1973. “Marginalization Paradoxes in Bayesian and Structural Inference.” Journal of the Royal Statistical Society B 35:189–233.Google Scholar
DeGroot, M. 1973. “Doing What Comes Naturally: Interpreting a Tail Area as a Posterior Probability or as a Likelihood Ratio.” Journal of the American Statistical Association 68:966–69.CrossRefGoogle Scholar
DeGroot, M. 1982. “Discussion of Shafer’s ‘Lindley’s Paradox.’” Journal of the American Statistical Association 378:337–39.Google Scholar
Gelman, A., Carlin, J., Stern, H., Dunson, D., Vehtari, A., and Rubin, D.. 2013. Bayesian Data Analysis. Rev. ed. New York: Chapman & Hall.CrossRefGoogle Scholar
Hyvärinen, A. 2005. “Estimation of Non-normalized Statistical Models by Score Matching.” Journal of Machine Learning Research 6:695–709.Google Scholar
Jeffreys, H. 1939. Theory of Probability. 1st ed. Oxford: Clarendon.Google Scholar
Johnson, V. 2013. “Uniformly Most Powerful Bayesian Tests.” Journal of the Royal Statistical Society B 41:1716–41.Google ScholarPubMed
Johnson, V., and Rossell, D.. 2010. “On the Use of Non-local Prior Densities in Bayesian Hypothesis Tests.” Journal of the Royal Statistical Society B 72:143–70.Google Scholar
Kass, R., and Wasserman, L.. 1996. “Formal Rules of Selecting Prior Distributions: A Review and Annotated Bibliography.” Journal of the American Statistical Association 91:1343–70.CrossRefGoogle Scholar
Lad, F. 2003. “Appendix: The Jeffreys-Lindley Paradox and Its Relevance to Statistical Testing.” Technical report, Conference on Science and Democracy, Palazzo Serra di Cassano, Napoli.Google Scholar
Lehmann, E., and Casella, G.. 1998. Theory of Point Estimation. Rev. ed. New York: Springer.Google Scholar
Liang, F., Paulo, R., Molina, G., Clyde, M., and Berger, J.. 2008. “Mixtures of g Priors for Bayesian Variable Selection.” Journal of the American Statistical Association 103:410–23.CrossRefGoogle Scholar
Lindley, D. 1957. “A Statistical Paradox.” Biometrika 44:187–92.CrossRefGoogle Scholar
Marin, J., and Robert, C.. 2007. Bayesian Core. New York: Springer.Google Scholar
Mayo, D., and Spanos, A.. 2004. “Methodology in Practice: Statistical Misspecification Testing.” Philosophy of Science 71:1007–25.CrossRefGoogle Scholar
Parry, M., Dawid, A., and Lauritzen, S.. 2012. “Proper Local Scoring Rules.” Annals of Statistics 41:561–92.Google Scholar
Robert, C. 1993. “A Note on Jeffreys-Lindley Paradox.” Statistica Sinica 3:601–8.Google Scholar
Robert, C. 2001. The Bayesian Choice. 2nd ed. New York: Springer.Google Scholar
Robert, C., Chopin, N., and Rousseau, J.. 2009. “Theory of Probability Revisited.” Statistical Science 24 (2): 141–254.Google Scholar
Robert, C., and Marin, J.-M.. 2010. “On Resolving the Savage-Dickey Paradox.” Electronic Journal of Statistics 4:643–54.Google Scholar
Shafer, G. 1982. “On Lindley’s Paradox.” Journal of the American Statistical Association 7:325–51.Google Scholar
Smith, A., and Spiegelhalter, D.. 1982. “Bayes Factors for Linear and Log-Linear Models with Vague Prior Information.” Journal of the Royal Statistical Society B 44:377–87.Google Scholar
Spanos, A. 2012. “Why the Decision Theoretic Perspective Misrepresents Frequentist Inference: ‘Nuts and Bolts’ vs. Learning from Data.” Technical report, arXiv.org. arxiv:1211.0638.Google Scholar
Spanos, A. 2013. “Who Should Be Afraid of the Jeffreys-Lindley Paradox?” Philosophy of Science 80:73–93.CrossRefGoogle Scholar
Sprenger, J. 2013. “Testing a Precise Null Hypothesis: The Case of Lindley’s Paradox.” Philosophy of Science 80:733–44.CrossRefGoogle Scholar
Stone, M. 1997. “Discussion of Papers by Dempster and Aitkin.” Statistics and Computing 7:263–64.CrossRefGoogle Scholar
Zellner, A. 1986. “On Assessing Prior Distributions and Bayesian Regression Analysis with g-Prior Distribution Regression Using Bayesian Variable Selection.” In Bayesian Inference and Decision Techniques: Essays in Honor of Bruno de Finetti, 233–43. Amsterdam: North-Holland.Google Scholar