Hostname: page-component-77c89778f8-7drxs Total loading time: 0 Render date: 2024-07-17T21:48:48.580Z Has data issue: false hasContentIssue false
  • English
  • Français

Publication Bias Reconsidered

Published online by Cambridge University Press:  04 January 2017

Lee Sigelman
Lee Sigelman*
Affiliation:
The George Washington University
Get access Rights & Permissions [Opens in a new window]

Abstract

In political science and many other disciplines, statistically significant results—rejections of the null hypothesis—are achieved more frequently in published than in unpublished studies. Such “publication bias” is generally seen as the consequence of a widespread prejudice against statistically nonsignificant results. I argue that evidence of such a prejudice is in surprisingly short supply and that publication bias can occur even in the absence of such a prejudice and even if the review process is functioning perfectly. More importantly, publication bias may stem from dutiful application of standards of scientific inquiry rather than from irrational prejudice.

Type
Research Article
Information
Political Analysis , Volume 8 , Issue 2 , 16 December 1999 , pp. 201 - 210
Copyright
Copyright © 1999 by the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bartels, L. M. 1997. “Specification Uncertainty and Model Averaging.” American Journal of Political Science 41: 641–674.Google Scholar
Begg, C. B. 1994. “Publication Bias.” In The Handbook of Research Synthesis, eds. Cooper, H. and Hedges, L. V. New York: Russell Sage Foundation, pp. 400–408.Google Scholar
Begg, C. B., and Berlin, J. A. 1988. “Publication Bias: A Problem in Interpreting Medical Data.” Journal of the Royal Statistical Society, Series A 151: 419–445.Google Scholar
Bero, L. A., Glantz, S. A., and Rennie, D. 1994. “Publication Bias and Public Health Policy on Environmental Tobacco Smoke.” Journal of the American Medical Association 272: 133–136.Google Scholar
Card, D., and Krueger, A. B. 1995. “Time-Series Minimum-Wage Studies: A Meta-Analysis.” American Economic Review 85: 238–243.Google Scholar
Coursol, A., and Wagner, E. E. 1986. “Effect of Positive Findings on Submission and Acceptance Rates: A Note on Meta-Analysis Bias.” Professional Psychology 17: 136–137.Google Scholar
Cowles, M., and Davis, C. 1982. “On the Origins of the .05 Level of Statistical Significance.” American Psychologist 37: 553–558.Google Scholar
De Long, J. B., and Lang, K. 1992. “Are All Economic Hypotheses False?” Journal of Political Economy 100: 1257–1272.Google Scholar
Dickersin, K. 1990. “The Existence of Publication Bias and Risk Factors for Its Occurrence.” Journal of the American Medical Association 263: 1385–1389.Google Scholar
Dickersin, K., and Min, Y.-I. 1993. “Publication Bias: The Problem that Won't Go Away.” Annals of the New York Academy of Science 703: 135–146.CrossRefGoogle ScholarPubMed
Dickersin, K., Min, Y.-I., and Meinert, C. L. 1992. “Factors Influencing Publication of Research Results: Follow-up of Applications Submitted to Two Institutional Review Boards.” Journal of the American Medical Association 267: 374–378.Google Scholar
Gill, J. 1999. “The Insignificance of Null Hypothesis Significance Testing.” Political Research Quarterly 52: 647–674.Google Scholar
Greenwald, A. G. 1975. “Consequences of Prejudice Against the Null Hypothesis.” Psychological Bulletin 82: 1–20.CrossRefGoogle Scholar
Kuhn, T. S. 1962. The Structure of Scientific Revolutions. Chicago: University of Chicago Press.Google Scholar
Lau, R. R., Sigelman, L., Heldman, C., and Babbitt, P. 1999. “The Effects of Negative Political Advertisements: A Meta-Analytic Assessment.” American Political Science Review (in press).Google Scholar
Mahoney, M. J. 1977. “Publication Prejudices: An Experimental Study of Confirmatory Bias in the Peer Review System.” Cognitive Therapy and Research 1: 161–175.CrossRefGoogle Scholar
Meehl, P. E. 1990. “Why Summaries of Research on Psychological Theories Are Often Uninterpretable.” Psychological Reports 66: 195–244.Google Scholar
Peters, D. P., and Ceci, S. J. 1982. “Peer-Review Practices of Psychological Journals: The Fate of Published Articles Submitted Again.” Behavioral and Brain Sciences 5: 187–255.CrossRefGoogle Scholar
Rosenthal, R. 1979. “The File Drawer Problem and Tolerance for Null Results.” Psychological Bulletin 86: 638–641.Google Scholar
Rosenthal, R. 1991. Meta-Analytic Procedures for Social Research. Newbury Park, CA: Sage.Google Scholar
Rosenthal, R., and Rubin, D. B. 1988. “Comment: Assumptions and Procedures in the File Drawer Problem.” Statistical Science 3: 120–125.Google Scholar
Simon, R. 1988. “Comment on Begg and Berlin.” Journal of the Royal Statistical Society, Series A 151: 419–459.Google Scholar
Sterling, T. D. 1959. “Publication Decision and the Possible Effects on Inferences Drawn from Tests of Significance—or Vice Versa.” Journal of the American Statistical Association 54: 30–34.Google Scholar
Sterling, T. D., Rosenbaum, W. L., and Weinkam, J. J. 1995. “Publication Decisions Revisited: The Effect of the Outcome of Statistical Tests on the Decision to Publish and Vice Versa.” American Statistician 49: 108–112.Google Scholar