Hostname: page-component-77c89778f8-cnmwb Total loading time: 0 Render date: 2024-07-18T15:29:59.696Z Has data issue: false hasContentIssue false
  • English
  • Français

Why Does the American National Election Study Overestimate Voter Turnout?

Published online by Cambridge University Press:  05 November 2018

Simon Jackman  and
Bradley Spahn
Simon Jackman
Affiliation:
Professor of Political Science and Chief Executive Officer of the United States Studies Centre, University of Sydney, NSW 2006, Australia. Email: simon.jackman@sydney.edu.au
Bradley Spahn*
Affiliation:
PhD candidate, Department of Political Science, Stanford University, Stanford, CA, USA. Email: bspahn@stanford.edu
*
*Email: bspahn@stanford.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Surveys are a key tool for understanding political behavior, but they are subject to biases that render their estimates about the frequency of socially desirable behaviors inaccurate. For decades the American National Election Study (ANES) has overestimated voter turnout, though the causes of this persistent bias are poorly understood. The face-to-face component of the 2012 ANES produced a turnout estimate at least 13 points higher than the benchmark voting-eligible population turnout rate. We consider three explanations for this overestimate in the survey: nonresponse bias, over-reporting and the possibility that the ANES constitutes an inadvertent mobilization treatment. Analysis of turnout data supplied by voter file vendors allows the three phenomena to be measured for the first time in a single survey. We find that over-reporting is the largest contributor, responsible for six percentage points of the turnout overestimate, while nonresponse bias and mobilization account for an additional 4 and 3 percentage points, respectively.

Keywords

survey designmeasurement errortreatment effects
Type
Articles
Information
Political Analysis , Volume 27 , Issue 2 , April 2019 , pp. 193 - 207
Copyright
Copyright © The Author(s) 2018. Published by Cambridge University Press on behalf of the Society for Political Methodology. 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Contributing Editor: Jonathan N. Katz

Authors’ note: We thank Matthew DeBell (Senior Research Scientist, ANES Stanford), Patricia Luevano (ANES Michigan) and Kyle Dropp for valuable research assistance and recognize support from the National Science Foundation (SES-0937715). Earlier versions of this paper were presented at the summer meetings of the Society for Political Methodology (University of Georgia, July 2014) and the University of Michigan’s Interdisciplinary Seminar in Quantitative Methods. We thank Bob Erikson, Don Green and two anonymous reviewers for valuable comments. An anonymized replication archive is available in Jackman & Spahn (2018).

References

Abadie, Alberto, and Imbens, Guido W.. 2006. Large sample properties of matching estimators for average treatment effects. Econometrica 74(1):235–267.Google Scholar
Abelson, Robert P., Loftus, Elizabeth F., and Greenwald, Anthony G.. 1992. Attempts to improve the accuracy of self-reports of voting. In Questions about questions , ed. Tanur, Judith M.. New York: Russell Sage Foundation, pp. 138–153.Google Scholar
Aldrich, John H., and McGraw, Kathleen M.. 2011. Improving public opinion surveys: Interdisciplinary innovation and the American National Election Studies . Princeton, NJ: Princeton University Press.Google Scholar
American National Election Studies. 2014. User’s guide and codebook for the ANES 2012 time series study. Ann Arbor, MI, and Stanford, CA: University of Michigan and Stanford University. http://electionstudies.org/studypages/anes_timeseries_2012/anes_timeseries_2012_userguidecodebook.pdf.Google Scholar
American National Election Studies. 2016. User’s guide and codebook for the ANES 2012 time series voter validation supplemental data. Ann Arbor, MI, and Stanford, CA: University of Michigan and Stanford University. http://www.electionstudies.org/studypages/data/anes_timeseries_2012/anes_timeseries_2012_voteval_sav.zip.Google Scholar
Anderson, Barbara A., and Silver, Brian D.. 1986. Measurement and mismeasurement of the validity of the self-reported vote. American Journal of Political Science 30(4):771–785. http://www.jstor.org/stable/2111272.Google Scholar
Ansolabehere, Stephen, and Hersh, Eitan. 2010. The quality of voter registration records: A state-by-state analysis. Report, Caltech/MIT Voting Technology Project. http://v-ads-web5.ads.caltech.edu/content/quality-voter-registration-records-state-state-analysis.Google Scholar
Ansolabehere, Stephen, and Hersh, Eitan. 2012. Validation: What big data reveal about survey misreporting and the real electorate. Political Analysis 20(4):437–459.Google Scholar
Arceneaux, Kevin, and Nickerson, David W.. 2009. Who is mobilized to vote? A re-analysis of 11 field experiments. American Journal of Political Science 53(1):1–16.Google Scholar
Belli, Robert F., Traugott, Michael W., Young, Margaret, and McGonagle, Katherine A.. 1999. Reducing vote overreporting in surveys: Social desirablity, memory failure, and source monitoring. Public Opinion Quarterly 63(1):90–108. http://poq.oxfordjournals.org/content/63/1/90.short.Google Scholar
Belli, Robert F., Traugott, Michael W., and Beckmann, Matthew N.. 2001. What leads to voting overreports? Contrasts of overreporters to validated voters and admitted nonvoters in the American National Election Studies. Journal of Official Statistics 17(4):479.Google Scholar
Berent, Matthew K., Krosnick, Jon A., and Lupia, Arthur. 2016. Measuring voter registration and turnout in surveys do official government records yield more accurate assessments? Public Opinion Quarterly 80: p. nfw021.Google Scholar
Brehm, John. 1993. The phantom respondents: Opinion surveys and political representation . Ann Arbor, MI: University of Michigan Press.Google Scholar
Burden, Barry C. 2000. Voter turnout and the national election studies. Political Analysis 8(4):389–398.Google Scholar
Campbell, Angus, Miller, Warren E., and Converse, Philip E.. 1960. The American voter . New York: Wiley.Google Scholar
Clausen, Aage R. 1968. Response validity: Vote report. Public Opinion Quarterly 32:588–606.Google Scholar
Czajka, John L. 2013. Can administrative records be used to reduce nonresponse bias? The Annals of the American Academy of Political and Social Science 645(1):171–184. http://ann.sagepub.com/content/645/1/171.abstract.Google Scholar
Green, Donald P., and Gerber, Alan S.. 2008. Get out the vote: How to increase voter turnout . Washington, DC: Brookings Institution Press.Google Scholar
Greenwald, Anthony G., Carnot, Catherine G., Beach, Rebecca, and Young, Barbara. 1987. Increasing voting behavior by asking people if they expect to vote. Journal of Applied Psychology 72:315–318.Google Scholar
Groves, Robert M. 2006. Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly 70(5):646–675. http://poq.oxfordjournals.org/content/70/5/646.abstract.Google Scholar
Groves, Robert M., and Couper, M. P.. 1998. Nonresponse in household interview surveys . New York: Wiley.Google Scholar
Hill, Kim Quaile, and Hurley, Patricia A.. 1984. Nonvoters in voters’ clothing: The impact of voting behavior misreporting on voting behavior research. Social Science Quarterly 65(1):199–206. http://www.jstor.org/stable/42861691.Google Scholar
Issenberg, Sasha. 2012. The victory lab: The secret science of winning campaigns . Random House Digital, Inc.Google Scholar
Kanazawa, Satoshi. 2005. Who lies on surveys, and what can we do about it? The Journal of Social, Political and Economic Studies 30(3):361–370.Google Scholar
Krumpal, Ivar. 2013. Determinants of social desirability bias in sensitive surveys: A literature review. Quality and Quantity 47(4):2025–2047. http://dx.doi.org/10.1007/s11135-011-9640-9.Google Scholar
Little, Roderick J. A., and Rubin, Donald B.. 2002. Statistical analysis with missing data , second edn. New York: Wiley.Google Scholar
McDonald, Michael. 2013. 2012 General election turnout rates. Technical report United States elections project, George Mason University. http://elections.gmu.edu/Turnout_2012G.html.Google Scholar
Nickerson, David W., and Rogers, Todd. 2010. Do you have a voting plan? Implementation intentions, voter turnout, and organic plan making. Psychological Science 21(2):194–199.Google Scholar
Olson, Kristen, and Witt, Lindsey. 2011. Are we keeping the people who used to stay? Changes in correlates of panel survey attrition over time. Social Science Research 40(4):1037–1050.Google Scholar
Parry, Hugh J., and Crossley, Helen M.. 1950. Validity of responses to survey questions. Public Opinion Quarterly 14(1):61–80. http://poq.oxfordjournals.org/content/14/1/61.abstract.Google Scholar
Persson, Mikael. 2014. Does survey participation increase voter turnout?: Re-examining the Hawthorne effect in the Swedish National Election Studies. Political Science Research and Method 2:1–11. doi:10.1017/psrm.2014.8.Google Scholar
Silver, Brian D., Anderson, Barbara A., and Abramson, Paul R.. 1986. Who overreports voting? American Political Science Review 80(2):613–624. http://www.jstor.org/stable/1958277.Google Scholar
Smith, Jennifer K., Gerber, Alan S., and Orlich, Anton. 2003. Self-Prophecy effects and voter turnout: An experimental replication. Political Psychology 24(3):593–604. http://dx.doi.org/10.1111/0162-895X.00342.Google Scholar
Spahn, Bradley, and Jackman, Simon. 2018. Replication data for: Why does the ANES overestimate voter turnout? https://doi.org/10.7910/DVN/NQVP7K, Harvard Dataverse, V1, UNF:6:Q4EoerxQKHMUI8ZcQeLkfQ==.Google Scholar
Traugott, Michael W., and Katosh, John P.. 1979. Response validity in surveys of voting behavior. Public Opinion Quarterly 43(3):359–377. http://poq.oxfordjournals.org/content/43/3/359.abstract.Google Scholar
Traugott, Santa. 1989. Validating self-reported vote: 1964–1988 . Ann Arbor, MI: Institute for Social Research, University of Michigan.Google Scholar
Verba, Sidney, Schlozman, Kay Lehman, and Brady, Henry E.. 1995. Voice and equality: Civic voluntarism in American politics . New Haven, CT: Harvard University Press.Google Scholar
Weisberg, Herbert F. 2009. The total survey error approach: A guide to the new science of survey research . Chicago, IL: University of Chicago Press.Google Scholar
Wolfinger, Raymond E., and Rosenstone, Steven J.. 1980. Who votes?   Vol. 22 . New Haven, CT: Yale University Press.Google Scholar
Supplementary material: File

Jackman and Spahn supplementary material

Appendix

Download Jackman and Spahn supplementary material(File)
File 34.5 KB