Hostname: page-component-848d4c4894-jbqgn Total loading time: 0 Render date: 2024-07-01T10:44:33.187Z Has data issue: false hasContentIssue false
  • English
  • Français

The Effect of Survey Mode and Sampling on Inferences about Political Attitudes and Behavior: Comparing the 2000 and 2004 ANES to Internet Surveys with Nonprobability Samples

Published online by Cambridge University Press:  04 January 2017

Neil Malhotra  and
Jon A. Krosnick
Neil Malhotra
Affiliation:
Department of Political Science, Stanford University, Encina Hall West, Room 100, Stanford, CA 94305-6044. e-mail: neilm@stanford.edu (corresponding author)
Jon A. Krosnick
Affiliation:
Departments of Communication, Political Science, and Psychology, Stanford University, 434 McClatchy Hall, 450 Serra Mall, Stanford, CA 94305. e-mail: krosnick@stanford.edu
Get access Rights & Permissions [Opens in a new window]

Abstract

Since the inception of the American National Election Study (ANES) in the 1940s, data have been collected via face-to-face interviewing in the homes of members of area probability samples of American adults, the same gold-standard approach used by the U.S. Census Bureau, other federal agencies, and some nongovernment researchers for many of the most high-profile surveys conducted today. This paper explores whether comparable findings about voters and elections would be obtained by a different, considerably less expensive method: Internet data collection from nonprobability samples of volunteer respondents. Comparisons of the 2000 and 2004 ANES data (collected via face-to-face interviewing with national probability samples) with simultaneous Internet surveys of volunteer samples yielded many differences in the distributions of variables and in the associations between variables (even controlling for differences between the samples in reported interest in politics). Accuracy was higher for the face-to-face/probability sample data than for the Internet/volunteer sample data in 88% of the possible comparisons. This suggests that researchers interested in assuring the accuracy of their findings in describing populations should rely on face-to-face surveys of probability samples rather than Internet samples of volunteer respondents.

Type
Research Article
Information
Political Analysis , Volume 15 , Issue 3 , Summer 2007 , pp. 286 - 323
Copyright
Copyright © The Author 2007. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Chang, LinChiat, and Krosnick, Jon A. 2002. Comparing self-administered computer surveys and auditory interviews: An experiment. Paper presented at the American Association for Public Opinion Research Annual Meeting, St. Petersburg, FL.Google Scholar
Converse, Philip E. 1964. The nature of belief systems in mass publics. In Ideology and discontent, ed. Apter, D. E., 206–61. New York: Free Press.Google Scholar
Erikson, Robert S., and Tedin, Kent L. 2007. American public opinion. 7th ed., updated. New York: Longman.Google Scholar
Holbrook, Allyson L., Green, Melanie C., and Krosnick, Jon A. 2003. Telephone vs. face-to-face interviewing of national probability samples with long questionnaires: Comparisons of respondent satisficing and social desirability response bias. Public Opinion Quarterly 67: 79–125.CrossRefGoogle Scholar
Rossi, Peter H., Wright, James D., and Anderson, Andy B. 1983. Handbook of survey research. Orlando, FL: Academic Press.Google Scholar
Saris, Willem. 1998. Ten years of interviewing without interviewers: The Telepanel. In Computer assisted survey information collection, ed. Couper, Mick P., Baker, Reginald P., Bethlehem, Jelke, Clark, Cynthia Z. F., Martin, Jean, Nicholls, William L. II, and O'Reilly, James M., 409–29. New York: Wiley.Google Scholar
Sears, David O. 1986. College sophomores in the laboratory: Influences of a narrow database on social psychology's view of human nature. Journal of Personality and Social Psychology 51 (September): 515–30.CrossRefGoogle Scholar
Verba, Sidney, Schlozman, Kay Lehman, and Brady, Henry E. 1995. Voice and equality: Civic voluntarism in American politics. Cambridge, MA: Harvard University Press.CrossRefGoogle Scholar
Wolfinger, Raymond E., and Rosenstone, Steven J. 1980. Who votes? New Haven, CT: Yale University Press.Google Scholar