Hostname: page-component-848d4c4894-wzw2p Total loading time: 0 Render date: 2024-06-07T15:57:20.257Z Has data issue: false hasContentIssue false
  • English
  • Français

A New Multinomial Accuracy Measure for Polling Bias

Published online by Cambridge University Press:  04 January 2017

Kai Arzheimer  and
Jocelyn Evans
Kai Arzheimer*
Affiliation:
University of Mainz, Dept. of Political Science, Colonel-Kleinmann-Weg 2, Mainz, Germany
Jocelyn Evans
Affiliation:
University of Leeds, POLIS, Leeds LS2 9JT, United Kingdom e-mail: j.a.j.evans@leeds.ac.uk
*
e-mail: kai.arzheimer@googlemail.com (corresponding author)
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

In this article, we propose a polling accuracy measure for multi-party elections based on a generalization of Martin, Traugott, and Kennedy's two-party predictive accuracy index. Treating polls as random samples of a voting population, we first estimate an intercept only multinomial logit model to provide proportionate odds measures of each party's share of the vote, and thereby both unweighted and weighted averages of these values as a summary index for poll accuracy. We then propose measures for significance testing, and run a series of simulations to assess possible bias from the resulting folded normal distribution across different sample sizes, finding that bias is small even for polls with small samples. We apply our measure to the 2012 French presidential election polls to demonstrate its applicability in tracking overall polling performance across time and polling organizations. Finally, we demonstrate the practical value of our measure by using it as a dependent variable in an explanatory model of polling accuracy, testing the different possible sources of bias in the French data.

Type
Research Article
Information
Political Analysis , Volume 22 , Issue 1 , Winter 2014 , pp. 31 - 44
Copyright
Copyright © The Author 2013. Published by Oxford University Press on behalf of the Society for Political Methodology 

Footnotes

Authors' note: We are grateful to two anonymous reviewers and to the editors of Political Analysis for their extremely helpful comments and suggestions. We would also like to thank Gilles Ivaldi (CNRS/University of Nice) for providing data on the French election surveys. Replication materials can be found in Arzheimer and Evans (2013). surveybias.ado, a Stata add-on that facilitates the computation of B and Bw, is available from the Boston College Statistical Software Components (SSC) archive. Supplementary materials for this article are available on the Political Analysis Web site.

References

Agresti, Alan. 2002. Categorical data analysis. 2nd ed. Hoboken, NJ: John Wiley.Google Scholar
Aitchison, John. 1982. The statistical analysis of compositional data (with discussion). Journal of the Royal Statistical Society, Series B 44(2): 139–77.Google Scholar
Alvarez, R. Michael, and Nagler, Jonathan. 1998. When politics and models collide: Estimating models of multiparty elections. American Journal of Political Science 42: 55–96.Google Scholar
Arzheimer, Kai, and Evans, Jocelyn. 2013. Replication data for: A new multinomial accuracy measure for polling bias. IQSS Dataverse Network/Distributor V1 [Version]. http://hdl.handle.net/1902.1/21603 (accessed August 6, 2013).Google Scholar
Bacon-Shone, John. 2011. A short history of compositional data analysis. In Compositional data analysis: Theory and applications, eds. Pawlowsky-Glahn, Vera and Buccianti, Antonella, 1–11. Chichester, UK: Wiley.Google Scholar
Callegaro, Mario, and Gasperoni, Giancarlo. 2008. Accuracy of pre-election polls for the 2006 Italian parliamentary election: Too close to call. International Journal of Public Opinion Research 20(2): 148–70.Google Scholar
Cheng, Simon, and Scott Long, J. 2007. Testing for IIA in the multinomial logit model. Sociological Methods & Research 35(4): 583–600.Google Scholar
Dow, Jay K., and Endersby, James W. 2004. Multinomial probit and multinomial logit: A comparison of choice models for voting research. Electoral Studies 23: 107–22.Google Scholar
Durand, Claire. 2008. The polls of the 2007 French presidential campaign: Were lessons learned from the 2002 catastrophe? International Journal of Public Opinion Research 20(3): 275–98.Google Scholar
Evans, Jocelyn. 2012. The sound foundations of a socialist victory. Renewal 20 (2–3): 123–28.Google Scholar
Fisher, Stephen D., Ford, Robert, Jennings, Will, Pickup, Mark, and Wlezien, Christopher. 2011. From polls to votes to seats: Forecasting the 2010 British general election. Electoral Studies 30(2): 250–57.Google Scholar
Jackman, Simon. 2005. Pooling the polls over an election campaign. Australian Journal of Political Science 40(4): 499–517.Google Scholar
Kropko, Jonathan. 2010. A comparison of three discrete choice estimators (unpublished dissertation chapter). University of North Carolina-Chapel Hill. http://www.unc.edu/∼kropko/paper1.pdf (accessed August 6, 2013).Google Scholar
Laakso, Markku, and Taagepera, Rein. 1979. “Effective” number of parties: A measure with application to West Europe. Comparative Political Studies 12(1): 3–27.Google Scholar
Long, J. Scott. 1997. Regression models for categorical and limited dependent variables. Thousand Oaks, CA: Sage.Google Scholar
Luce, R. Duncan. 1959. Individual choice behavior. A theoretical analysis. New York: John Wiley & Sons.Google Scholar
Martin, Elizabeth A., Traugott, Michael W., and Kennedy, Courtney. 2005. A review and proposal for a new measure of poll accuracy. Public Opinion Quarterly 69(3): 342–69.Google Scholar
McFadden, Daniel. 1973. Conditional logit analysis of qualitative choice behaviour. In Frontiers of Econometrics, ed. Zarembka, Paul, 105–42. New York: Academic Press.Google Scholar
McLean, Iain. 1995. Independence of irrelevant alternatives before Arrow. Mathematical Social Sciences 30(2): 107–26.Google Scholar
Mosteller, Frederick, Hyman, Herbert, McCarthy, Philip, Marks, Eli, and Truman, David. 1949. The pre-election polls of 1948: Report to the committee on analysis of pre-election polls and forecasts. New York: Social Science Research Council.Google Scholar
Pawlowsky-Glahn, Vera, and Buccianti, Antonella, eds. 2011. Compositional data analysis: Theory and applications. Chichester, UK: Wiley.CrossRefGoogle Scholar
Pearson, Karl. 1897. Mathematical contributions to the theory of evolution: On a form of spurious correlation which may arise when indices are used in the measurements of organs. Proceedings of the Royal Society of London 60: 489–502.Google Scholar
Ray, Paramesh. 1973. Independence of irrelevant alternatives. Econometrica 41(5): 987–91.Google Scholar
Royston, Patrick, and Sauerbrei, Willi. 2008. Multivariable model-building: A pragmatic approach to regression analysis based on fractional polynomials for modelling continuous variables. Chichester, UK: Wiley.CrossRefGoogle Scholar
Schaffer, Lena-Maria, and Schneider, Gerald. 2005. Die Prognosegte von Wahlbörsen und Meinungsumfragen zur Bundestagswahl 2005. Politische Vierteljahresschrift 46(4): 674–81.Google Scholar
Train, Kenneth. 2009. Discrete choice methods with simulation. 2nd ed. Cambridge, UK: Cambridge University Press.Google Scholar
Whitten, Guy D., and Palmer, Harvey D. 1996. Heightening comparativists' concern for model choice: Voting behavior in Great Britain and the Netherlands. American Journal of Political Science 40(1): 231–60.CrossRefGoogle Scholar
Supplementary material: File

Arzheimer and Evans supplementary material

Supplementary Material

Download Arzheimer and Evans supplementary material(File)
File 22.6 KB