Article contents
Analyze the attentive and bypass bias: mock vignette checks in survey experiments
Published online by Cambridge University Press: 03 February 2023
Abstract
Respondent inattentiveness threatens to undermine causal inferences in survey-based experiments. Unfortunately, existing attention checks may induce bias while diagnosing potential problems. As an alternative, we propose “mock vignette checks” (MVCs), which are objective questions that follow short policy-related passages. Importantly, all subjects view the same vignette before the focal experiment, resulting in a common set of pre-treatment attentiveness measures. Thus, interacting MVCs with treatment indicators permits unbiased hypothesis tests despite substantial inattentiveness. In replications of several experiments with national samples, we find that MVC performance is significantly predictive of stronger treatment effects, and slightly outperforms rival measures of attentiveness, without significantly altering treatment effects. Finally, the MVCs tested here are reliable, interchangeable, and largely uncorrelated with political and socio-demographic variables.
- Type
- Original Article
- Information
- Copyright
- Copyright © The Author(s), 2023. Published by Cambridge University Press on behalf of the European Political Science Association
References
- 8
- Cited by