Over the last decade the survey environment has been rapidly changing, creating new opportunities and new challenges for survey methodologists. These include: relatively low response among all contact modes and designs; the rapid replacement of landline telephones for mobile and smart phones; the integration of new and different types of data for sampling and new sampling methods; the use of different survey interview modes; how to study behaviors and opinions about sensitive topics; and the development of survey designs for experts and opinion leaders. All have implications for data quality and for our ability to consistently and reliably answer substantive and theoretical questions about politics and society.

Survey experiments are a core tool for causal inference. Yet, the design of classical survey experiments prevents them from identifying which components of a multidimensional treatment are influential. Here, we show how conjoint analysis, an experimental design yet to be widely applied in political science, enables researchers to estimate the causal effects of multiple treatment components and assess several causal hypotheses simultaneously. In conjoint analysis, respondents score a set of alternatives, where each has randomly varied attributes. Here, we undertake a formal identification analysis to integrate conjoint analysis with the potential outcomes framework for causal inference. We propose a new causal estimand and show that it can be nonparametrically identified and easily estimated from conjoint data using a fully randomized design. The analysis enables us to propose diagnostic checks for the identification assumptions. We then demonstrate the value of these techniques through empirical applications to voter decision making and attitudes toward immigrants.

We propose the use of Kernel Regularized Least Squares (KRLS) for social science modeling and inference problems. KRLS borrows from machine learning methods designed to solve regression and classification problems without relying on linearity or additivity assumptions. The method constructs a flexible hypothesis space that uses kernels as radial basis functions and finds the best-fitting surface in this space by minimizing a complexity-penalized least squares problem. We argue that the method is well-suited for social science inquiry because it avoids strong parametric assumptions, yet allows interpretation in ways analogous to generalized linear models while also permitting more complex interpretation to examine nonlinearities, interactions, and heterogeneous effects. We also extend the method in several directions to make it more effective for social inquiry, by (1) deriving estimators for the pointwise marginal effects and their variances, (2) establishing unbiasedness, consistency, and asymptotic normality of the KRLS estimator under fairly general conditions, (3) proposing a simple automated rule for choosing the kernel bandwidth, and (4) providing companion software. We illustrate the use of the method through simulations and empirical examples.

Transparency is often viewed as crucial to government accountability, but its measurement remains elusive. This concept encompasses many dimensions, which have distinct effects. In this article, we focus on a specific dimension of transparency: governments' collection and dissemination of aggregate data. We construct a measure of this aspect of transparency, using an item response model that treats transparency as a latent predictor of the reporting of data to the World Bank's World Development Indicators. The resultant index covers 125 countries from 1980 to 2010. Unlike some alternatives (e.g., Freedom House), our measure—the HRV index—is based on objective criteria rather than subjective expert judgments. Unlike newspaper circulation numbers, HRV reflects the dissemination of credible content—in that it has survived the World Bank's quality control assessment. In a validation exercise, we find that our measure outperforms newspaper circulation as a predictor of Law and Order and Bureaucratic Quality as measured by the ICRG, particularly in autocracies. It performs as well as newspaper circulation in predicting corruption. These findings suggest that data dissemination is a distinct, and politically relevant, form of transparency.

In this article, we propose a polling accuracy measure for multi-party elections based on a generalization of Martin, Traugott, and Kennedy's two-party predictive accuracy index. Treating polls as random samples of a voting population, we first estimate an intercept only multinomial logit model to provide proportionate odds measures of each party's share of the vote, and thereby both unweighted and weighted averages of these values as a summary index for poll accuracy. We then propose measures for significance testing, and run a series of simulations to assess possible bias from the resulting folded normal distribution across different sample sizes, finding that bias is small even for polls with small samples. We apply our measure to the 2012 French presidential election polls to demonstrate its applicability in tracking overall polling performance across time and polling organizations. Finally, we demonstrate the practical value of our measure by using it as a dependent variable in an explanatory model of polling accuracy, testing the different possible sources of bias in the French data.

The estimation of causal effects has a revered place in all fields of empirical political science, but a large volume of methodological and applied work ignores a fundamental fact: most people are skeptical of estimated causal effects. In particular, researchers are often worried about the assumption of no omitted variables or no unmeasured confounders. This article combines two approaches to sensitivity analysis to provide researchers with a tool to investigate how specific violations of no omitted variables alter their estimates. This approach can help researchers determine which narratives imply weaker results and which actually strengthen their claims. This gives researchers and critics a reasoned and quantitative approach to assessing the plausibility of causal effects. To demonstrate the approach, I present applications to three causal inference estimation strategies: regression, matching, and weighting.

In this article, we challenge the conclusion that the preferences of members of Congress are best represented as existing in a low-dimensional space. We conduct Monte Carlo simulations altering assumptions regarding the dimensionality and distribution of member preferences and scale the resulting roll call matrices. Our simulations show that party polarization generates misleading evidence in favor of low dimensionality. This suggests that the increasing levels of party polarization in recent Congresses may have produced false evidence in favor of a low-dimensional policy space. However, we show that focusing more narrowly on each party caucus in isolation can help researchers discern the true dimensionality of the policy space in the context of significant party polarization. We re-examine the historical roll call record and find evidence suggesting that the low dimensionality of the contemporary Congress may reflect party polarization rather than changes in the dimensionality of policy conflict.

In this article, we present data from a three-mode survey comparison study carried out in 2010. National surveys were fielded at the same time over the Internet (using an opt-in Internet panel), by telephone with live interviews (using a national Random Digit Dialing (RDD) sample of landlines and cell phones), and by mail (using a national sample of residential addresses). Each survey utilized a nearly identical questionnaire soliciting information across a range of political and social indicators, many of which can be validated with government data. Comparing the findings from the modes using a Total Survey Error approach, we demonstrate that a carefully executed opt-in Internet panel produces estimates that are as accurate as a telephone survey and that the two modes differ little in their estimates of other political indicators and their correlates.

International relations scholars frequently rely on data sets with country pairs, or dyads, as the unit of analysis. Dyadic data, with its thousands and sometimes hundreds of thousands of observations, may seem ideal for hypothesis testing. However, dyadic observations are not independent events. Failure to account for this dependence in the data dramatically understates the size of standard errors and overstates the power of hypothesis tests. We illustrate this problem by analyzing a central proposition among IR scholars, the democratic trade hypothesis, which claims that democracies seek out other democracies as trading partners. We employ randomization tests to infer the correct p-values associated with the trade hypotheses. Our results show that typical statistical tests for significance are severely overconfident when applied to dyadic data.

Estimators within the Cox family are often used to estimate models for repeated events. Yet, there is much we still do not know about the performance of these estimators. In particular, we do not know how they perform given time dependence, different censoring rates, and a varying number of events and sample sizes. We use Monte Carlo simulations to demonstrate the performance of a variety of popular semi-parametric estimators as these data aspects change and under conditions of event dependence and heterogeneity, both, or neither. We conclude that the conditional frailty model outperforms other standard estimators under a wide array of data-generating processes, and data limitations rarely alter its performance.

Latent variable models can only be compared across groups when these groups exhibit measurement equivalence or “invariance,” since otherwise substantive differences may be confounded with measurement differences. This article suggests examining directly whether measurement differences present could confound substantive analyses, by examining the expected parameter change (EPC)-interest. The EPC-interest approximates the change in parameters of interest that can be expected when freeing cross-group invariance restrictions. Monte Carlo simulations suggest that the EPC-interest approximates these changes well. Three empirical applications show that the EPC-interest can help avoid two undesirable situations: first, it can prevent unnecessarily concluding that groups are incomparable, and second, it alerts the user when comparisons of interest may still be invalidated even when the invariance model appears to fit the data. R code and data for the examples discussed in this article are provided in the electronic appendix (http://hdl.handle.net/1902.1/21816).

We examine the quality of two probability-based polls, one interviewer administered (telephone) and one self-administered (Internet and mail mixed mode survey). The polls use the same sampling frame (registered voters) and the same questions. First, we examine the representativeness of both surveys using information known about the population, and although we find important differences between the two in terms of sampling and nonresponse bias, we also find that both surveys represent the underlying population despite low response rates. We also test for mode effects between surveys due to social desirability and how it influences nondifferentiation or satisficing. Using a variety of methods (t-tests, multivariate regression, and genetic propensity matching), we find evidence that the presence of an interviewer alters response patterns on ego-driven questions. The implications of our work are important, due to the increasing popularity of mixed mode surveys. Researchers need to be methodologically sensitive to these differences when analyzing surveys that allow for different response modes.

Multifactor error structures utilize factor analysis to deal with complex cross-sectional dependence in Time-Series Cross-Sectional data caused by cross-level interactions. The multifactor error structure specification is a generalization of the fixed-effects model. This article extends the existing multifactor error models from panel econometrics to multilevel modeling, from linear setups to generalized linear models with the probit and logistic links, and from assuming serial independence to modeling the error dynamics with an autoregressive process. I develop Markov Chain Monte Carlo algorithms mixed with a rejection sampling scheme to estimate the multilevel multifactor error structure model with a pth-order autoregressive process in linear, probit, and logistic specifications. I conduct several Monte Carlo studies to compare the performance of alternative specifications and approaches with varying degrees of data complication and different sample sizes. The Monte Carlo studies provide guidance on when and how to apply the proposed model. An empirical application sovereign default demonstrates how the proposed approach can accommodate a complex pattern of cross-sectional dependence and helps answer research questions related to units' sensitivity or vulnerability to systemic shocks.

This article presents a scaling approach to jointly estimate the locations of voters, parties, and European political groups on a common left-right scale. Although most comparative research assumes that cross-national comparisons of voters and parties are possible, few correct for systematic biases commonly known to exist in surveys or examine whether survey data are comparable across countries. Our scaling method addresses scale perception in surveys and links cross-national surveys through new bridging observations. We apply our approach to the 2009 European Election Survey and demonstrate that the improvement in party estimates that one gains from fixing various survey bias issues is significant. Our scaling strategy provides left-right positions of voters and of 162 political parties, and we demonstrate that variables based on rescaled voter and party positions on the left-right dimension significantly improve the fit of a cross-national vote choice model.

A state-dependent dynamic system is one in which (1) the marginal effect of x on y at time t () depends on the prior value of the dependent variable , and (2) the persistence of the dependent variable () depends on xt. We present a methodological strategy for dealing with state-dependent dynamic systems and demonstrate the consequences of ignoring state-dependence. As an applied example, we find evidence of state-dependence in the relationship between presidential approval and economic performance: high unemployment rates are most damaging to presidential approval among presidents with the highest initial approval ratings.

This article outlines a new method for surveys to study elections and voter attitudes. Pre-election surveys often suffer from an inability to identify and survey the likely electorate for the upcoming election. We propose a new and inexpensive method to conduct representative surveys of the electorate. We demonstrate the performance of our method in producing a representative sample of the future electorate that can be used to study campaign dynamics and many other issues. We compare pre-election outcome forecasts to election outcomes in seven primary and general election surveys conducted prior to the 2008 and 2010 primary and general elections in three states. The results indicate that the methodology produces representative samples, including in low-turnout elections such as primaries where traditional methods have difficulty consistently sampling the electorate. This new methodology combines Probability Proportional to Size (PPS) sampling, mailed invitation letters, and online administration of the questionnaire. The PPS sample is drawn based on a model employing variables from the publicly available voter file to produce a probability of voting score for each individual voter. The proposed method provides researchers a valuable tool to study the attitudes of the voting public.