1. ADD A VARIABLE

Much scholarly attention has been devoted to topics such as vote choice and the onset of war. Standard sets of variables have been deployed to explain such phenomena, but a new influence can sometimes be identified or at least revisited as a result of a change in circumstance or because the influence has been overlooked or not emphasized in previous research. Therefore, the first way to advance knowledge is to test for the influence of a new or neglected variable. This approach provides an opportunity to theorize why a variable should be important and can produce evidence about whether the variable is a substantial influence.

For example, state-level variables such as the presence or level of democracy have been the traditional focus of research about the onset of war. However, Horowitz, McDermott, and Stam (Reference Horowitz, McDermott and Stam2005) presented evidence that the age of a leader is an individual-level influence on the likelihood that the leader will engage in a military dispute, with older leaders more prone to initiate conflict.

2. INTERACT VARIABLES

Many political science models presume that the influence of a variable is linear and additive, but the impact of a variable is often moderated by other factors. Therefore, the second way to advance knowledge is to test whether the impact of an explanatory measure varies in a predictable way. This approach provides an opportunity to theorize why a particular variable should be moderated by another factor and can produce evidence that a variable does or does not moderate other variables.

Horowitz, McDermott, and Stam (Reference Horowitz, McDermott and Stam2005) interacted age with regime type and found that in personalist regimes, younger leaders were more likely to initiate and intensify conflict. This result might have gone unnoticed without a theoretical expectation that the discretion of a leader to pursue personal interests moderates the influence of age.

3. SPLIT THE SAMPLE

Interaction terms are appropriate for testing the effect of a moderator variable on another variable, but if multiple variables are expected to be conditioned on the value of the moderator, then splitting the sample according to levels of the moderator may be preferable to including multiple interaction terms. Therefore, the third way to advance knowledge is to estimate a model on two or more groups within a sample. This approach provides an opportunity to theorize why a moderator variable should interact with multiple variables and can produce evidence that a variable is or is not conditioned on several other variables.

Examining female membership of national legislatures, Matland (Reference Matland1998) estimated separate models for advanced industrial countries and less-developed countries. He discovered that some predictors of female membership in the legislatures of developed countries, such as women's participation in the labor force, did not explain female membership in the legislatures of less-developed countries. This finding provided an opportunity to discuss methodological and theoretical explanations for the lack of influence of these factors in less-developed countries.

4. DISAGGREGATE

Exemplars of the previous type compare results from one group with another, but some publications focus on a single group either because of data availability or because a theory may be relevant only for a particular group. Therefore, the fourth way to advance knowledge is to analyze a disaggregated group. This approach provides an opportunity to theorize why the group should be examined separately and can produce evidence that the group is or is not different from other groups, at least in terms of previous findings.

Maoz and Russett (Reference Maoz and Russett1993) examined the onset of war between 1946 and 1986, a time frame that contained approximately 265,000 dyad-year observations. However, the authors removed dyads that were “nearly irrelevant” (627), restricting their investigation to the 12% of dyad-years that involved a politically relevant dyad, which they defined as a pair of countries that had at least one major power or were directly or indirectly contiguous with each other.

5. CHECK FOR INDIRECT EFFECTS

Statistically insignificant effects do not demand the inference that a variable has no impact, because the effect of the variable may be indirect. Therefore, the fifth way to advance knowledge is to test for an indirect influence. This approach provides an opportunity to theorize why the influence of one variable should be mediated through another variable and can produce evidence that a model presuming no indirect effects may or may not be appropriate.

Gibson (Reference Gibson1989) reported evidence that perceptions of procedural justice do not affect a willingness to accept the decision of a local council, a local court, or the Supreme Court. In contrast, Tyler and Rasinski (Reference Tyler and Rasinski1991) proposed that perceptions of procedural justice may indirectly influence acceptance of a Supreme Court decision, as shaped by perceptions of the institutional legitimacy of the Court. The authors found support for a chain reaction consistent with their theory: perceptions of procedural justice correlated with perceptions of the institutional legitimacy of the Court, and these perceptions of institutional legitimacy correlated with acceptance of a Court decision.Footnote ²

6. CHANGE PERSPECTIVE

Research tends to focus on the mean and absolute levels of variables, but the influence of a variable may be more nuanced. Therefore, the sixth way to advance knowledge is to test for a more nuanced influence of a variable. This approach provides an opportunity to theorize about the nuance and can produce evidence that the effect of a variable is more complex than would be expected from a naïve model.

Tenn (Reference Tenn2005) reported evidence that education has a stronger influence in relative rather than absolute terms. Kriner (Reference Kriner2006) examined approval ratings of Franklin D. Roosevelt in terms of their variance, uncovering insights about the influence of the level of war casualties on approval ratings that could not be gleaned from models of mean approval levels.

7. IMPROVE A MEASURE

Empirical models are only as good as the measures on which they are based. Therefore, the seventh way to advance knowledge is to enhance the methodology or theory underlying a measure. This approach provides an opportunity to theorize how a variable should be measured and can produce evidence that previous research has or has not led to inferences that are robust to alternate measurements of key variables.

Quackenbush (Reference Quackenbush2006) refined the concept of a politically relevant dyad by adding another dimension—alliances—to power status and contiguity, so that an opportunity for militarized dispute between dyad members X and Y is also present if (a) X is allied with a country Z that is contiguous to Y, (b) X is allied with a global power Z that is in a dispute with Y, or (c) X is allied with a regional power Z that is in a dispute with Y and shares a region with Y. Quackenbush reported that the new measure captured more observed militarized disputes than its competition: 357 of the 3,002 dyad-year observations with a militarized dispute that occurred between 1816 and 2000 were classified as not politically relevant using the standard contiguous/major power criteria, but only 150 of the 3,002 were misclassified using the Quackenbush criteria.

8. ANALYZE AN EXISTING MEASURE

Existing measures also need to be confirmed as reliable and valid before the inferences that they produce can be trusted. Therefore, the eighth way to advance knowledge is to test the reliability or validity of an existing measure. This approach provides an opportunity to theorize why the measure is or is not appropriate and can produce evidence that the measure should or should not be refined.

Dunlap, Gallup, and Gallup (Reference Dunlap, Gallup and Gallup1993) reported results from a cross-national survey that contained several questions about the environment. Neumayer (Reference Neumayer2002) validated some of these measures by comparing subjective opinions to related objective data; for example, the percentage of respondents in a country stating that inadequate sanitation was a very serious local problem correlated with World Bank data on the percentage of the population lacking access to sanitation.

9. TRAVEL TO ANOTHER PLACE

Much social science research has focused on the United States, but inferences made from this research may not apply elsewhere; on the other hand, theories tested in other countries may require testing in the United States to assess their generalizibility. Therefore, the ninth way to advance knowledge is to retest theories in a different geographic context. This approach provides an opportunity to theorize why a theory should or should not generalize to a different area and can produce evidence that a theory is or is not appropriate for explaining phenomena in a particular location.

MacKuen, Erikson, and Stimson (Reference MacKuen, Erikson and Stimson1992) tested competing explanations of economic considerations on presidential approval ratings in the United States: citizens may base approval ratings on the current or expected economic performance, or, on another dimension, they may focus on their personal economic situation or national business conditions. Weyland (Reference Weyland1998) tested this model in Venezuela but lacked a measure for retrospective sociotropic evaluations, which was unavailable in the data source. Results indicated that the net approval ratings of Venezuelan president Carlos Andrés Pérez were most influenced by prospective pocketbook concerns, in contrast to the prospective sociotropic dimension that dominated economic evaluations in the United States.

10. TRAVEL THROUGH TIME

Social science theories may be constrained by more than geography: theories may be doomed to obsolescence by generational replacement, modifications to institutional rules, or other temporal changes. Therefore, the tenth way to advance knowledge is to retest theories in a different era. This approach provides an opportunity to explain why a theory may be more or less applicable in earlier or later days and can produce evidence that a theory is or is not generalizable across time.

Russett and Antholis (Reference Russett and Antholis1992) examined war among city-states in ancient Greece and found that neighboring democracies in ancient Greece were not more likely to engage each other in war, even though modern experience is that neighboring countries are more likely to battle one another, as indicted in the contiguous dimension of the politically relevant dyad measure. The authors also noted that ancient states sometimes deployed their military in the expectation of financial gain, so theories suggesting that an economic cost-benefit calculation reduces the likelihood of war in modern times cannot be applied to all times.

11. TRAVEL ACROSS OR WITHIN INSTITUTIONS

Theories developed and tested at the highest level of an institution in one country may not apply to either lower levels of the institution in the same country or corresponding institutions in other countries because of variance in institutional rules, lessened salience, or different national environments. Therefore, the eleventh way to advance knowledge is to retest institutional theories in a different context. This approach provides an opportunity to explain why a theory may be more or less likely to apply in a particular environment and can produce evidence that previous findings may or may not be restricted to particular institutional arrangements.

U.S. Supreme Court research suggests that chief justices strategically use their opinion assignment authority to achieve their policy goals (Maltzman and Wahlbeck Reference Maltzman and Wahlbeck2004; Katz Reference Katz2006). But chief justices in other countries possess powers that are unavailable to the chief justice of the United States. Supreme courts in Canada and South Africa, for example, invest their chief justices with the right to assign judges to panels. Hausegger and Haynie (Reference Hausegger and Haynie2003) found that chief justices in these countries are more likely to assign a like-minded judge to a panel for a salient civil rights or civil liberties case, indicating that these officials may be strategically exploiting their additional powers.

12. TRAVEL BY ANALOGY

Theories may sometimes be conceptualized at a more restricted level than is necessary. Therefore, the twelfth way to advance knowledge is to conceptualize a theory more broadly or in a different domain. This approach provides an opportunity to explain why a theory should be expressed at a more abstract level and can produce evidence that a theory is more generalizable than previous research had indicated.

Theories of minority group threat were initially tested by analyzing the behavior of white voters in the presence of blacks, with the expectation that white political participation would increase as the percentage of blacks in an area increased. Campbell (Reference Campbell2006) found evidence that white evangelical Christians responded in a parallel manner, being more likely to support Republican candidates in areas with a higher percentage of persons who do not attend religious services and do not belong to a religious denomination.

13. TRAVEL THEORIES FROM ANOTHER DISCIPLINE

Scholars in different disciplines often work on similar problems, but the specialization of social science has created barriers that inhibit generalization. Therefore, the thirteenth way to advance knowledge is to incorporate theories from other disciplines. This approach provides an opportunity to connect disciplines to one another and promote generalization.

Linguist and cognitive scientist George Lakoff (Reference Lakoff2002) proposed that parenting metaphors inform political ideologies: conservatives prefer a strict father model that emphasizes discipline and respect for authority, while liberals prefer a nurturant parent model that prioritizes empathy and helping the needy. Barker and Tinnick (Reference Barker and Tinnick2006) tested this idea by examining whether values about the most important characteristics for children predicted political attitudes such as support for limited government and self-placement along the liberal-conservative continuum.

14. TRAVEL METHODS FROM ANOTHER DISCIPLINE

Borrowing from other disciplines can also involve techniques for collecting or analyzing data. Therefore, the fourteenth way to advance knowledge is to use a method developed or popularized in another discipline. This approach can improve the measurement and prediction of political phenomena and confirm or undercut inferences based on previous research.

Rosenbaum and Rubin (Reference Rosenbaum and Rubin1985) used propensity score matching to examine the influence of prenatal barbiturate exposure on psychological development; subsequently, Kam and Palmer (Reference Kam and Palmer2008) adopted this method to examine the influence of education on political participation. Similarly, the Tobit model was used to predict the ratio of household expenditures to disposable income (Tobin Reference Tobin1958) before it was adopted to predict the number of scholarly citations generated by Canadian political scientists' publications (Montpetit, Blais, and Foucault Reference Montpetit, Blais and Foucault2008).

15. CHANGE ESTIMATION TECHNIQUE

Sometimes, research can benefit from a re-analysis using methods common in political science. Therefore, the fifteenth way to advance knowledge is to retest a hypothesis with a different estimation technique. This approach provides an opportunity to theorize about the most appropriate modeling method and can produce evidence about the correctness of inferences drawn from previous research.

Ordinary least squares (OLS) regression is frequently used in analysis (Krueger and Lewis-Beck Reference Krueger and Lewis-Beck2008) but may sometimes be inappropriate. Mondak and Sanders (Reference Mondak and Sanders2003; Reference Mondak and Sanders2005) discussed a 15-item tolerance battery that asks respondents whether certain political rights should be extended to particular groups. Summing responses to these dichotomous items creates a tolerance scale ranging from zero (no restrictions on the indicated political rights of the specified groups) to 15 (restriction of political rights in each scenario). Previous research had employed OLS regression to analyze such summed 0-to-15 tolerance measures. OLS regression treats zero as any other number on a scale, but Mondak and Sanders noted that a zero score representing perfect tolerance may be fundamentally different from other values on the scale, which represent the degree of intolerance (Reference Mondak and Sanders2003, 497). Therefore, the factors that predict the presence of intolerance may not be the same factors that predict the level of intolerance.

Mondak and Sanders (Reference Mondak and Sanders2005) modeled responses to the 15-item tolerance battery using two different estimation techniques: an OLS regression that modeled responses to the continuous tolerance scale and a zero-inflated negative binomial regression that modeled responses to the continuous tolerance scale and values of a dichotomous presence-of-tolerance measure, coded 0 if the respondent did not restrict political rights on any of the 15 items and 1 if the respondent restricted a political right on at least one item. A statistical test comparing models indicated that the zero-inflated model fit the data better than the OLS regression model. Results from the zero-inflated model also indicated that some variables had an influence in only one aspect of tolerance: ideology, for example, was a statistically significant predictor of the presence of intolerance but not the level of intolerance.

16. CRITICIZE AN ARTICLE BY ANALYZING DATA

Methods in a publication may appear imperfect or mistaken, and re-analysis of data may corroborate such suspicions. Therefore, the sixteenth way to advance knowledge is to critique apparent errors in a publication through re-analysis of the original or related data. This approach provides an opportunity to discuss potential problems in the publication and can produce evidence that inferences drawn from previous research should or should not be trusted.

Objecting to the dichotomous political tolerance measure proposed in Mondak and Sanders (Reference Mondak and Sanders2003), Gibson (Reference Gibson2005a) noted that the tolerance battery was limited to three political rights of five specific groups: atheists, racists, Communists, militarists, and homosexuals. However, some respondents completed a follow-up interview in which they were asked to identify which groups they disliked most from a much larger list that included feminists, the American Civil Liberties Union, television evangelists, and the National Rifle Association, among other groups. 20% of respondents in the follow-up sample expressed tolerance on each item of the initial 15-item battery, but 75% of these 20% nonetheless restricted at least one of six political rights when asked about their least-liked or second-least-liked group from the larger list. Gibson suggested that the number of persons who could be considered true zeros on the tolerance scale is so small that tolerance is best measured continuously.Footnote ³

17. CRITICIZE AN ARTICLE WITHOUT ANALYZING DATA

Some critiques are informed by data analysis, but contributions to the field can also be made in the absence of data. Therefore, the seventeenth way to advance knowledge is to detail apparent errors in a publication without re-analyzing the original data. This approach provides an opportunity to identify possible issues in the publication and can develop the skills necessary for peer review.

Rothman, Lichter, and Nevitte (Reference Rothman, Lichter and Nevitte2005a) presented evidence that conservatives teach at lower-quality colleges and universities than do liberals, even when controlling for faculty achievement. But Ames et al. (Reference Ames, Barker, Bonneau and Carman2005) identified several potential problems in the analysis and presented alternate explanations for the dearth of conservatives in elite academic institutions, such as self-selection into teaching colleges because of a preference for living in areas that have fewer elite schools.Footnote ⁴

18. ADVISE

Critiquing a particular study is useful, but identifying shortcomings in multiple publications can serve as a springboard for general advice or a general rebuttal. Therefore, the eighteenth way to advance knowledge is to advise others. This approach can lead to more appropriate inferences and improve the presentation of papers.

Epstein, Martin, and Schneider (Reference Epstein, Martin and Schneider2006) and Epstein, Martin, and Boyd (Reference Epstein, Martin and Boyd2007) focused on conveying research results, and Gelman, Pasarica, and Dodhia (Reference Gelman, Pasarica and Dodhia2002) and Kastellec and Leoni (Reference Kastellec and Leoni2007) provided specific counsel about replacing tables with graphs. Kirshner (Reference Kirshner1996) observed that Alfred Hitchcock's method of filmmaking reveals lessons for communicating research results in writing, such as eliminating unnecessary information and focusing on a single concept.

19. REVIEW LITERATURE

Literature reviews are a common component of an empirical paper, but they can stand alone as a publishable manuscript. Therefore, the nineteenth way to advance knowledge is to review literature. This approach can educate readers unfamiliar with a field about the state of its knowledge and can promote deliberation about potential flaws in a theory or research area.

Field essays are broad summaries of the literature on a particular topic. Jelen and Wilcox (Reference Jelen and Wilcox2003) reviewed studies of abortion attitudes along several dimensions, such as measurement issues, changes in support for legal abortion over time, and the influence of abortion attitudes as a predictor of partisanship and electoral behavior.

Review essays can use publications as a springboard for a discussion of issues within a field. Finnemore (Reference Finnemore1996) examined the implications of sociological institutionalism for the study of international relations, noting that paradigms such as realism and liberalism begin with the presumption that states are agents, but institutionalism considers such agents to be products of social structure and thus gives rise to theories about phenomena such as statehood that international relations scholars take for granted.

20. META-ANALYZE

Publications can also serve as data for scholars who would like to review literature in a more empirical manner. Therefore, the twentieth way to advance knowledge is to conduct a meta-analysis. This approach provides the opportunity to summarize a large body of research in a systematic way and can produce statistical inferences about the state of knowledge on a research question.

Lau et al. (Reference Lau, Sigelman, Heldman and Babbitt1999) evaluated 117 relevant findings spread across 52 published and unpublished papers on the subject of negative political advertising. Meta-analysis of these results did not provide evidence that (a) negative political advertisements were viewed less favorably than positive ones; (b) negative political advertisements had a greater effect than positive ones; or (c) negative political advertisements had unintended consequences, such as decreasing participation levels.