Hostname: page-component-77c89778f8-m8s7h Total loading time: 0 Render date: 2024-07-16T20:15:19.836Z Has data issue: false hasContentIssue false

Notes from the Editors

APSR Reviewers and the Review Process during the Global Pandemic

Published online by Cambridge University Press:  27 October 2022

Rights & Permissions [Opens in a new window]

Abstract

Type
Editorial
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of the American Political Science Association

In the two years since our team assumed the editorship of the American Political Science Review, the United States and world have experienced a series of tumultuous local, national, and world events. Like many other political scientists, we not only study the politics of such events but have also had our lives touched by them. Millions of people have died during the global COVID-19 pandemic, which also disrupted work and education around the world. As journalistic accounts (Flaherty Reference Flaherty2020) and early research make clear (Kim and Patterson Reference Kim and Patterson2022; Shalaby, Allam, and Buttorff Reference Shalaby, Allam and Buttorff2021), our experiences of pandemic-related disruptions are not unique. Through our regular correspondence with hundreds of authors and reviewers, we hear painful narratives about COVID diagnoses, hospitalizations, family crises and loss, and childcare and eldercare demands, as well as exits from the profession. We also hear from reviewers who feel distracted, angry, dispirited, and unable to do their work because of the personal and devastating consequences of current political events.

Therefore, we want to begin this Notes from the Editors by thanking the 3,938 reviewers who have completed or are completing 5,331 reviews of the 1,877 new manuscript submissions our team has sent out for review since June 1, 2020. Many of these reviewers have reviewed revised manuscripts as well, in some cases more than once. To our reviewers we say:

We are grateful for and amazed by your generosity to the authors, to the members of our team, and to the discipline in sharing your expertise as part of the review process, particularly given the difficulties of the last two years. We may not always agree with your recommendations, and you may not always agree with our decisions, but this work could not be done without your contributions to the journal and discipline. Thank you.

At our APSA 2022 reception, we recognized reviewers who have provided exceptional service since June 2020. In the rest of this Notes from the Editors, we describe who is typically invited to review and who ultimately reviews manuscripts for the journal, with some attention to the probable effects of the global pandemic on various aspects of the review process.

Who Is Invited to Review for the APSR?

In our Vision Statement (APSA N.d.), we articulated six principles that guide our work. Among these principles is a commitment to pursue “substantive, representational, and methodological diversity” in the work that we publish and to “increase the diversity of submissions, authors, reviewers, and citations along lines including race, gender, sexuality, ability, national origin, and type of institution.”

In pursuit of these objectives, we always seek to invite reviewers who are experts in the substance or methodology of a particular manuscript. This implies that as our submissions become more substantively or methodologically diverse, so should our invited reviewers. Like previous teams, we also seek to expand our pool of reviewers by inviting junior scholars and others with no prior APSR reviewing experience to review for the journal. The two most recent APSR editorial teams (2016–2020, 2020–2024) each articulated goals related to expanding the reach and relevance of the journal, either with geographic (APSA 2016) or substantive and methodological (APSA 2020) diversity. As a result, 53.1% of reviewers who submitted reviews for the prior Mannheim teamFootnote 1 (2016–2020) were new reviewers and had not completed a review for the journal in the previous eight years. We’ve continued to build on their efforts; 41.3% of our reviewers have not reviewed for the journal for any of the last three editorial teams (2008–2020). In addition, a significant majority of APSR reviewers has not recently published in the journal. Under the leadership of the prior editorial team, only 7.1% of reviewers had published in the APSR during the previous eight years. Similarly, only 10.8% of our reviewers published in the APSR under the two previous editorial teams.

Our ability to measure our progress toward more specific goals of diversifying the reviewer pool is limited by the demographic information that the APSR collects. The reviewer database does not include data about things like gender identity, sexual orientation, ethnicity, or race. The database also does not keep historical profile information and therefore may also not accurately reflect reviewer rank, affiliation, or areas of specialization if the reviewer was initially added several years ago and has not updated their profile recently.

With these caveats in mind, we provide an overview of APSR reviewers that were invited by our team and recent teams in Table 1. The most notable shift under our team is the increase (from 30.2% under the previous team) in the proportion of invited reviewers who are women (37.0%),Footnote 2 which is similar to the percentage of women in the APSA membership in February 2020 (36.8%; APSA 2020). Based on current reviewer profiles, we have also made modest progress in diversifying the geographical distribution of reviewers. We are inviting more reviewers from regions outside of North America and Europe than did any previous team.

Table 1. Reviewer Characteristics and Review Statistics for New Submissions with Initial Decisions

Note: Gender coding is approximate and based on probabilistic coding of first names using Genderize API, which uses self-identified gender from social media to predict gender, and 1.27% of reviewers do not have a coded gender. Gender and region as percentage of invited may not sum to 100% due to missing values.

Before we discuss other aspects of the review process, we want to answer a question that is often asked by junior scholars about how to become a reviewer for the APSR. Many benefits accrue to scholars from participating in the review process (Miller et al. Reference Miller, Pevehouse, Rogowski, Tingley and Wilson2013, 120). If you are interested in reviewing for the APSR, we suggest that you create an account in our submission system, taking care to add your areas of specialization in your profile. This will increase the likelihood that the system recommends you for manuscripts in your areas of interest and expertise. If you already have an account, you should make sure your email, rank, affiliation, and specialization are all up to date.

Who Completes Reviews for the APSR and What Does Our Process Look Like?

Though concerns about reviewer fatigue and burden at the APSR and in political science predate the pandemic (Brannon et al. Reference Brannon, Breuning, Gross, Widmeier and Backstrom2016; Breuning et al. Reference Breuning, Backstrom, Brannon, Gross and Widmeier2015; Djupe Reference Djupe2015), the pandemic certainly laid bare and exacerbated challenges facing the peer-review system (Flaherty Reference Flaherty2022; Warner Reference Warner2022). Like others, we have also noticed that it has become more difficult to secure timely reviews of new manuscripts as the pandemic progressed. According to the summary statistics in Table 1, our average number of completed reviews for new manuscripts with an initial decision is very similar to that of previous teams. However, the percentage of our invitations that lead to completed reviews (47.24%) is lower. Several factors contribute to that overall percentage, including instances when editors are able to make a decision with fewer completed reviews than those initially invited. We also report the completion rate by likely gender and region of residence. Although others have noted (based on 2013 data) that men and women accept and complete APSR review invitations at a similar rate (Breuning et al. Reference Breuning, Backstrom, Brannon, Gross and Widmeier2015, 597), here we employ a slightly different definition that focuses on the percentage of invitations that have a completed review by the time a decision is made. By this metric, the gap in completion rates between men and women has increased, particularly during our tenure. Although the reasons that women complete a lower percentage of reviews are likely complex, the ongoing pandemic no doubt looms large.

Furthermore, we invited a larger average number of reviewers per manuscript than the Mannheim team did, but our rate is similar to that of the North Texas team. In Table 1, we also report the median number of days from submission to being under review or having an initial decision. Again, our times to “under review” (9 days) and to an initial decision to reject (68 days) or to invite a revision (86 days) are most similar to that of the North Texas team. Figure 1 illustrates an inverse relationship between the number of invited reviewers and the time to an initial decision after review. When editors initially invite more reviewers per manuscript or add new reviewers when original reviewers have not completed their reviews in a timely fashion, average times to decision tend to be shorter.

Figure 1. APSR Invitations, Completed Reviews, and Days under Review, 2008–2022

As suggested at the outset, reviewers often, but not always, mention specific personal circumstances that explain why they are declining a review invitation or why their completed review is delayed. To capture potential shifts in reviewer behavior due to the pandemic, we coded whether reviewers mentioned certain keywords in their private comments to the editors. In Table 2, we report the percentage of private comments to the editors that include one or more of the keywords related to illness, family, or school.Footnote 3 Although these are simple measures, they are applied across all reviewer comments to all editorial teams and should therefore be consistent, even if they underestimate reasons for reviewer behavior. Mentions of family are common and steady, particularly among those who decline reviews, including during the last two years. Mention of illness is generally more often cited as a reason to decline an invitation to review than to explain a late review and was mentioned more in the last two years than before.

Table 2. Percentage of Confidential Comments to the Editors that Include Keywords

Note: Gender coding is approximate and based on probabilistic coding of first names using Genderize API, which uses self-identified gender from social media to predict gender, and 1.27% of reviewers do not have a coded gender. Keywords for Ill, Family, and School are percentage of private comments to the editors from reviewers (when they decline a review request or complete a review) that contain at least one of the keyword stems or combination of keyword stems for that category.

The most dramatic shift involves references to schooling (i.e., children being out of school, daycares or preschools closed, etc.), which have increased in the last two years. The most extreme “shutdowns” of daycares and schools during the pandemic began during the final months of the Mannheim team’s leadership and may also account for the increase in references to school-related keywords by those who declined invitations to review. Although these statistics likely underestimate the overall impact of the pandemic, particularly the effect of school and daycare closures and increased caregiver responsibilities, they are consistent with recent descriptions of how the pandemic is disrupting our community (e.g., Flaherty Reference Flaherty2020; Reference Flaherty2022; Kim and Patterson Reference Kim and Patterson2022; Shalaby, Allam, and Buttorff Reference Shalaby, Allam and Buttorff2021).

In spite of these many disruptions and widespread reviewer fatigue, we are pleased to report that our colleagues have stepped up to review our manuscripts in record numbers, allowing us to report fewer days required to get pieces under review, to reject after review, or to offer a revise and resubmit. Admittedly, this was accomplished by calling on larger numbers of our colleagues as reviewers for each piece we consider. We are grateful to our reviewers for making this possible, especially under the circumstances of the last few years. It is obvious that family and health pressures have presented significant challenges throughout the review process. Furthermore, although there is much that we do not know, we are making some progress toward diversifying our reviewer pool. Additional data-gathering efforts would allow us to say more about the subfield, rank, methodological and epistemological orientation, and the races, ethnicities, gender and gender identities, and sexual orientations of those who review for the APSR. In the meantime, however, we want to reiterate our gratitude to those who have supported the journal with their reviews over the last two years.

Footnotes

1 The Lead Editor and some Associate Editors were affiliated with University of Mannheim, whereas other Associate Editors were affiliated with the London School of Economics and Political Science, University College London, and University of Cologne.

2 Gender coding is approximate and based on probabilistic coding of first names using Genderize API, which uses self-identified gender from social media to predict gender.

3 Ill includes *health*, sick*, ill*, hospita*, corona*, covid*. Family includes words commonly used to describe children, parents, partners, siblings, maternity, paternity, as well as elder* and *care. School includes *school, day* and *care, child* and *care, and multiple words for children with home.

References

APSA. 2016. “APSA Announces New Editorial Team for the American Political Science Review.” PS: Political Science & Politics 49 (4): 906–07.Google Scholar
APSA. 2020. “APSA membership dashboard.” February 2020. https://www.apsanet.org/RESOURCES/Data-on-the-Profession/Dashboard/Membership.Google Scholar
Brannon, Jeremy, Breuning, Marijke, Gross, Benjamin Isaak, Widmeier, Michael, and Backstrom, Jeremy. 2016. “Overburdened Reviewers and the Future of Peer Review.” European Science Editing 42 (2): 42.Google Scholar
Breuning, Marijke, Backstrom, Jeremy, Brannon, Jeremy, Gross, Benjamin Isaak, and Widmeier, Michael. 2015. “Reviewer Fatigue? Why Scholars Decline to Review Their Peers’ Work.” PS: Political Science & Politics 48 (4): 595600.Google Scholar
Djupe, Paul A. 2015. “Peer Reviewing in Political Science: New Survey Results.” PS: Political Science & Politics 48 (2): 346–52.Google Scholar
Flaherty, Colleen. 2020. “No Room of One’s Own: Early Journal Submission Data Suggest COVID-19 Is Tanking Women’s Research Productivity.” Inside Higher Ed, April 21. https://www.insidehighered.com/news/2020/04/21/early-journal-submission-data-suggest-covid-19-tanking-womens-research-productivity.Google Scholar
Flaherty, Colleen. 2022. “The Peer-Review Crisis.” Inside Higher Ed, June 13. https://www.insidehighered.com/news/2022/06/13/peer-review-crisis-creates-problems-journals-and-scholars.Google Scholar
Kim, Eunji, and Patterson, Shawn. 2022. “The Pandemic and Gender Inequality in Academia.” PS: Political Science & Politics 55 (1): 109–16.Google Scholar
Miller, Beth, Pevehouse, Jon, Rogowski, Ron, Tingley, Dustin, and Wilson, Rick. 2013. “How to Be a Peer Reviewer: A Guide for Recent and Soon-to-Be PhDs.” PS: Political Science & Politics 46 (1): 120–23.Google Scholar
Shalaby, Marwa, Allam, Nermin, and Buttorff, Gail J.. 2021. “Leveling the Field: Gender Inequity in Academia during COVID-19.” PS: Political Science & Politics 54 (4): 661–67.Google Scholar
Warner, John. 2022. “The Professors Who Refuse to Do the ‘Free Work’ Their Older Colleagues Did for Years.” Slate, July 27. https://slate.com/human-interest/2022/07/humanities-academics-working-conditions-state-of-academic-labor.html.Google Scholar
Figure 0

Table 1. Reviewer Characteristics and Review Statistics for New Submissions with Initial Decisions

Figure 1

Figure 1. APSR Invitations, Completed Reviews, and Days under Review, 2008–2022

Figure 2

Table 2. Percentage of Confidential Comments to the Editors that Include Keywords

Submit a response

Comments

No Comments have been published for this article.