Abstract
Peer review of research before publication is both an essential and an integral part of scientific knowledge production. For reputable journals, the peer review process distinguishes knowledge claims in journal articles from those in sources with unknown or varying veracity. The peer review process assures readers that the published work is credible (i.e., conducted in line with prescribed norms of research) and meets a certain threshold with respect to contributions and potential impact. Leading journals are perceived as such not only because the best research is submitted to them but also because of the efforts of the best reviewers and editors in evaluating and, when applicable, developing the initially submitted manuscripts1 to publishable form.
The sustained quality of reviews is critical for journals such as Information Systems Research (ISR). With the number of submissions to ISR growing each year, as well as an explicit policy of encouraging and celebrating inclusive excellence (Sarker 2023), there is a need for more reviewers for the journal (and the discipline, more generally) who have the necessary expertise to evaluate submitted papers, who understand and are attuned to the norms of the different traditions and genres of work submitted, and who know how to craft reviews that ensure the review process supports effective knowledge production.
In this editorial, we draw on the expertise of some of the experienced associate editors (AEs) at ISR2 who represent different research traditions to provide guidance on how ISR reviewers can contribute reviews that AEs and authors are likely to find valuable. The primary audience of this editorial is Ph.D. students and early career scholars who occasionally review for, or seek to review for, ISR and similar journals. Although experienced reviewers likely know most of what we will say in the next few pages, we are hopeful that the editorial can provide a useful recapitulation of characteristics of reviews that are appreciated by ISR editors, irrespective of the reviewers’ experience. Finally, revisiting what reviewers look for in manuscripts can prove helpful for authors submitting papers to journals such as ISR.
Before proceeding, we would like to acknowledge the efforts of editors and editorial board members from various journals who have organized reviewer development workshops (e.g., Rai 2019, Whitley 2023), and reflections on the review process and effective reviews by notable scholars in our discipline (e.g., Lee 1995; Saunders 2005a, b; Straub 2009; Kohli and Straub 2011; Davison 2015; Rai 2016; Leidner et al. 2022); see Table 1. Our editorial does not seek to supplant this accumulated wisdom but seeks to add nuances to the various guidelines that have been offered in the past. We illustrate key points with examples from various research traditions.