anonymity

Kathleen Fitzpatrick

27 August 2009

¶ 1 Leave a comment on paragraph 1 0 Perhaps this caution about open review was an attempt to avoid throwing out the baby of quality control with the bathwater of anonymity. The editors of Atmospheric Chemistry and Physics, however, presented evidence (based on their two-stage review process) that open review significantly increases the quality of articles a journal publishes:

¶ 2 Leave a comment on paragraph 2 0 Our statistics confirm that collaborative peer review facilitates and enhances quality assurance. The journal has a relatively low overall rejection rate of less than 20%, but only three years after its launch the ISI journal impact factor ranked Atmospheric Chemistry and Physics twelfth out of 169 journals in ‘Meteorology and Atmospheric Sciences’ and ‘Environmental Sciences’.

¶ 3 Leave a comment on paragraph 3 0 These numbers support the idea that public peer review and interactive discussion deter authors from submitting low-quality manuscripts, and thus relieve editors and reviewers from spending too much time on deficient submissions. (Koop and Pöschl)[1.20]

¶ 4 Leave a comment on paragraph 4 1 Evidence such as this begins to suggest that traditional closed, anonymous peer review processes and quality control aren’t quite as related as we often assume. In fact, it’s arguable that the primary results of a closed peer review process are negative. As Fiona Godlee has argued, anonymous review “has the effect of giving reviewers power without responsibility” (65), since reviewers are freed by the veil of anonymity to behave, in some instances, in a variety of unprofessional ways, ranging from the relatively innocuous unleashing of snark on an undeserving target to several utterly unacceptable forms of academic dishonesty. Such behaviors are of course not the norm, but they occur frequently enough that they should give us pause.[1.21] On the other side of the review process, of course, are the authors, ostensibly equal participants in a conversation about their work. The anonymous peer review process, however, effectively closes the author out of the main chronology of the conversation, which instead becomes a backchannel discussion between the reviewer and the editor. As such, the author is hindered in her ability to learn from the review process even if she is given a copy of the reviewer’s comments, as there is no forum in which she can respond to those comments in kind. By the time the comments arrive, generally speaking, the decision about the manuscript’s fate has been decided, the conversation is over, and the author is too often left with no one listening.[1.22]

¶ 5 Leave a comment on paragraph 5 0 Reviewer anonymity, however, has long enough been a part of the process that many academics express alarm at the thought of that protection being removed, insisting that their anonymity as reviewers is necessary in order for them to have the freedom to say that a manuscript should not be published. Such a position would certainly be justifiable if the primary purpose of peer review is quality control, and if it can be demonstrated that the process is both scrupulous and effective. However, as Douglas Peters and Stephen Ceci famously uncovered in their 1982 article, “Peer Review Practices of Psychological Journals,” reviewer reliability is not at all a given. In their experiment, Peters and Ceci selected one article from each of 12 journals in the field, published between 18 and 32 months previously, and resubmitted the article to the same journal, with some minor modifications: they changed the authors’ names (but, significantly, not their sexes); they created new institutional affiliations for their authors (notably replacing “high-status” institutions with low- or no-status ones); and they slightly altered the phrasing, but not the meaning, of the articles’ opening paragraphs. Only three of these 12 articles were discovered by either the editors or the reviewers to have been previously published, and of the 9 that went undiscovered, 8 were rejected, most on the grounds of “serious methodological flaws” (Peters and Ceci 202).[1.23] Their conclusion is that one of two things has occurred: either the initial reviewers who approved the articles as originally published were incompetent – which seems unlikely – or “systematic bias was operating to produce the discrepant reviews” (202).[1.24]

¶ 6 Leave a comment on paragraph 6 3 One of the correctives that has been suggested in response to evidence of such “systematic bias,” as well as some of the other more egregious abuses of peer review, is a further layer of anonymity: blind review, in which the identity of the author is cloaked, as well as that of the reviewer. Blind review is imagined by many (including Shatz) to be a mode of avoiding certain forms of reviewer bias, for instance, preventing the continuation of an “old boys network” that excludes the work of women, or ensuring that personal grudges play no role in the review process. However, the effectiveness of blind review in genuinely masking authorial identity has been subject to some critical scrutiny, by authors who suggest, for instance, that blinding is futile: “Alas, anyone capable of evaluating research in a given specialty generally knows that specialty sufficiently to identify the probable author of the manuscript under review” (Guédon and Siemens 18).[1.25] In many cases, in fact, the author has previously presented and discussed the material in public, whether via informal networks or in more formal conference settings. Moreover, blind review can only correct for ad hominem bias on the part of reviewers, and cannot compensate for the reviewer who operates within a cloud of intellectual bias, dismissing any arguments or conclusions that disagree with his or her own.[1.26]

¶ 7 Leave a comment on paragraph 7 0 It’s also necessary to note that neither reviewer nor author identity are hidden from the editor, who may have his or her own biases; as Godlee notes, “[e]vidence suggests that editors may be susceptible to the pull of prestige,” citing the results presented by Zuckerman and Merton, which suggest that “if a paper had higher-ranking authors, editors were more likely to come to a decision without sending it out for peer review” (Godlee 73). Moreover, the editor’s selection of reviewers for a manuscript may be influenced by the author’s identity, and the editor’s evaluation of the reviewers’ reports may similarly be affected by the differing levels of prestige of reviewer and author.

¶ 8 Leave a comment on paragraph 8 1 Finally, one cannot help but wonder about the logic of correcting for the abuses of anonymity on one side of a conversation by establishing anonymity on the other, creating further barriers between peers rather than encouraging open, effective, productive discussion of intellectual issues. As Drummond Rennie argued in 1994, “We have an ample history to tell us that justice is ill served by secrecy. And so it is with peer review. Two or three hundred years ago, scientific papers and letters were often anonymous. We now regard that as quaint and primitive. I hope that in 20 years, that’s exactly how we will look on our present system of peer review” (Commentary 1143).