credentialing

Kathleen Fitzpatrick

30 August 2009

¶ 1 Leave a comment on paragraph 1 0 If closed peer review processes aren’t serving scholars in their need for feedback and discussion, and if they can’t be wholly relied upon for their quality-control functions — if they appear, at least to some, “quaint and primitive” — why do we cling so ferociously to them? Arguably, the primary purpose that anonymous peer review actually serves today, at least in the humanities,[1.27] is that of institutional warranting, of conveying to college and university administrations that the work their employees are doing is appropriate and well-thought-of in its field, and thus that these employees are deserving of ongoing appointments, tenure, promotions, raises, and so forth. As Rennie has noted, “editorial peer review is seen by investigators and research institutions as a convenient quality control mechanism, for which they usually do not have to pay” (“Editorial Peer Review” 10). This mechanism, on the level of the academic book, has been described by Lindsay Waters as a means for departments to “outsource” the evaluation of junior scholars to university presses; the existence of a book by a reputable press comes to serve as a convenient binary signifier of the quality of that scholar’s work.[1.28]

¶ 2 Leave a comment on paragraph 2 0 To some degree, one must wonder whether using the results of peer review as a shortcut in faculty performance evaluations isn’t misguided in and of itself; much of the most important work published by scholars today is already published in forms that aren’t subject to conventionally-understood modes of peer review, such as edited volumes. Moreover, understanding a successful navigation of peer review as a sufficient sign of quality work is a category error of sorts. As Paul Ginsparg has argued, the mere existence of an author’s peer reviewed publication is insufficient evidence, for hiring and promotion purposes, of the scholar’s level of accomplishment; “otherwise there would be no need to supplement the publication record with detailed letters of recommendation and other measures of importance and influence. On the other hand, the detailed letters and citation analyses would be sufficient for the above purposes, even if applied to a literature that had not undergone that systematic first editorial pass through a peer review system” (9). In other words, our institutional misunderstanding of peer review as a necessary prior indicator of “quality,” rather than as one means among many of assessing quality, dooms us to misunderstand the ways that scholars establish and maintain their reputations within the field.

¶ 3 Leave a comment on paragraph 3 3 Another obvious question to ask is whether peer review as it is currently practiced is really able to support credentialing in the ways we assume. It’s at least imaginable, if as yet untested, that the intellectual purposes that we expect of peer review — most importantly, quality control — could be undermined by this functionalist use of the process’s results, as some extremely well-meaning reviewers, all too aware of the stakes of their evaluations, could unconsciously tend toward a sort of scholarly grade-inflation. And many scholars work with a sense, however vague, that certain publications use peer review as a means of supporting pre-determined ideas held by a field’s in-group, suggesting that the credentialing cart may have been put before the peer-review horse.

¶ 4 Leave a comment on paragraph 4 0 The internet, as Guédon and Siemens indicate in one of this chapter’s epigraphs, has in any event disrupted our ability to draw an association between the fact that a scholarly text has been published and the quality of work it may therefore contain. The result, conventionally, has been the dismissal by many faculty and administrators of all electronically published texts as inferior to those that appear in print, or, where those authority figures are sufficiently forward-looking as to argue for the potential value of electronic publishing, the insistence that the new forms adhere to older models of authorization — and thus the reinforcement of “the way things have always been done” at the expense of experimental modes that might produce new possibilities. Such conservatism shouldn’t come as much surprise, of course; those faculty and administrators who are in the position of performing assessments of the careers of other, usually younger, faculty are of necessity those who have sufficiently benefitted from the current credentialing system as to rise to that position. As Guédon and Siemens suggest, those who hold such privilege will find ways to keep it, preferably without drawing attention to their having done so, precisely by making a virtue — and a besieged one, at that — out of the status quo.

¶ 5 Leave a comment on paragraph 5 1 But I want to emphasize at this point that, while I have spent a great deal of time in this chapter on the various abuses and shortcomings of the peer review process as currently constituted, the core notion behind it — that one’s work as a scholar should be reviewed and assessed by one’s peers — is of course a good one. The problem is in the implementation of that notion as an exercise in gatekeeping, and its subsequent transformation into a means of creating authority in and of itself. Those two shifts have the potential not only to interfere with peers’ ability to communicate directly and fully with one another, but they also create enormous amounts of extra, unproductive work for everyone involved. Scholars pour hours upon hours into peer review each year, time which is not only usually uncompensated but which also results in a product for which reviewers can receive no “credit,” as peer reviews, unlike post-publication reviews, cannot ever themselves be counted among the reviewer’s published work. For all of these reasons, I want to suggest that the time has come for us to consider whether, really, we might all be better served by separating the question of credentialing from the publishing process, by allowing everything through the gate, and by designing a post-publication peer review process that focuses on how a scholarly text should be received rather than whether it should be out there in the first place. What if peer review learned from social software systems such as Slashdot and Digg, and became peer-to-peer review?