Weathering the Current Climate for Open Review

¶ 1 Leave a comment on paragraph 1 2 While the parameters listed above are intended to ensure that all open peer reviews adopt a rigorous and structured — if also non-standardized — set of norms for participation that guide how a given community goes about its tasks, none of these ideas will necessarily ensure that an open review process will be viewed by skeptics as a viable alternative to a closed system. While we do not see the purpose of this document as offering strategies for combating skeptics, we do recognize the importance of having open peer review count as a legitimate mode of evaluation for scholarly, pedagogical and other forms of academic work. This requires striking a balance between meticulously documenting the winnowing process for moving (and not moving) a work through publication stages (as well as for defining and refining the peer review process itself) and articulating a vision for peer review that recognizes the processes of critical engagement, debate, discussion and dialog as central to producing better humanities-based scholarship (a set of ideas widely pursued in other aspects of academic life, from conferences to workshops). In other words, proponents of open peer review must move beyond claims that such processes are either just as rigorous as traditional blind review or intended to accomplish different results, and instead must highlight how open peer review adopts and enhances the best aspects of humanities based scholarly practices.

¶ 2 Leave a comment on paragraph 2 2 The “openness” of open review can also prove daunting when it comes to evaluating scholarship for the purposes of tenure and promotion, where even the slightest hint of criticism can sink a scholar’s chances. While unrelated to the actual practices of open review, the inflation in standards for evaluating scholars (it is a cliché that almost every tenure review letter ranks candidates as amongst the top 5% of scholars in their field — and that any deviation from this will be regarded as a “red flag” by review committees) can have a chilling effect on participation in a process that makes critique and disagreement visible. Personnel review processes will need to acknowledge that what John Guillory has referred to as the immanent scene of judgement[28] is one of debate and disagreement, and that this dissent is a sign of engagement in a field — that the more engaged an open review community is with a member’s work, the greater its significance. While open review communities must adopt clear protocols for distinguishing between “engagement” and “rejection” in these environments (documenting the winnowing process is once again key), concerns over how open review will be perceived by tenure and promotion committees suggest that the work currently being done to value open peer review must be made part of a larger process of reforming academic standards for evaluation. Therefore, proponents of open peer review must take a holistic view and be active in other areas of academic life, situating arguments about “openness” when it comes to recognizing “peers” and performing “review” in relation to broader questions about the future of scholarly discourse and the roles of the 21st century academic. The advisory board felt strongly that the white paper acknowledge the context in which open review currently occurs and recognize that discussions about open review should ideally be part of a broader conversation (one beyond the scope of this project) about reforming other academic practices and procedures.

¶ 3 Leave a comment on paragraph 3 0 While technology can be a tool in helping to accomplish many of these philosophical and pragmatic goals, this section has highlighted some of the core concerns and desires that must drive technological innovations. In the next section, we discuss technological systems that can be deployed or designed to accomplish these ends.

[28] See John Guillory, “Evaluating Scholarship in the Humanities: Principles and Procedures.” ADE Bulletin 137 (Spring 2005): 18–33.

Page 12

Comments

0 Comments on the whole Page

2 Comments on paragraph 1

Adeline Koh 20 June 2012 at 4:43 pm

How can champions of open peer review argue for that, specifically (that open peer review ‘adopts and enhances the best aspects of humanities based scholarly practices’)? Some examples would be helpful.

Reply to Adeline Koh
1. Jack Dougherty 21 June 2012 at 2:36 am
  
  Yes, we definitely need more evidence. In the “Conclusions:What We Learned” section of Writing History in the Digital Age, we offered some descriptive data and illustrative excerpts on the 1000+ comments posted on the site to make our case for how and why open peer review improved the quality of our writing and intellectual coherence of the manuscript. By making drafts and commentary visible, we can trace how different author-reader exchanges influenced the final manuscript. Try doing that with traditional closed review.
  
  Reply to Jack Dougherty

2 Comments on paragraph 2

Jason Mittell 18 June 2012 at 8:21 am

While this is a larger question, acknowledging the different constraints outside of the U.S. tenure/promotion system might be useful. In my own conversations this year in Europe, I’ve heard from nearly every European scholar that they could never do the type of open review I’m doing with my book, given the strict measures by which their universities & government overseers measure impact & publication benchmarks. Of course, the irony is that by publishing openly, non-American scholars could actually get their work more broadly read & increase their actual impact, but the systems are not set-up for such ironies.

Reply to Jason Mittell
rogerwhitson 20 June 2012 at 2:55 pm

The other difficult reality here is the fact that current scholars interested in OA publications will probably have to do double the work of traditional scholars – since they will have to prove they have the chops for traditional publications AND experiment with OA.

Reply to rogerwhitson