Why Open Review?

¶ 1 Leave a comment on paragraph 1 0 In exploring the possibilities for what we originally thought of as “peer-to-peer review” — review practices and tools that would enable the direct communication among a network of peers around scholarly publications — we began to focus on the ways that opening up review practices might further some crucial values and goals in the humanities. We aspire, as scholars, to engage our students, our colleagues, and a range of broader publics in exploring aspects of our complex histories and cultures. We also seek to model the emergence of critical thinking and intellectual pursuit through Socratic forms of inquiry, stressing the essential role played by discussion and debate in knowledge formation.

¶ 2 Leave a comment on paragraph 2 3 Much of this work is already done in the open; we present work at conferences, discuss it in workshops, share it with our colleagues, and so forth. But typically our publication processes have operated off-stage. In recent years, however, more and more important new work in the humanities is simply being published online, without the presumed benefits of pre-publication review; scholars are keeping individual blogs and participating in group blogs, and more and more conference presentations and working papers are being posted online. Further, many scholars are finding that the feedback they receive through these formats is as substantive and productive as traditional peer reviews have been. Given the ways that open practices are enabling scholars’ work to develop, it seems increasingly important for the humanities to take account of such practices, and to explore the possibilities that they present for our fields more broadly.

¶ 3 Leave a comment on paragraph 3 11 What do we mean by our use of “open” in thinking about open review, however? Must everything be open to everyone, or are there degrees of openness that might be useful to different communities of practice at different times? Might, for instance, a frank discussion among a defined cluster of scholars be particularly important at certain times, while a discussion opened to broader publics would be crucial at others? Must all reviews be submitted under reviewers’ real names, or are there situations in which some degree of anonymity or pseudonymity remains useful? Moreover, are these two forms of openness — openness of access to the review process and openness of reviewer identity — related, or are they separable? We do not want to conflate distinct functions or formations in the scholarly communication process, nor do we want to foreclose choices that particular communities of practice might find beneficial. At root, we value openness for the ways that it can enable the members of a community of practice to perform and develop in collaboration with one another, rather than to assume or prescribe pre-existing standards for contribution.

¶ 4 Leave a comment on paragraph 4 4 Open processes such as those we explore in this document can go badly, of course, or they can go well. Open processes require careful cultivation within a community, as well as careful attention to the heterogeneity of that community. These processes, however, turn the “problem” of a diverse community into a value, creating a self-consciousness within the community about its presuppositions and assumptions, and facilitating the development of a range of new perspectives and voices.

Page 7

wpatty 26 October 2012 at 5:50 pm

It sounds like there is already something of a framework for supporting open review. I wonder if there is a way to collect a sample of the feedback mentioned in this paragraph to analyze how it can be just as useful as traditional peer review.

Reply to wpatty
Matthew T. Keough 27 October 2012 at 6:16 am

This raises an interesting point. Since there is more diverse ways of publishing do we need have a review process for both the text and the means that people use to find that text? There is a ton of material available online. Some is valuable and a lot of it is not. What if a valuable work that does not show up in popular searches? I would say this is a problem and people should also be reviewing this process along with the content of the digital publication itself.

Reply to Matthew T. Keough
roshna kapadia 29 October 2012 at 8:42 pm

I disagree that open review is already being practiced, because, typically, conference and workshops tend to attract–and be attended by–similar-field academic peers. The beauty of open review is that people from all disciplines can offer their thoughts. Great improvements can result from scientists’ unique insight for humanities scholars (and vice versa). This sort of academic intermingling has been stymied by the requirement of physical attendance at a conference or workshop. But in a digital environment, the process is easier.

Reply to roshna kapadia

11 Comments on paragraph 3

Jason Mittell 18 June 2012 at 7:38 am

Another variable you might add here is “publicness” – to what degree might a review process be open to participation but not publicly accessible (like an anonymous survey)? Or reviews posted publicly but not open to participation outside of invitations or a closed community?

Reply to Jason Mittell
dianeh 18 June 2012 at 12:31 pm

We have spent some time in Harley et al. 2010 ( Assessing the Future Landscape of Scholarly Communication: An Exploration of Faculty Values and Needs in Seven Disciplines), Harley and Acord, 2011, http://escholarship.org/uc/item/1xv148c8 and in two relatively new publications, discussing issues around sharing pre-publication work in a variety of disciplines, and note the importance of disciplinary culture and also of credit, time and personality in dictating who shares what, with whom, and when. As they say, it’s complicated. I think the passage below from a recent New Media and Society (NMS) article (Acord and Harley, Credit, time, and personality. In Press, and which was posted in November for open peer review and has undergone final revisions; it attracted almost no open comments) sums up our thoughts on open peer review. The whole paper gives that passage better context of course. With regards to sharing pre-publication work generally, we note: As with everyday communication practices more generally (cf. Goodwin, 2000), individuals design scholarly communication practices to maximize impact with a select ‘target’ audience. As scholars formulate, develop, edit, and fact-check their work-in-progress, they gradually share their work with wider and wider circles of trusted, targeted individuals….(W)hile the web has extended the reach of these types of sharing, the functions they serve developmentally have not changed radically. As one historian noted, ‘It’s really not substantially different than what’s been in practice for several hundred years…[except] it’s faster and it’s more global’…. “Taken together, guarded prepublication sharing practices function as a safety net for scholars to not only improve the work and avoid ‘making fools of themselves’, but also to stake a claim on ideas and maintain a visibility in their research areas. Our work suggests that scholars seek out informal peer review in a highly strategic manner based on social variables and disciplinary values. That is, in deciding when and where to share their work-in-progress, scholars make decisions based upon their discipline’s culture and degree of trust (of the interlocutor), comfort (how well they feel their work is developed), and audience (who needs to be aware of their work and how are they best reached).”If open commentary experiments proliferate, it will be important to assess who is offering comments (i.e., what portion of such open comments come from ‘friends’) in these venues, and whether the overall impact and costs –to authors, editors, and readers–exceed the normal levels of traditional informal and formal peer review. This report is a contribution answering those questions.

Reply to dianeh
1. Sandy Thatcher 22 July 2012 at 10:17 pm
  
  As a practical matter, it would likely be very difficult to identify which commenters were “friends” (as distinguished, say, from fellow members of the author’s department or former dissertation advisers).
  
  Reply to Sandy Thatcher
Sarah Werner 19 June 2012 at 12:44 am

I’m intrigued by Jason’s suggestion of degrees of publicness. I’d add the possibility of comments that might be public to the editor and author, but not public to viewers (one of the factors that I’ve encountered running an open peer review is the fear that reviewers have of seeming stupid in front of their peers and the ways this inhibits their willingness to comment publicly).

Reply to Sarah Werner
1. Jason M. Kelly 21 June 2012 at 11:32 am
  
  In the open peer review system that we’re designing for Drupal, we have set the default to “open.” However, we plan to give editors the ability to keep the identities of some reviewers anonymous if they wish. However, keeping their comments hidden seems to defeat much of the purpose of open peer review, it seems to me. Open peer review is not simply about product; it’s about process. Discussions and disagreements “in the margins” are every bit as important to scholarship as the text. Perhaps this is mentioned later in the document, but it might be worth mentioning here.
  
  Reply to Jason M. Kelly
  1. Sarah Werner 22 June 2012 at 3:04 am
    
    A clarification: I meant there could be an option of comments appearing anonymously to the public but revealed to author and editor(s). I do agree that even that seems a bit odd, in that it works against the purpose of openness, but it would counter the anxiety I’ve encountered from potential reviewers about how idiotic they might appear in front of others. Random attributions that are consistent could be assigned: rather than appearing as “Sarah Werner” my comments could be attributed to “ABC” consistently, which would at least give a sense of the individual biases of my perspective, even if my name isn’t attached. Not that I’m biased, of course…
    
    Reply to Sarah Werner
rogerwhitson 20 June 2012 at 1:23 pm

Another element of “openness” is clarity of writing. I know, for example, Alice Bell argues that openness can also be “simply a new way to rub scientists’ cleverness in people’s faces, letting more of them feel lost and stupid in the face of such impressive expertise.” (http://www.timeshighereducation.co.uk/story.asp?c=1&sectioncode=26&storycode=419684).

Reply to rogerwhitson
Adeline Koh 20 June 2012 at 1:43 pm

I like Roger’s comment here–this is an important dimension to open scholarship.

Reply to Adeline Koh
Jack Dougherty 20 June 2012 at 9:31 pm

What bothers me about this paragraph is that it challenges “openness” without raising any questions about its opposite: “secrecy.” Why aren’t we interrogating secrecy within the academe to the same degree? When Kristen Nawrotzki and I designed our open peer review process for Writing History in the Digital Age, we eventually found ourselves taking a stance of “transparency by default” in the writing and editorial process, except in cases where was a compelling case for privacy (such as our decision to inform contributors by private email when their submissions were not selected to advance to the final manuscript). We write about this in “Conclusions: What We Learned,” under the subheading titled, “Did the benefits of publishing on the web, with open peer review, outweigh its risks?”

Reply to Jack Dougherty
1. Jason Mittell 21 June 2012 at 9:10 am
  
  I second this point – framing “open” as a default that can be overridden when needed goes a long way toward decentering the status quo model where “blindness” is often accepted as inherently valued simply because it is the default.
  
  Reply to Jason Mittell
Xenia van Edig 3 July 2012 at 5:18 am

I work for Copernicus Publications, an Open Access Publisher based in Germany. We already have 11 years of experience with Public Peer Review (http://publications.copernicus.org/services/public_peer_review.html). We experienced that people often mix up the terms “public peer review” (which we regard as a more transparent form of the peer review process) and “open peer review” which mostly is seen equal to eponymous referees. In our approach, we allow each referee to choose whether he or she wants to be called by name or as “referee” in a public discussion.

Reply to Xenia van Edig

4 Comments on paragraph 4

Jason Mittell 18 June 2012 at 7:35 am

I would add an addendum like this to the first sentence: “—of course, traditional closed review processes can also succeed and fail in a wide range of often unacknowledged ways.” And maybe cite Planned Obsolescence on this? It’s important to remind people that the status quo isn’t a perfect machine, it’s just the one whose faults we’ve come to accept as unavoidable or palatable.

Reply to Jason Mittell
1. Dorothea Salo 18 June 2012 at 7:10 pm
  
  Wholeheartedly seconded. It’s hard to move beyond the status quo until it’s clear that it’s broken in important ways.
  
  Reply to Dorothea Salo
Dayna Goldstein 18 June 2012 at 9:00 pm

This paragraph is facile and doesn’t address the actual objections it alludes too.

Reply to Dayna Goldstein
geno12 29 October 2012 at 6:18 pm

I love the notion of open peer review. The open review seems to encourage a reinvigorated collegial, scholarly communication, as opposed to the blind reviews which are often done by professors for payment if the journal is large enough. I do see areas for concern in this process, however. I am not sure that all professors/reviewers in this current Capitalistic society who reside in an arguably competitive milieu would operate perfectly within a system based upon something quite akin to a process based upon generalized reciprocity. Are there safety nets, legal or otherwise, for such situations?

Reply to geno12