Who Else Is Exploring These Issues?

¶ 1 Leave a comment on paragraph 1 3 In addition to the web forum held alongside Nature’s 2006 open review experiment,[16] much has recently been written about open review practices and related issues. Diane Harley and the Center for Studies in Higher Education at UC Berkeley have published an extensive report on the uses of peer review in the academic publishing and promotion processes; while this report does not focus on open review, it includes an assessment of the workload that traditional peer review practices place on scholars, suggesting that new alternatives could uphold quality control and support better scholarly communication while lightening the burden of peer review under which scholars labor.[17] Other publications have focused more explicitly on open review: Kathleen Fitzpatrick has written about the history and future of peer review, arguing for the development of new open review practices, in Planned Obsolescence.[18] Fitzpatrick has also co-written an article with Katherine Rowe exploring the process and results of the Shakespeare Quarterly open review experiment,[19] which has been covered by the Chronicle of Higher Education, the New York Times, and University Affairs/Affaires universitaires.[20] More recently, the journal postmedieval published an open web forum to discuss its own experiment with open review,[21] and Jack Dougherty, Kristen Nawrotzki, Charlotte Rochez, and Timothy Burke published a thoughtful conclusion to their online review process for Writing History in the Digital Age.[22] Discussions such as these are not universally laudatory; their frank evaluations of open review processes identify both strengths and challenges in ways intended to provoke careful deliberation by other editorial and collaborative groups considering conducting their own experiments with open review.

¶ 2 Leave a comment on paragraph 2 2 Additionally, a number of efforts related to open peer review are underway, including projects such as the Open Annotation Collaboration (OAC), an effort to create technical standards and tools to enable the creation of web annotations that can be be shared in multiple contexts, and the Open Researcher and Contributor ID project (ORCID), which is working to develop a standard for the unique identification of scholarly authors; both projects intersect with work being done by those seeking alternative means of accounting for the impact of scholarly research, such as the altmetrics group.[23] Projects such as Hypothesis seek to bring these kinds of information together, linking open web annotation with reputation management, in ways that might be useful to open review.[24] PressForward will provide tools through which groups of scholars can capture the best work being published across the open web, through a combination of crowd-sourcing and editorial management.[25] Social reading platforms like BookGlutton and SocialBook connect texts and readers in flexible, community-oriented discussions.[26] And sites such as Academia.edu are working to create communities of scholars sharing their work with one another.[27] In addition, our work with open review bears much in common with that of other groups seeking to introduce other forms of openness into scholarly research such as the open access and open data movements. The recommendations that we develop below presuppose that open review projects will learn from and collaborate with projects such as these in the coming years.

[16] Nature’s peer review debate <http://www.nature.com/nature/peerreview/debate/index.html>.

[17] Diane Harley, et al., “Peer Review in Academic Promotion and Publishing.”

[18] Fitzpatrick, Planned Obsolescence, pp. 15-49.

[19] Kathleen Fitzpatrick and Katherine Rowe, “Keywords for Open Review,” Logos 21.3-4 (2010): pp 133-41.

[20] Jennifer Howard, “Leading Humanities Journal Debuts ‘Open’ Peer Review, and Likes It,” Chronicle of Higher Education (26 July 2010) <http://chronicle.com/article/Leading-Humanities-Journal/123696/>; Patricia Cohen, “Scholars Test Web Alternative to Peer Review,” New York Times (24 August 2010) <http://www.nytimes.com/2010/08/24/arts/24peer.html>; Rosanna Tamburri, “Opening up peer review,” University Affairs/Affaires universitaires (5 March 2012) <http://www.universityaffairs.ca/opening-up-peer-review.aspx>

[21] postmedieval Forum, “Forum II: The State(s) of Review” (March 2012) <http://postmedieval-forum.com/forums/forum-ii-states-of-review/>

[22] “Conclusions: What We Learned from Writing History in the Digital Age (Spring 2012),” Writing History in the Digital Age. <http://writinghistory.trincoll.edu/conclusions-2012-spring/>

[23] Open Annotation Collaboration <http://www.openannotation.org/>; Open Researcher and Contributor ID <http://about.orcid.org/>; altmetrics <http://altmetrics.org/manifesto/>.

[24] Hypothesis <http://hypothes.is/>.

[25] PressForward <http://pressforward.org/>.

[26] BookGlutton <http://www.bookglutton.com/>; SocialBook <http://livemargin.com>.

[27] Academia.edu <http://academia.edu>.

Page 9

Comments

2 Comments on the whole Page

rogerwhitson 20 June 2012 at 1:42 pm

Can I leave a little bit of self-promotion? Brian Croxall and I delivered a keynote at the University of Florida last April titled “Theses on the Open Humanities,” which also discusses these issues. We archived the talk, our slides, and links to videos here: http://www.rogerwhitson.net/?p=1693

Reply to rogerwhitson
pfrishauf 24 June 2012 at 4:18 pm

Frankly, your “open” process does not seem very open!
I have written extensively about open peer review, as has my colleague, Richard Smith, former editor of BMJ. See:

http://www.jopm.org/opinion/commentary/2009/10/21/reputation-systems-a-new-vision-for-publishing-and-peer-review/

And

http://e-patients.net/archives/2010/08/a-troubled-trifecta-peer-review-academia-tenure.html

Reply to pfrishauf

3 Comments on paragraph 1

dianeh 18 June 2012 at 12:30 pm

If read beyond the executive summary, the Harley and Acord 2011 publication http://escholarship.org/uc/item/1xv148c8, makes clear how peer review is a multi-faceted activity and every scholar is bound to its many guises. We also note the great expense peer review, in all of its forms, places on the academy and publishers. We did in fact do a review of open peer review on pages 45-48 in the 2011 publication cited above. This section was embedded in a larger chapter on reviewing new models (in a variety of disciplines) that attempt to modify or upend traditional peer review processes, and provide trusted filters to an ever expanding literature in all fields. We noted there and elsewhere a number of reasons we think open peer review (either pre-publication or “open commentary post-publication,” such as PLoS ONE) will not be embraced by the majority of scholars. Most importantly, we do not think open peer review has even a small chance of ” lightening the burden of peer review under which scholars labor.” Just the opposite. We think it will add an immense burden, and will be eschewed by most scholars who are already overwhelmed with requests for reviewing (be it for publication, advancement dossiers, grad student work, grant proposals, etc.) and are looking to more filters, not fewer, for the avalanche of literature they must keep up with. Peer reviewed publications with prestigious imprimaturs offer those filters for most.We also emphasize the potential for increased editorial costs to publishers who must manage and sift the open commentary. We suspect this might be particularly acute in the humanities. We would argue that issues of time and money cannot be ignored given the economic complexities of the current scholarly communication landscape (an additional theme in the 2011 paper). That being said, open peer review of the sort demonstrated at MediaCommons may succeed in tightly-knit communities with affinities to new media, and where authors can invest time into raising visibility of their work.

Reply to dianeh
1. Sandy Thatcher 22 July 2012 at 10:42 pm
  
  And that is the reason that incentives need to be built into the system, such as Kathleen Fitzpatrick discusses in “Planned Obsolescence.” If authors cannot submit an article to a journal unless they have built up some credit through reviewing, that is one very strong incentive, for example. If there is a way of measuring the utility of reviews and then making this a part of a scholar’s portfolio of academic achievements, that is another kind of incentive. Without such incentives, you are absolutely right that constraints of time and opportunity costs will keep participation rates low.
  
  Reply to Sandy Thatcher
Sarah Werner 19 June 2012 at 2:16 am

Clarification: the postmedieval forum pieces discuss not the postmedieval experiment but a range of responses from others involved in such work, including contributions from Katherine Rowe and from myself on our two SQ open peer reviews.One thing missing nearly entirely from all the reflections that are listed here are perspectives of authors and commenters. The recent conclusion to Writing History in the Digital Age does offer some correction to that, but the voices of contributors, rather than editors, are still largely unheard.

Reply to Sarah Werner

2 Comments on paragraph 2

Katina Rogers 21 June 2012 at 6:23 pm

Because credentialing is such an important aspect of peer review, I wonder if it might be useful to mention Jason Priem et al’s work on http://altmetrics.org/ alongside some of these other projects.

Reply to Katina Rogers
Xenia van Edig 3 July 2012 at 7:11 am

I would like to fortify the link between open access publishing and public peer review. I think this combination offers great opportunities as it makes results of research accessible for everyone and enhances the transparency of the process of peer review.

Reply to Xenia van Edig

1 Pingback or trackback

Tributaries » Blog Archive » Change and Persistence 27 September 2012 at 1:03 pm

[…] but so is the need to share them — and to separate the wheat from the chaff. Open pre- and post-publication peer review is becoming more widespread. CUNY faculty can look elsewhere for […]