One: Peer Review

Kathleen Fitzpatrick

25 August 2009

¶ 1 Leave a comment on paragraph 1 0 In a world where knowledge is being made available at a rate of millions of pages per day, it is comforting to know that some subset of that knowledge or science has been critically examined so that, were we to use it in our thinking for our work, we would be less likely to have wasted our time.

— Ray Spier, “The History of the Peer-Review Process”

¶ 2 Leave a comment on paragraph 2 0 [E]lectronic publishing distinguishes between the phase where documents are placed at the disposal of the public (publishing proper) and the phase where ‘distinctions’ are being attributed. It used to be that being printed was ‘the’ distinction; electronic publishing changes this and leads us to think of the distinction phase completely separately from the publishing phase.
However, doing so changes the means by which distinction is imparted, and imparting distinction is a sure sign of power. In other words, those who now hold that privilege are afraid of losing it (‘gate keepers’) and they will [use] every possible argument to protect it without, if possible, ever mentioning it.

— Jean-Claude Guédon and Raymond Siemens, “The Credibility of Electronic Publishing: Peer Review and Imprint”

¶ 3 Leave a comment on paragraph 3 1 We police ourselves into irrelevance and insignificance.

— Cathy Davidson, “ ‘Research’: How Peer Review Counts and Doesn’t”

¶ 4 Leave a comment on paragraph 4 0 For the last two years, I have worked in collaboration with the Institute for the Future of the Book, my colleague Avi Santo, and a range of prominent scholars in media studies, on MediaCommons, an all-electronic scholarly publishing network. We’ve planned, we’ve blogged, we’ve held meetings, we’ve tested some small scale implementations of the technologies we hope the network will employ, we’ve published a few test-run articles – and in all of the feedback that we’ve received, in all of the conversations we’ve had with scholars both senior and junior, both beginning and established, one question has repeatedly resurfaced: what are you going to do about peer review?

¶ 5 Leave a comment on paragraph 5 0 I’ve suggested elsewhere that peer review threatens to become the axle around which the whole issue of electronic scholarly publishing gets wrapped, like Isadora Duncan’s scarf, choking the life out of many innovative systems before they are fully able to establish themselves.[1.1] This is a flippant response, to be sure; concerns about peer review are quite understandable, given that peer review is in some sense the sine qua non of the academy. We employ it in almost every aspect of the ways that we work, from hiring decisions through tenure and promotion reviews, in both internal and external grant and fellowship competitions, and, of course, in publishing. The work we do as scholars is repeatedly subjected to a series of vetting processes that enable us to indicate that the results of our work have been scrutinized by authorities in the field, and that those results are therefore themselves authoritative.[1.2]

¶ 6 Leave a comment on paragraph 6 0 But as authors including Michael Jensen of the National Academies Press have recently argued, the nature of authority is shifting, and shifting dramatically, in the era of the digital network.[1.3] Scholars in media studies have been exploring such shifts as they affect media production, distribution, and consumption, focusing on the extent to which, for instance, bloggers are decentralizing and may even be displacing the authority structures surrounding traditional journalism, or the ways that a range of phenomena including mashups and fan vids are shifting the previously assumed hierarchies that existed between media producers and media consumers, or the growing tensions in the relationship between consumers, industries, and industry regulators highlighted by file-sharing services and battles with the RIAA. These changes are at the heart of much of the most exciting and influential work in media studies today, including publications such as Siva Vaidhyanathan’s The Anarchist in the Library, Henry Jenkins’s Convergence Culture, and Yochai Benkler’s The Wealth of Networks, projects that have grown out of an understandable interest in the extent to which the means of media production and distribution are undergoing a process of radical democratization in the Web 2.0 era, and a desire to test the limits of that democratization.

¶ 7 Leave a comment on paragraph 7 5 To a surprising extent, however, scholars have resisted exploring a similar sense in which intellectual authority might likewise be shifting in the contemporary world.[1.4] One might see such a resistance manifested in the often unthinking and over-blown academic response to Wikipedia – for instance, one might see reports of the Middlebury College history department’s ban on the use of the online encyclopedia as a research source, and the debate that ensued – which seems to indicate a serious misunderstanding about the value of the project.[1.5] Treating Wikipedia like any other encyclopedia, by consulting only the entries, runs the risk of missing the point entirely; as Bob Stein has suggested, a user has to learn to read Wikipedia differently, given that the real intellectual heart of the project lies on the history and discussion pages, where one can see the controversies inherent in the production of any encyclopedia entry enacted in public, rather than smoothed over into an untroubled conventional wisdom.[1.6] More centralized projects like Citizendium, which seek to add traditional, hierarchical modes of review to a project like Wikipedia,[1.7] overlook the fact that, first, the wiki is in its very architecture a mode of ongoing peer review, and second, that not only the results of that review but the records of its process are available for critical scrutiny. Failing to engage fully with the intellectual merits of a project like Wikipedia, or with the ways in which Wikipedia represents one facet of a far-reaching change in contemporary epistemologies, is a mistake that we academics make at our own peril. As one librarian frames the issue, “Banning a source like Wikipedia (rather than teaching how to use it wisely) simply tells students that the academic world is divorced from real-world practices” (Badke, qtd in Regalado). The production of knowledge is of course the academy’s very reason for being, and if we cling to an outdated system for the establishment and measurement of authority at the very same time that the nature of authority is shifting around us, we run the risk of becoming increasingly irrelevant to the dominant ways of knowing of contemporary culture.[1.8]

¶ 8 Leave a comment on paragraph 8 2 For this reason, what I am absolutely not arguing in what follows is that we need to ensure that peer-reviewed journals online are of equivalent value to peer-reviewed journals in print; in fact, I believe that such an equation is instead part the problem I’m addressing. Imposing traditional methods of peer review on digital publishing might help a transition to digital publishing in the short term, enabling more traditionally-minded scholars to see electronic and print scholarship as equivalent in value, but it will hobble us in the long term, as we employ outdated methods in a public space that operates under radically different systems of authorization. Instead, we must find ways to work with, to improve, and to adapt those new systems for scholarly use – but we must also find ways to convince ourselves, our colleagues, and our institutions of the value that is produced by the use of such systems.