Web Writing: Why and How for Liberal Arts Teaching and Learning (archived site)

I was thinking more about this on my way to work this morning. It seems to me that the difference I was thinking about is the difference between reviewing and editing. And it makes sense to keep some editorial processes behind closed doors.

I’ve noticed that the authors in this volume respond differently to the comments. Some people thank the commenter for their insights; others defend or explain their original text. I suppose those differences shouldn’t be surprising.

It is interesting to me that some portions of the project continue to be private, taking place behind the scenes. I’m not arguing with the idea of retaining private emails in which authors are notified that their papers are not advancing, as that seems like the soul of courtesy. But it does suggest limits on the process of open peer review that probably have implications for understanding the wider process.

Kristen, just saw your tweets about open peer review guidelines, and will link to the timely blog post you cited for others to read:

Cristina Costa, “Open Peer Review Is a Welcome Step Towards Transparency, but Heightened Visibility May Also Mean Vulnerability,” Impact of Social Sciences: The London School of Economics and Political Science, September 17, 2013, http://blogs.lse.ac.uk/impactofsocialsciences/2013/09/17/peerreviewcosta/.

I like your encouragement of authors to participate very much, but what I don’t see (perhaps I overlooked it?) is direct encouragement to add comments or to engage with comments  on one’s own essay.  From my own experience as a commenter, getting a reply makes me comment more often (and, perhaps even more fruitfully), to have a real conversation.  A while back I had a little discussion a propos of this with Michelle Moravec with respect to the Subjecting History project (on her “History in the City” blog at http://historyinthecity.blogspot.de/2013/03/whats-public-where-is-open.html.).  Michelle writes there that  “I really really wanted to pose questions in the comments, [on her own essay] which I do when ‘writing in public’ but didn’t know if this was part of the format of this particular volume.”  I found this interesting, since I assumed this commenting on one’s own writing  would be a natural part of the open review dialogue, but as you can see Michelle had some doubts, which I’d say may be related to the “format” of the volume, as she describes it, but also the culture as established by the instructions given and the models provided (performed) by editors and others who either set the tone or else (probably in a healthy way) don’t much care what the extant tone is.  So I wonder whether open peer review on this model should not only accommodate but also encourage authors asking questions to readers about their own texts. Or is it important that the draft texts speak for themselves, at first pass?  Can they even do that once comments (written by anyone, let alone the author) have appeared?

Great question, Kristen. (For those who may not know, she and I co-edited our first experiment in large-scale open peer review, Writing History in the Digital Age, in fall 2011, and together we figured out much of the process that informs this venture.)

As for instructions/encouragement to authors to participate in the open peer review, I sent this message to each section when their batch of essays went live, a day before the official launch: “Your essays are now public under the ___ section of Web Writing, with more to come. Would you help us get the ball rolling on Monday by commenting on at least one other author’s essay?”

When we officially launched on Monday morning, here’s the relevant text I emailed all contributors (with a link to blog announcement): “Feel free to circulate the announcement above — and a direct link to your essay — via email, Facebook, Twitter, etc. We’re relying on our authors to start the ‘commenting party’ by posting feedback on other essays. Feel free to ask questions, request clarifications, challenge assumptions, offer constructive criticism, or add links to suggested reading. Take advantage of this valuable opportunity for developmental editing before the final book is completed.”

One day into this process, several authors have posted comments on each other’s essays, but most have not at this early stage. No worries – at least they’ve finished their essays, and I’m late in getting my own uploaded to the site.

I’d love to hear thoughts from you (and others) about what a “how-to open peer review” tutorial would look like, and whether it’s necessary or not.

Are authors required to participate in the open peer review in order to advance to the final manuscript? No. To check this, Kristen and I counted each comment and its source after the 2011 review of Writing History in the Digital Age. Looking back at the spreadsheet notes, I see that authors of 24 out of 28 essays posted comments, either on other essays or in response to their own. Of the 4 essays with non-commenting authors, we advanced 3 to the final volume. Kristen and I wrote about how we made these editorial judgements, which certainly were not driven by a mechanical popularity contest (as some highly-commented essays did not advance), which you can read in “Conclusions: What We Learned.”

Do I still recommend that authors participate in the open peer review process? You betcha. As writers, developmental feedback from readers of our drafts is what we crave. Give to others and you’re more likely to get some back.

Moreover, all of the contributors to this edited volume have a stake in raising the quality of the work as a whole, because a stronger and more coherent book is more likely to be read and recommended to others, thereby increasing the odds that broader audiences will find each individual essay. Author participation in open peer review may be an ideal case study of the benefits of altruistic behavior.

I have a longstanding interest in the roles of authors, editors, and reviewer/commenters in open peer review, I’d be interested to know what instructions or particular encouragement  (if any) you’ve given your authors as to how to participate in the open review as an author.

Do authors’ own comments (quantity or quality) play any role in the selection?  I.e. do you think an author who replies to comments (at all, or in some depth) would have any edge over an author who doesn’t reply or engage in discussion about his/her text in the open review period?

Do you think authors would benefit from an “open peer review how-to”, or do you think they, like commenters, will find their way? It seems to me that authors have a lot more at stake than others (except, perhaps, editors) in “doing open peer review” right.