Nowadays, I rarely sign my reviews.
In general, I think it’s best if reviews are anonymous. This is my opinion as an author, as a reviewer, and as an editor. What are my reasons? Anonymous reviews might promote better science, facilitate a more even paying field, and protect junior scientists.
The freedom to sign reviews without negative repercussions is a manifestation of privilege. The use of signed reviews promotes an environment in which some have more latitude than others. When a tenured professor such as myself signs reviews, especially those with negative recommendations, I’m exercising liberties that are not as available to a PhD candidate.
To explain this, here I describe and compare the potential negative repercussions of signed and unsigned reviews.
Unsigned reviews create the potential for harm to authors, though this harm may be evenly distributed among researchers. Arguably, unsigned reviews allow reviewers to be sloppy and get away with a less-than-complete evaluation, which will cause the reviewer to fall out of the good graces of the editor, but not that of the authors. Also, reviewer anonymity allows scientific competitors or enemies to write reviews that unfairly trash (or more strategically sabotage) the work of one another. Junior scientists may not have as much social capital to garner favorable reviews from friends in the business as senior researchers. But on the other hand, anonymous reviews can mask the favoritism that may happen during the review process, conferring an advantage to senior researchers with a larger professional network.
Signed reviews create the potential for harm to reviewers, and confer an advantage to influential authors. It would take a brave, and perhaps foolhardy, junior scientist to write a thorough review of a poor-quality paper coming from the lab of an established senior scientist. This could harm the odds of landing a postdoc, getting a grant funded, or getting a favorable external tenure evaluation. Meanwhile, senior scientists may have more latitude to be critical without fear of direct effects on the ability to bring home a monthly paycheck. Signed reviews might allow more influential scientists to experience a breezier peer review experience than unknown authors.
When the identity of reviewers is disclosed, these data may result in more novel game theoretical strategies that may further subvert the peer-review process. For example, I know there are some reviewers out there who seem to really love the stuff that I do, and there is at least one (and maybe more) who appear to have it in for me. It would only be rational for me to list the people who give me negative reviews as non-preferred reviewers, and those who gave positive reviews as recommended reviewers. If I knew who they were. If everybody knew who gave them more positive and more negative reviews, some people would make choices to help them exploit the system to garner more lightweight peer review. The removal of anonymity can open the door to corruption, including tit-for-tat review strategies. Such a dynamic in the system would further exacerbate the asymmetries between the less experienced and more experienced scientists.
The use of signed reviews won’t stop people from sabotaging other papers. However signed reviews might allow more senior researchers to use their experience with the review system to exploit it in their favor. It takes experience receiving reviews, writing reviews, and handling manuscripts to anticipate the how editors respond to reviews. Of course, let’s not undersell editors, most of whom I would guess are savvy people capable of putting reviews in social context.
I’ve heard a number people say that signing their reviews forces them to write better reviews. This implies that some may use the veil of their identity to act less than honorably or at least not try as hard. (If you were to ask pseudonymous science bloggers, most would disagree.) While the content of the review might be substantially the same regardless of identity, a signed review might be polished with more varnish. I work hard to be polite and write a fair review regardless of whether I put my name on it. But I do admit that when I sign a review, I give it a triple-read to minimize the risk that something could be taken the wrong way (just as whenever I publish a post on this site). I wouldn’t intentionally say anything different when I sign, but it’s normal to take negative reviews personally, so I try to phrase things so that the negative feelings aren’t transferred to me as a person.
I haven’t always felt this way. About ten years ago, I consciously chose to sign all of my reviews, and I did this for a few years. I observed two side effects of this choice. The first one was a couple instances of awkward interactions at conferences. The second was an uptick in the rate which I was asked to review stuff. I think this is not merely a correlative relationship, because a bunch of the editors who were hitting me up for reviews were authors of papers that I had recently reviewed non-anonymously. (This was affirmation that I did a good job with my reviews, which was nice. But as we say, being a good reviewer and three bucks will get you a cup of coffee.)
Why did I give up signing reviews? Rejection rates for journals are high; most papers are rejected. Even though my reviews, on average, had similar recommendations as other reviewers, it was my name as reviewer that was connected to the rejection. My subfields are small, and if there’s someone who I’ve yet to meet, I don’t want my first introduction to be a review that results in a rejection.
Having a signed review is different than being the rejecting subject editor. As subject editor, I point to reviews to validate the decision, and I also have my well-reasoned editor-in-chief, who to his credit doesn’t follow subject editor recommendations in a pro forma fashion. The reviewer is the bad guy, not the editor. I don’t want to be identified as the bad guy unless it’s necessary. Even if my review is affirming, polite, and as professional as possible in a good way, if the paper is rejected, I’m the mechanism by which it’s rejected. My position at a teaching-focused institution places me on the margins of the research community, even if I am an active researcher. Why the heck would I put my name on something that, if taken the wrong way, could result in further marginalization?
When do I sign? There are two kinds of situations. First, some journals ask us to sign, and I will for high-acceptance rate journals. Second, if I recommend changes involving citations to my own work, I sign. I don’t think I’ve ever said “cite my stuff” when uncited, but sometimes a paper that cites me and follows up on something in my own work, and I step in to clarify. It would be disingenuous to hide my identity at that point.
The take home message on peer review is: The veil of anonymity in peer review unfairly confers advantages to influential researchers, but the removal of that veil creates a new set of more pernicious effects for less influential researchers.
–
Thanks to Dezene Huber whose remark prompted me to elevate this post from the queue of unwritten posts.
Small Pond Science 
There’s a big difference between being required (or feeling obligated) to sign and opting to do so in an otherwise anonymous context. I’m now a default review signer, and would add a couple of other positive reasons for doing so. I always make sure to say that if there’s anything in my review that is unclear, or that I have completely misunderstood (and that does happen), I’m happy to discuss it with the authors informally. This in my experience leads to a much more constructive approach to peer review and is less like playing ping-pong with the authors. A little discussion often smooths over minor issues. I’d like to believe that no editor would allow an author to pull rank or ignore a reviewer just because they knew who they were; regardless of their status, they’re still one of the people most likely to read, cite and apply the findings in future. Finally, many of the inherent biases you mention are already present because in most cases we know the identities of the authors. It may sound contradictory, but I am fully in favour of double-blind review, and think it should be standard. If I know their names already then it seems unfair to hide my own.
I’m not sure whether my feelings would be different in another field; the community I work in is notably friendly, collegiate and not wracked with the tensions that plague highly competitive areas. Signing reviewers are also in the minority so there’s no sense of social obligation. I do get your point that it is an exercise of power and could lead to a club mentality, I just don’t see that as a risk given the people I know, and on balance have decided that the positives of signing outweigh the negatives.
I’m in favor of double-blind reviews as well, except for one detail: I can remember at least two instances when, after searching for the author’s name on scientific databases, I found recently-published articles that were very similar in form and content to the one I was reviewing (same data, some of the same figures, slightly different take on the subject…even whole paragraphs in one case). On both times, I still reviewed the manuscript fairly, but was explicit into pointing the similarity to authors and editor. And I now make a point to start my reviews by looking up the author’s publication history.
As long as the handling editors do this kind of check themselves, I do agree that double-blind has the least potential for bias or backlash (yes, you can tell who the senior authors are most of the time, based on subject and citations, but young researchers would be better protected).
Really interesting blog. As a grad student some of the issues you raise are good to think about when reviewing. I’ve yet to sign a review, in part for some of the reasons you mentioned. I have had reviewers sign reviews on my papers. In those cases the reviews were very harsh and did not offer any constructive comments. It was embarrassing and terrifying because they were from well known people in my field. The upside in both cases were the second reviewers (anonymous) were positive and in both cases the papers were accepted. The review process still remains confusing to me.
As a grad student I find reviewing to be a very difficult skill to learn. I actually cherish a few amazing reviews I have received that constructively broke down my papers and in the end made them much better. On this note, do you have any good guidelines or advice for being a good reviewer?
Hi Mason.
Thanks for sharing your experiences with peer review. As an early career scientist myself, I’m still working on developing my reviewing skills. I imagine I’ll get better at it as I receive and write more reviews, but I do wish my grad school training had included peer review. It’s an important aspect of our work as members of the scientific community (regardless of whether it’s pre- or post-publication peer review).
As it happens, this article on suggestions for effective peer review showed up in my Twitter feed right around Terry’s post. I’ll keep this advice in mind when I write my next review.
http://www.theguardian.com/higher-education-network/blog/2013/sep/27/peer-review-10-tips-research-paper
I actually linked to an *exceptional* guide to doing reviews for novice reviewers last month, a document made by the British Ecological Society, but I think it’s broadly useful for all sciences (and perhaps beyond).
Here’s the Rec reads post: https://smallpondscience.com/2014/02/14/friday-recommended-reads-17/
And the link to the BES pdf itself: http://www.britishecologicalsociety.org/wp-content/uploads/Publ_Peer-Review-Booklet.pdf
Good post, and some great points.
I fully believe that the option should exist either way. I’d definitely rebel against a journal that *required* reviewers to sign their reviews. But I’d also have some difficulty with one that would not allow it at all either.
Markus’ point: “I always make sure to say that if there’s anything in my review that is unclear, or that I have completely misunderstood (and that does happen), I’m happy to discuss it with the authors informally.”
…is a good one. And this is one of the main reasons that I often sign reviews. I have received reviews that I had questions about, but had no one to specifically ask. I suppose I could have asked the editor to relay the question, but that is not great for extended dialogue. Signed reviews allow me to contact the reviewer to ask what she meant, and vice versa.
I always work to be a careful reviewer (which is perhaps why I receive quite a few review requests), because I have received a few too many crappy, six-sentence reviews in my time and I don’t want to be “that guy.” As you note, however, if I decide to sign a review, I give it a third and fourth check before sending it back to the journal. While it should not make a difference in quality/behavior, the truth is that accountability does push people a bit harder even when they would already do the right thing.
I also agree with the comment on double-blind review. That, in my opinion, is great, but few journals practice it. I know some of the return arguments – e.g., it’s never fully blind since only a few groups in the world would be capable of doing study X. But it still removes a great deal of the potential for bias on both ends of the equation.
Personally I reserve the right as a reviewer to self-identify in cases where I deem it useful. I also reserve the right as a reviewer to be anonymous when it is my preference for whatever reason.
A while back I made a decision to sign all my reviews, even as an early career researcher, in an effort to promote a culture of open science & whatever else. The very next review I received was for a paper by a very senior researcher that I felt had fundamental problems (that I fairly addressed and presented solutions for). At that point idealism met reality and I submitted my review anonymously so your point about the imbalance is well taken. Would it have affected me in the long run? Probably not. I have little contact with the author, but the perception is certainly there, and in many cases that’s enough.
Having said that, I continue to anonymize my reviews, even when I’m unlikely to be harmed (and might actually benefit). If I’m picking and choosing when to sign then it indicates that I’m making a value judgement about the benefits that might come to me from signing, and implies something about the author of the paper as well. Do I sign when I feel like my seniority is higher than the paper’s author? Am I asserting some sort of intellectual or cultural dominance when I sign?
This certainly hampers the ability to discuss aspects of the review with the author, but some journals are moving to more interactive comments. I recently reviewed for Frontiers in Ecology and Evolution and have reviewed for Climate of the Past, both of which have a semi-interactive forum for review. Potentially using this format we can preserve anonymity but allow for more interactive review.
I actually decided to sign a review I did when I was a graduate student, for a paper by some authors considerably senior to me. It was mainly on principle, but I am not sure I would do it again. On the one hand, I didn’t experience any negative backlash from it, even though I was critical of some of the author’s interpretations of their data. I tend to believe that the majority of senior people in my field would not be petty enough to actively undermine me just for being a bit critical of one of their papers, especially if the criticism was trying to be constructive. And even if they did, I also don’t believe even really senior people have as much power to keep junior scientists down as we tend to believe they do, but perhaps I am deluding myself here. I figure anyone who would act that way must have plenty of enemies — enemies who would be more than willing to help out someone who happened to stray into their line of fire. That being said, I am white and male, which would, sadly, make me more resistant to such an attack than if I was female and/or non-white. On the other hand, I did have to do a symposium with the authors I reviewed a few months later, and it turned out to be really awkward, so for that reason alone I am not sure I will sign a review again.
Probably a better way of dealing with some of the issues that signing reviews are supposed to help solve, would be to keep reviewers anonymous, and instead make the authors anonymous as well. I have trouble understanding why most journals are not double-blind by now, given the documented issues.
Excellent post – sums up my feelings exactly – I only sign if there is no other choice
While it’s not perfect, I’m in agreement with the people who favor double-blind reviews. Less potential for bias all around. This is not uncommon in my personal (and thus both limited, since I’m a mid-stage grad student, and field-dependent) experience.
I just spoke with a colleague today and we discussed this very issue because I know at one time it was her mantra to sign peer-reviews. She started signing her reviews during her time as a postdoc, believing it was for the betterment of science. She started to realize over the subsequent years that maybe it wasn’t such a bright move, albeit based on highly idealistic reasons. For one, she started to see her name pop up in acknowledgement sections of papers she reviewed, yet the published product did not incorporate her comments/requests. This was definitely an unintended consequence, because it left the impression that she was on board with the content of the paper (which she wasn’t). Later on, she began to wonder if her open-reviews inadvertently affected her career in a negative way. She could think of a few instances where her honest and well-meaning criticism may have been taken personally and perhaps spread through the very small social network of her discipline. Now, she does not sign her reviews, although she still laments her idealism for open and transparent science.
As one on the far distant margins of the academic ecology establishment, I usually sign any reviews that come my way because I appreciate it when I receive signed reviews. I attempt to be rigorous but kind in my reviews. Perhaps I am identified as the villain when manuscripts are rejected, but I figure bad attention is better than no attention.
This post (and the story andreakirkwood posted) partly mirror my own evolving view on signing reviews. I do believe as a matter of ethics that one should decide whether to sign or not BEFORE reading the manuscript.
With respect to anonymous reviewers feeling more able to be rude to the author, it’s important not to discount the role of a competent editorial office. Most of our (Mol Ecol) reviews seem polite and objective, but if one gets out of hand we edit out the offensive bits and ask the reviewer to approve the new version. They always do, and typically apologize for being rude in the first place.
I would like too add two more thoughts. First is about the ideal model. Most people seems to agree double blind is a good thing to do. I agree, but then, when the paper is published there should be double credit too, and the reviewer names should be disclosed with the paper. Is a way of getting credit, but also of taking responsibility (yes I am fine too publishing the revision history).
Second, I do sign my reviews. I rarely get piss at my reviewers (even less when they sign) because in my field they tend to be constructive (or maybe I was lucky so far). I think that for every one author that may blame the reviewer and take it personally, there is 10 authors who will appreciate your criticism and take it well. May be I am too naive, but I was never contacted by a pissed author, but I was contacted by an author of a rejected paper I reviewed to thank me about my review.
A friend of mine who recently edited a special issue in a journal, noticed how signed reviews were fairer and more constructive than anonymous ones. However, I don’t sign my reviews for fear of backlash. The best solution, I think, would be double-blind review. There are too many prejudices against scientist who are on the sides of a big competitive field. I have had several very rude and unfair comments from both reviewers and mostly editors, which I suspect wouldn’t have been made if the work had been done in a big lab, managed by a big name. My work may have flaws, but that is not an excuse not to check facts first (and the supplementary material for that matter), and be disrespectful.
If the people here are saying that they have it bad being in the outskirts of the field in a teaching institution, try doing research in a Latin American institution and get it published.
I think you are wrong. When you receive a biased anonymous review you cannot defend yourself against it. You are massacred by an armed person after having been disarmed. This is extremely cruel and unfair. It is actually sadistic. The reviewer would think twice doing that if he were not anonymous. Somebody who writes an outrageous dishonest report should be accountable for it and be apt to be convicted in Court and have is career ruined for his misconduct. I would like to do that to some of those anonymous people who caused me enormous prejudice and even ruined my health. After all, that is what people who write biased reports are after, is it not: Cause immense suffering in strict impunity and take pleasure from it. It just amounts to career murder. When one asks you to review a paper of a known person not anonymously, you are not obliged to accept. Most of all, there many many more less well known scientists than very influential big shots. So your counterexample to justify anonymous peer review addresses the exception rather than the rule, and I therefore think you have some interests that you are hiding us. It is not a two-sided medal as you want to make us believe. The crime is sweet and there are loads of people who want to continue that way. And that is why it is still around despite the fact that everybody knows it is bankrupt.
I’m sorry, I didn’t catch your name?
It is also anonymous you see. That is the whole point.
Many times a victim (of anonymous peer review)