I think most reviews are good and fair. Regardless, when I get an unwelcome decision back from an editor, it’s annoying. Getting annoyed is natural. Here’s how I process bad reviews.

If it’s a rejection or reject-and-resubmit, I just give the review a very quick glance. After some while (ranging from minutes to days, depending on my mood and schedule), I’ll read over the reviews.

In a very general sense, a “bad review” is any kind of review that results in an adverse decision. Let’s say that, after reflection, the review is fair, factual, and constructive. What I do is revise the paper and move on, addressing the concerns of the reviewer as much as possible. That’s how things are supposed to work, after all.

The bad reviews that I’m writing about here, though, are the ones that are poor in quality. There are a lot of things that can wreck a review:

• the reviewer didn’t read the paper carefully
• the reviewer wishes you ran a different experiment
• the reviewer sees you as a competitor and wants to undermine your work
• the reviewer is insulted that you’re not using their favorite theories
• the reviewer doesn’t understand the statistics you used
• the reviewer doesn’t understand statistics in general
• the reviewer makes generalized claims but those claims are entirely unsupported
• the reviewer is an asshole
• the reviewer took a minor fixable issue and insisted it was a major unfixable flaw
• the reviewer thinks their interpretation is more valid than yours and expects you to spend more real estate on their ideas rather than your own
• the reviewer does not approve of your gender
• the reviewer thinks that your experimental technique was inadequate and implies that your results are not valid as a consequence
• the reviewer says your paper is poorly written but doesn’t even provide a single example or amplify on this topic any way, so it’s not clear in what manner it might be poorly written
• the reviewer says that you have failed to cite a bunch of relevant literature but doesn’t even vaguely mention what kind of things they want you to cite
• The reviewer is just so rude and critical that you can’t tell which things are major concerns on their part and just minor things that they don’t think are a big deal
• What else am I missing?

Clearly, these issues can result in an adverse decision. How to deal with them? I think it helps to try to see things through the eyes of the editor, who is making the actual decisions. Just like being on grant panels helps you write better proposals, being an editor helps you navigate the publishing process more effectively.

As an author, I don’t often get reviews with these issues. Most of my reviews have been fair, I think. A good number of them could have been less acerbic, but when I look at the actual criticisms, they’re usually quite legitimate. (When I do have issues with the review, it’s from someone who obviously didn’t even read the whole paper, or they want me to say things in the discussion that I don’t think do much good.) However, as an editor, by volume I see more reviews than I do as an author, and while the vast majority of the reviews I solicit are somewhere between good and great, I’ve seen most of these problems in one form or another.

Let’s take a step back to consider why bad reviews happen. It is possible for someone to intentionally try sabotage a competitor. I’m not sure this is so common in my fields, though of course it has to happen. Even if there isn’t any intentional torpedoing, it’s a fact that people are chock full of biases. This is a simple consequence of human nature. Honest folks want to write reviews without biases. But our backgrounds and experiences inform what we notice and don’t notice. We can’t be unbiased.

Editors send manuscript to experts because their expertise should result in a thorough evaluation, and the identification of issues that should be addressed by authors. Some reviewers will scrutinize statistical methods or interpretation more heavily than other reviewers. Others will care more about grammar, and others will focus on the details of the methods.

Experts have blind spots, and biases are often implicit. That means that people don’t know when they manifest these biases. People can be aware of the fact that they have been known to demonstrate implicit biases, but by definition, are not aware when they manifest. Yes, that means you, and me, and everybody else. I think it’s our duty to work to eliminate them, but our brain is disposed to create cognitive shortcuts that fail to represent reality. These implicit biases aren’t just about gender or ethnicity or institutional prestige. They’re also about choice in software, fonts, statistical philosophies, vocabulary, and so many other things about the manuscripts we review. We are a big bundle of biases, and recognizing these biases is how we can endeavor to write a review that is as fair as possible.

Bad reviews can happen because a reviewer is malicious. Other times, it can happen because they are lazy or sloppy. However, a lot of times, it’s because a manuscript has a mismatch with the biases of the reviewer that gets in the way of a solid review.

Before deciding how to tackle the bad review, I try to keep in mind that a bad review does not mean, necessarily, that the bad review resulted in a bad decision. For this to happen, this would require the editor to be unaware that the review is bad. As an editor, I get bad reviews once in a while. Sometimes they’re just two-line reviews that say, “looks great!” and others amount to diatribes. Others are just misinformed, or attack the expertise of the authors instead of the content of the manuscript. Other times, there are detailed and comprehensive reviews that have a lot of helpful remarks, and then there are a couple suggestions that are downright bad mixed in. Because we solicit reviews who are more expert in the manuscript then ourselves, it’s possible that we’re not catching some subtleties. Nevertheless, I’m of the general opinion that when such reviews cross my desk, that I know how to read them in context, and I hope I end up making fair and reasonable decisions. By default, I am expecting the same wisdom from other editors. In a lot of journals, it’s not just a decision made by one editor. The handling editor renders a recommendation and then it passes across the Editor-in-Chief’s desk, and the EIC is responsible that the handling editor’s read on the situation is appropriate.

What do editors do when they get a bad review? Well, it depends on how bad the review is. If it’s downright inappropriate and directly offensive, then it could be trashed. I think this is extraordinarily rare. If it’s mainly low quality and is of limited use, then the editor probably will probably solicit a third review. On the other hand, if the non-bad review is particularly helpful, and the editor feels qualified to evaluate the manuscript as a reviewer, then they might decide to go ahead with one good review, one bad review, and the editor’s own assessment.

There’s one big reason why it’s hard to read through an editor’s decision making process when you get the reviews back: in most journals that I’m aware of, the reviewers receive a copy of the decision. Keep in mind that even though an editor received a crappy review, they’re not positioned to overtly identify a crappy review as such. After all, if a reviewer was kind enough to accept a review, the last thing you want to do is insult them after submitting a review that was not useful. So instead, when reading a decision letter, you have to read between the lines. The editor, hopefully, does more than provide a pro forma response about the rejection or the revision, and provides a clear rationale about the major concerns.

For example, let’s think about a situation where a paper gets rejected and it has two critical reviews, one that is fair and one that is obviously bad. We really have no way of knowing what the editor was thinking about the rejection, unless they say so. A strong editor would say it that the rejection was made because of X concern (from the good review), and maybe use some delicate language that Y concern (from the bad review) was not as problematic. They can’t overtly say, “I rejected you because of what Reviewer 1 said and not what Reviewer 2 said,” though they might do their best to imply this. That’s because it’s not politic for an editor to throw one of their own reviewers under the bus, even if it’s merited.

But on the other hand, if you get a rejection that you suspect is unfair, because it is based on a bad review. If you don’t have any guidance from the editor, then what should you do? Well, it’s okay to move on to another journal. But if you think that it’s useful to pursue this particular journal further, the way to do this is to write back to the editor, very politely asking about the specific reasons for the rejection, and provide specific thoughts about why Reviewer #2’s concerns are spurious. It’s not so much about asking them to reverse their decision, but looking to understand the decision. And if you think the editor needs an education about the content of the reviews, then this is your chance to inform them. Editors cannot be expert in everything they handle, so this isn’t entirely inappropriate, and if an editor got hoodwinked by a review in an area where their expertise is relatively lacking, then they should appreciate learning about this, you would hope.

If you’re wondering if your correspondence hits the right note, run it by a trusted colleague with some more experience. But as long as you’re staying collegial and respectful, then things really should be fine.

It’s a bit more sticky when asked to revise a manuscript after you’ve gotten a bad quality review. I don’t want to make a manuscript worse, nor do I want to fail to make changes that the editor thinks are critical. In theory, a strong editor will give specific guidance about which changes to the manuscript are necessary, and then guide towards understanding which reviewer remarks I can choose to gloss over.

Sometimes, when the editor says something along the lines of, “make all the changes the reviewers want,” that’s pretty firm. But on the other hand, if I think that some of the remarks in the reviews don’t merit a change in the manuscript, I’ll just address them with a response in the cover letter. Then it’s up to the editor to decide whether or not my response is more valid than whatever (bad) idea the reviewer came up with.

If it’s something that won’t entirely undermine the manuscript, I have to admit that I usually just go along with what the reviewer and editor want. I do that, in part, because it’s the path of least resistance. And I also do it because I trust their judgment. I’m close enough to my own work that I do have to put some trust in the peer review process. But if there’s a change that really would undermine the paper or result in something that is not good science, then I’d have to address in the cover letter.

The way I’ve approached responding to bad reviews has changed as I’ve gained more experience. If a review has a change that doesn’t improve or doesn’t harm the manuscript, I just go ahead and do it, because it can help mollify other parties. But if there’s a change that I think is a bad idea, and the editor doesn’t specifically require the change, then I don’t worry as much about not doing it. I don’t go to great lengths in the response letter to explain why I’m right and why the reviewer is wrong. I simply write a sentence or two in the response about why my approach is better, directly addressing the concern of the reviewer. If I write more than this, then I’ve giving too much credence to a topic that doesn’t necessarily deserve as much verbiage. I won’t let any content in a review go without remark, but I don’t have to fawn over bad ideas. If I work too hard at discrediting a bad review, I might give it more power by establishing an adversarial relationship. What I sometimes will do is take a third approach, better than what I initially did and not what the reviewer suggested. This allows the editor to realize that I’ve given it substantial consideration, and worked for improvements, and made a change to deal with the issue of the reviewer, even if it’s not what they wanted.

If there are minor things that I don’t want to change, then I’ll just go ahead and deal with it very briefly in the cover letter, with a short sentence explaining why I didn’t make the change. (I recently handled a revision that had a series of prescriptions that made the paper more obtuse. The author of the response letter was clearly on the same page as me, as the response to each of these suggestions was merely, “Not changed.” I should add a disclaimer: Don’t try this at home.)

How do I handle a revision when the reviewer is just straight up wrong about something really big, and I’m not sure if the editor realizes this? On one occasion, I contacted the editor before embarking on the revision, because I didn’t want to go to war with Reviewer 2 if the editor was on the other side. My email said something along the lines of, “You wanted me to make a revision with changes and responses to reviews, and here are some big changes I’m planning. But when it comes to these two items from Reviewer 2, in your opinion is it critical that I make these changes? I’m asking because I have concerns that this is a problem [because it has conceptual flaws, isn’t appropriate statistically, et cetera], and I’m hoping I can just deal with this in the cover letter. What do you think?”

In that case, the editor got back to me, and wrote in rather frank terms, “I think that’s just fine – I didn’t expect you to make all of those changes, just explain it in the cover letter.” While this was frustrating because I would have hoped for this guidance in the decision letter, it’s was nice to know that the editor wasn’t going to take me to task for doing all of the dumb things the reviewer wanted.

As an editor, I think I’ve only gotten one query along these lines, but then again, I do my best to make it clear in decision letters which changes I think are critical for the manuscript. If there are revisions that are essential, I go ahead and say that (risking the ire of a reviewer, but then again, if it’s a less experienced reviewer, then they hopefully will see this as useful feedback).

On another recent manuscript of mine, the opposite thing happened. A reviewer wanted me to rewrite the paper with a different set of theories (which would require, of course, referring to a different set of papers). To make the long story short, it became clear that the editor didn’t seem to read my rebuttal, they just thought (for whatever reasons I don’t know, because they were not expressed) that the reviewer was right and their prescriptions needed to be followed. So we had a couple rounds of review before it was clear that I just needed to cave in or submit elsewhere. And I caved (mainly because I wanted to make sure my student coauthor had this paper in press for grad school applications). Would it have made sense for me to write this editor to take their temperature about Reviewer 2’s demands? I guess? But on the other hand, my rebuttal was substantial in content and it made sense to include it with a whole revised manuscript.

What would I do if I think the editor is wrong? I think there are two possible courses of action. The first would be just to move on and let the editor be wrong. That’s a fine choice, and it’s one I’ve made on multiple occasions. (I do not engage in many righteous battles, because the alternative is perpetual conflict.) The other choice is to write back to the editor who sent the decision. If you think the decision is unfair because it’s based on a bad review, or a bad interpretation of a review, then you can ask about this. I think as long as I stay polite, don’t make demands, and explain that I’d like to understand the reasoning, then I think everything should be fine. If I get rejected because they think the result isn’t just interesting or important enough, then I think that’s the way things are. To me, that usually just means my manuscript itself failed to communicate the importance of the work. But if I get rejected because the reviewer identifies a flaw that isn’t a flaw, or a mistake that isn’t a mistake, then it only makes sense to clarify the matter with the editor and ask them to reconsider. If you approach the conversation as a goal of reaching mutual understanding of the facts, then I imagine that is probably more likely to result in success.

That said, I’ve only written back to an editor once after a rejection, and I was totally blown off. It was a desk reject, and I wasn’t appealing the decision, I was just asking for a clarification. They said that my particular system wasn’t a fit for the scope journal. Meanwhile, I had actually chosen to submit to that particular journal because there was a paper with same system and a similar question, published in the journal in the previous year! I asked the editor how was it that my paper wasn’t a good fit, while this other paper was? I was just just seriously confused, I was just trying to understand the reasoning. Since I had inadvertently caught them out in a bit of hypocrisy, I can understand why I just got ignored. But, still. I get how “not a fit for the scope” can be code for “not important enough,” but at the very least, authors might expect editors at the very least to not write things that are factually incorrect. (Since then, I’ve heard some unsolicited stories about how this person is generally a jerk, so I’m not surprised I got blown off.) I’m inclined to think that this experience is not the norm.

Ultimately, publishing a paper in an academic journal is a power relationship, in which editors have control over what gets published. Reviewers, on the other hand, only derive their power in their level of influence over the editor. The author doesn’t know the relationship between the editor and the reviewer, and some reviews and some reviewers will command higher credence than others. If you honestly don’t know how to deal with a particular review, as is so often the case, the straightforward course of action is to ask. Editors will typically be pleased to a brief exchange of emails as long as their designed to help the authors create a better revision. For the author, this means getting an understanding of what “better revision” means in the mind of the editor.

How do you handle bad reviews? How quick are you to pull the trigger on emailing the editor with questions? How often are appeals successful, and how often it it worth the trouble? Any additional thoughts?