Apparently, there are some editors of academic journals who will readily send manuscripts out to “non-preferred reviewers” — the specific people that authors specify who they don’t want to receive the paper for review.
I think this is all kinds of messed up.
Is this a simple, clear-cut issue? Well, no. There are a lot of factors at work on the surface, and a lot going on under the surface that the involved parties might not be aware of. However, notwithstanding the complexity of the issues at hand, it’s hard for me to imagine a circumstance where this kind of action would be advisable (and I’d like to be clear that this is a practice that I do not engage in).
To try to make sense of how and why this might happen, I’ll break down what I see as the potential motives that authors have for listing non-preferred reviewers, and the motives that editors might have to disregard these requests. These scenarios are not mutually exclusive.
Author motives for excluding certain reviewers
Competing research agenda: If more than one lab group is working on the same research question, then letting another lab have access to the work before it is made public may put the interests of the authors at risk. (Yeah, “open science” folks are just going to say this problem is fixed by “post a preprint.” Fine. Sure. Then this isn’t a problem for you folks.) I’ve heard stories from multiple people how they think that other labs have intentionally sat on their papers in review, while the reviewer’s group rushed out papers on the same topic first.
Antagonistic personalities: It could just be that there’s someone in your field who is a jerk — either in general, or specifically to a small number of people. Someone who will intentionally tank the success of others because that’s what they do. For example, there are some people, I’ve learned through experience as an editor, there are some toxic people who I will never ever want to subject authors to again. If authors have this kind of information — or even a very solid hunch, this could be a motive.
Negative personal history: There can be bad blood from prior collaborations or other personal involvements. And, of course, sexual harassment and sexual assault is an epidemic in academia, and many authors have been targeted by powerful people in their field. Such malefactors obviously should not have this form of authority.
To avoid a cabal: There is no shortage of sub-subfields that are run by an old boys’ club. They often review one another’s papers, and they may collaborate together, and they will have a set of ideas about what is true and what is not, what works and what doesn’t work, and what is acceptable and what is not. If you’re a researcher who is working on topics related to such a cabal, then you don’t want to get burned by treading on their territory.
To avoid a review from someone whose work is being criticized: If work in a paper is critical of ideas expressed by authors of a prior work, then I can see how an author might want to avoid having their work reviewed by the author subject to their critique.
To avoid a challenging review: It could be that an author is aware that someone in the field writes reviews that are particularly detailed and offer cogent challenges to the ideas in the papers they review, and they’d want to avoid this kind of review.
To avoid a well-informed review: If there are particular weaknesses in a manuscript that the authors are aware of, and the authors believes that this weakness would only be identified by a small number of people in the world, then they might to avoid this criticism by listing these people as non-preferred reviewers.
Maybe there are some reasons I’m missing out on. I think the latter three are unacceptable reasons. In my experience, nobody has ever actually stated explicitly that they’re just trying to avoid a challenging or well-informed review, or review from the person whose work is being criticized. But sometimes people will list some names and then not provide an explanation, even though the manuscript submission systems provide space for an explanation and one is expected. Is avoiding a cabal of entrenched and experienced reviewers a problem? Well, that depends on the details in a situation. And one of the factors here is what the relationship is between the editor and the members of the cabal. If these are people who have known one another for a long time, it’s quite possible the editor might have a more charitable interpretation of the motives and perspectives of the cabal than those with more distance.
One thing to keep in mind is that plenty of authors might have some highly legitimate reasons for wanting to exclude some reviewers, but also have highly legitimate reasons for not wanting to divulge these reasons to editors. An editor is not a therapist, the submission form is not a confessional, and authors would be wise to not trust in the confidentiality and good judgment of all of the editors that handle all of their submissions. There are lots of fallible editors out there. So if a junior scientist doesn’t offer a great reason, that doesn’t necessarily mean they don’t have one.
Okay, now let’s consider the reasons that editors might have. Here I’m trying to represent a viewpoint that I do not hold, but I’ll try to present them as clearly as possible.
Editor motives for sending manuscripts to non-preferred reviewers
The editor overestimates their capacity to judge if a review is unfair or biased: Good editors don’t take reviews at face value — we need to take into account that reviews have different realms of expertise, and also may come into the review process with different priorities, convictions, and biases. Considering we know who the reviewers are, what they’ve published in the past (and maybe something more about them as human beings), we can contextualize what they say and what they don’t say in the reviews to come to our own evaluation of the manuscript. In short, reviews are advisory, and any decent editor exercises their own judgment. A question that I think is fair to consider is: if a reviewer is intentionally biased against an author, can they be effective at obscuring this from the editor in the form of criticisms that can appear unbiased to a savvy editor? (I think the answer is yes.)
The editor dismisses the veracity or validity of the rationale: I guess it’s possible that an editor simply might not believe the reason provided by an author.
The editor dismisses the validity of the rationale: It’s possible the editor might understand the request to exclude a reviewer, but disagree that the reason is a good one. I imagine this crops up most often when the manuscript contains work that critiques or contradicts work by another author. In this case, I think it’s just fine for a scholar to review a manuscript criticizing their work prior to publication. I think the editor will be need to be well versed to be able to parse the legitimacy of arguments from both sides — and that’s why editors have been selected for the role!
The editor thinks the reasons for picking a particular reviewer supersede the concerns of the author: I guess it’s possible for an editor to believe the person has a valid reason to request that a certain person not review their manuscript, but it’s just that the editor might have other reasons that they feel take precedence over the expressed interests of the author.
The editor assumes that the author has an inadequate reason for their request: This is the one that I’ve seen other editors pull out as an excuse explanation for their choice to send a paper to a non-preferred reviewer. If the authors don’t provide an explanation, and they list multiple people who are well established in the field, then it possible that the editor might conclude that this person is simply fishing to game the system by avoiding what might be well-informed and challenging reviews. Without an explanation, then an assumption is required. Under what circumstances are these assumptions reasonable?
So there the editor’s reasons are, though maybe I’ve missed some.
How should an editor proceed?
I’m not here to tell you what to do, as I work to avoid dispensing general advice. I’m just saying what I think and how I handle (or would handle) situations. Adopt these opinions as you see fit. So, with that caveat out of the way, here’s the first question I always ask myself when faced with one or more non-preferred reviewers listed by authors:
If I don’t send the manuscript to the non-preferred reviewers, is this manuscript still capable of receiving high quality reviews from experts with the knowledge required to identify substantial weaknesses or flaws that should be part of a decision to reject or to revise?
If I can answer “yes” to that question without reservations, then I’m just going to go ahead and honor the request of the authors. If I can get great reviews that would address what needs to be addressed, then why fuss?
On the other hand, what would I do if I think that one of the non-preferred reviewers is necessary for a proper assessment of the manuscript? That’s a stickier situation. I think this scenario is a genuine possibility — in some specialities, there might be only a few people in the world who would be aware of some potential flaws or weaknesses, because of their intimate knowledge of the system. For example, if someone is doing an experiment on the biology of some particular organism, it would make sense that at least one of the reviewers has direct experience working with that particular critter. But sometimes, that is a very short list of people. And those people might be direct competitors, or enemies, or whatnot.
So, then, how would I handle a situation where one of the non-preferred reviewers is one of the necessary reviewers? Well, I think that depends on why they’re a non-preferred reviewer. If there is what I consider to be legitimate reason, then I’d have to honor this, and do my best to perform a robust review in the absence of that reviewer. I would make a point to read the manuscript even more carefully myself, and perhaps solicit a third review to cover what I think are the necessary bases. But I wouldn’t send it to a reviewer if the reason offered by the author is a good one.
But let’s say I think the author asked to exclude a reviewer and I think their reason isn’t a valid one, and at least one of those reviewers is important for a quality review process. What do I do then. Well, I think this is a very rare circumstance — I haven’t found myself in this position yet (though I guess I’ve handled under 200 manuscripts to date, far less than experienced EICs). But, this situation apparently pops up for some editors, who feel like they need to send a manuscript out to an author that was listed as non-preferred. (What reasons would I think are non-valid? It would be because the paper is critical of that person’s prior work. Or if they listed someone without a clear explanation.)
There’s no way I could just go ahead and send out a manuscript to be reviewed by a person listed on non-preferred. I think that’s a betrayal of any trust that an author would put in the editor, the journal, or the academic society running a journal. What I would do is open up a conversation with the author about the review process. I’d either have an editorial assistant or the EIC contact the authors to explain policies on excluding reviewers. They’d explain that the person handling the manuscript (the associate editor or the subject editor or whatever they’re called) may feel the need to submit the manuscript to a non-preferred reviewer if there isn’t a fair reason to exclude them. They then would have the opportunity to volunteer a more appropriate reason to exclude a reviewer, or to withdraw the paper.
Keep in mind I’d only engage in this conversation about non-preferred reviewers if, in my judgment, necessary for a solid evaluation. If this communication is not being done by the person handling the paper, then this would preserve at least some kind of fig leaf of anonymity during review. Though if a reviewer is necessary because of their particular experience or knowledge, then this would presumably come out in the review (and they might just sign it anyway). This is far superior than sending a paper to a person listed as a non-preferred reviewer against the wishes of the author and without their knowledge.
The bottom line is that sometimes authors have very good reasons for asking us to exclude reviews and they may feel that they are put at risk if they divulge their reasons to editors. There are too many bad actors out there, and even experienced editors might not know everybody’s dirty laundry. As a measure of precaution, I honor the wishes of authors, and if circumstances might compel me to consider not honoring this request, then this would never be without the author being aware that it’s a possibility.
I handled a manuscript once where the authors requested only Chinese persons perform the reviews! The authors were also all from China. It is not a China-based journal. I didn’t think that was an appropriate reason.
How rampant is bias against researchers outside of USA and Wester Europe? I suspect this request came in response to fear of that type of bias. I think being aware for that potential is necessary for later addressing it and building ways to prevent it. Interestingly, both more and less transparency are potential ways to address it (e.g. double blind reviewing, or completely open peer review), and I think they are not mutually exclusive.
Is this fairly rare to see non-preferred reviewers listed? More common in my experience is the AE who says that they routinely exclude all of the authors’ list of suggested reviewers. When I respond that that requesting suggested reviewers just so that they can be excluded is a dishonest subterfuge, and is unethical behavior, it’s put a damper on the conviviality of the editorial board social.
I had one encounter in which the author wanted 9 reviewers excluded, and suggested 3 that I considered inappropriate based on their prior collaboration with her. This excluded almost all expertise on her niche topic. Her basis for exclusion from her review article was that academics who had accepted industry funding were tainted, which was a problem because in my field (ecotoxicology), industry funding is sometimes the only game in town. I had a letter drafted telling her that she could either relax her opposition, and I, as editor would consider potential conflicts if the review seemed unduly critical. Otherwise, she could be free to send it elsewhere. But then I found a few knowledgeable reviewers that she had overlooked in her exclusions request.
Just invite someone listed as non-preferred seems improper. In my sub-subdomain, we don’t do a very good job at specifically asking reviewers if they might have conflicts. I’ll take a reviewer with expertise and conflicts any day over one with neither, but conflicts should be acknowledged.
Nice post.
On Mon, May 21, 2018 at 6:01 AM, Small Pond Science wrote:
> Terry McGlynn posted: “Apparently, there are some editors of academic > journals who will readily send manuscripts out to “non-preferred reviewers” > — the specific people that authors specify who they don’t want to receive > the paper for review. I think this is all kinds of messe” >
I have trouble with an editor judging whether a reason is valid enough to exclude a reviewer. Particularly in cases where a grad student or postdoc has been sexually harassed by someone powerful in their field, they may not be comfortable explaining to someone they don’t know and therefor can’t trust that the reason they don’t want Professor X as a reviewer is that he sexually harassed them. They may leave the reason field empty, or offer a vague “personal differences” reason, which an editor might view as an invalid reason for excluding that person as a reviewer.
I wonder how often it is simply carelessness. My family jokes that our field is heritable, because it’s a very small niche and both my parents and I work on similar topics. I publish under my married name and as a result always List both of my parents as non-preferred reviewer’s with the reasoning that they are my parents, despite not having the same last name. More than once both my father and my mother have received the same manuscript of mine for review, and few editors have said more than “whoops” when they were asked why a next of kin relationship wasn’t a good enough reason to not use the non-preferred reviewers. Obviously they are not only using this list, but not even reviewing the rational for including them as non-preferred.
As an Editor I look for people who are on neither the preferred nor non-preferred lists; sometimes difficult, but worth the effort especially if I can get some ECRs involved.
I never list non-preferred reviewers: a colleague of mine has had his manuscripts consistently reviewed by the persons he listed as non-preferred, one of which returned his (signed, to be fair) review with the predictable rejection/no resubmission allowed containing statements as ‘this ms is rubbish”. I think it is deeply dishonest by editors acting that way, to me is a way of cheating authors. It’s better to leave the review to the lottery system and pray that editors do not send the work to those jerks
I was once asked to review an ms despite the authors (who as an aside I’d never heard of) having listed me as a non-preferred reviewer. The editor really wanted my view anyway, because the topic was right in my wheelhouse. And you know what? I really liked the ms! I don’t know why the authors asked that I be excluded, though I have some speculations (the authors may have misread a blog post I wrote on the topic). An anecdote obviously, so your mileage may vary. I guess I’d say that, if you’re thinking of listing Dr. X as a non-preferred reviewer because of some vague sense that Dr. X might not like your ms, well, first of all that’s not a legitimate reason to exclude a reviewer (as Terry notes), and second of all, are you sure Dr. X won’t like your ms?
As a handling editor at Oikos years ago, and now at Am Nat, it was rare for authors to list non-preferred reviewers for the mss I handled. When they did, I usually respected it, except in rare cases where they listed someone whose opinion I really wanted. Hypothetical example: if your ms is presented as the first empirical test of theory X, I’m probably going to want a review from the person who developed theory X, even if you ask that that person be excluded, unless you give me a legitimate reason why they should be excluded. I can only recall one case as an editor in which I asked for a review from someone the author had listed as a non-preferred reviewer.
Re: preferred reviewers, my policy as a handling editor is not to use more than one of them (I think that’s a pretty common policy, though I don’t have data on that). I’m most likely to include one of the authors’ preferred reviewers among the reviewers I choose if (i) it’s someone I might well have chosen anyway, or (ii) the ms is far from my area of greatest expertise, making it more difficult for me to identify potential reviewers.
Functional Ecology recently published some interesting data on how preferred and non-preferred reviewers evaluate mss: https://dynamicecology.wordpress.com/2016/04/11/author-suggested-reviewers-and-their-effects-data-from-functional-ecology/
Shameless self-promotion: here’s my old advice post on suggesting preferred and non-preferred reviewers: https://dynamicecology.wordpress.com/2012/12/19/advice-how-to-suggest-referees/
I agree with Terry’s general argument, and would add that journals should give clear examples of valid or non-valid reasons for excluding reviewers. Beyond that, of course it’s a honor system, but even if we know people abuse the non-preferred reviewer list (e.g. avoid getting the same critical reviewers when they resubmit to another journal), it’s a fairly minor offense on the grand scale of academic misconduct.
I’m a bit puzzled by the comments from editors saying that they try to get reviewers who are not on the preferred reviewers list or never use more than one of them. I thought the whole point of suggesting reviewers is to make finding reviewers easier for editors. And calling them “preferred” and avoiding them for this reasons implies that the authors are suggesting them solely for the reason that they think they will like the paper or review it positively. When suggesting reviewers, I try to think of all the people with whom I do not have any conflict of interest and who would be able to review the paper, whether I think they will like it/review it positively or not. Sometimes I list 5 or 6 or more people in this category. Purposely avoiding asking more than one of the listed suggestions just seems like extra effort for the editor, and sort of defeats the point of asking authors to suggest reviewers.
I agree that editors specifically choosing to avoid most of the recommended reviewers makes that step seem like make-work for authors. Identifying appropriate reviewers, tracking down their current contact information, and filling it all in to the form isn’t hugely onerous, but it can take half an hour. Why ask authors to do a half hour of work on something that won’t be used?
@Anonymous:
“Why ask authors to do a half hour of work on something that won’t be used?”
Speaking for myself, three reasons. First, it’s my job as an editor to pick reviewers. Choosing all the reviewers from among the authors’ preferred reviewers would amount to letting the authors choose their own reviewers. Authors who provide preferred reviewers are doing the same thing as reviewers who provide reviews: giving the editor advice and input, which it is up to the editor to decide whether and how to use.
Second, not all authors are as conscientious as you. My understanding from talking to EiCs is that it’s not rare for some authors to list preferred reviewers with whom they have a conflict of interest, or who are close personal friends, or etc. As an editor, I can’t always tell easily if an author has done this. One easy way for me to limit the potential damage if the author has done this is to not use more than one of the authors’ preferred reviewers.
Third, data from Functional Ecology (linked to in my earlier comment) show that author-preferred reviewers score mss higher than other reviewers, even when there’s no conflict of interest. That’s probably because authors tend to suggest reviewers who work on the ms topic and so are predisposed (on average) to like the ms. But as an editor at a selective journal that aims to publish papers of broad interest to ecologists, I need to know what a broad range of ecologists think of the ms.
So maybe ask for a smaller number of reviewers, then? Some (entirely reputable) journals I’ve submitted to have requested six or seven suggested reviewers. Sometimes that taps out everyone in my subspecialty that I haven’t worked with.
Response rates to review requests are declining, including response rates from author-preferred reviewers. At Functional Ecology, author-suggested reviewers decline review requests just as often as other reviewers. If as an author you would like the editor to be able to get a review from a reviewer you suggest, well, that means you have to suggest several names, not just one or two.
I am an editor and in the journals I work with we do not explicitely ask for non preferred, only for suggestions. Now frequently the suggested reviewers are buddies and upon close examination earlier coauthors or people from the same lab. The “dishonesty” level in that regards is huge. Also their review acceptance rate is not higher than the general reviewer population. Finally, it is sometimes interesting to see how some of the suggested reviewers, in anonymous peer review, actually trash deliberately a paper. You do not know what people do when they are anonymous. Many are equal opportunity yes or nay sayers…. so this preferred non preferred is not that relevant… a bigger issue is that some reviewers NEVER reject a paper, no matter how bad, while I saw some who never recommended publication…..
You want to be outraged, be outraged about that…. an editor with experience can set your paper up to pass or get accepted just based on his experience with the reviewers…
In the case where a manuscript explicitly critiques or disagrees with a prior paper, I believe the editor has an obligation to offer an author of that work an opportunity to review the manuscript. However, this “interested” review should not be anonymous. The editor should wait a reasonable time for this review before sending the manuscript and interested review to independent experts. Depending on the interested review’s contents, the editor may send that review back to the authors to revise or respond before going to independent review.