I’ve noticed that junior scientists tend to be really picky about conflicts of interest, whereas senior scientists don’t tend to be sticklers.
I don’t think this is necessarily nefarious. I think senior scientists have come to terms with having some kind of history with a lot of people in their field. For example, let’s say I’m looking for reviewers of a manuscript. A doctoral candidate or a postdoc might express concern that they coauthored a paper with one of the middle authors of the manuscript a few years back. A more senior researcher probably wouldn’t even blink at that relationship as a potential conflict of interest. It’s a small world.
That said, some people (independent of career stage) are less inclined to volunteer conflicts of interest when reviewing the work of their peers. There clearly are some people out there who have few qualms about reviewing work of people who they have close relationships with, and also recommending friends to review their own work. Other people take the guidelines for conflict of interest a lot more seriously. Sometimes, the person in charge of a review process (for a manuscript or a grant or an award or whatever) won’t be positioned to know whether these conflicts exist and sometimes it’s hard to tell.
The funny thing is we’re expected to demur from reviewing when we have a close positive relationship with someone, but not when anybody has a robust negative relationship with you. So if you happen to be cursed with an academic enemy, then they’re not expected to say “no” when asked to review your manuscript, but if there’s someone you are great friends with and publish together all of the time, they’re expected to volunteer such a connection and say “no.” Of course, if you have someone who is out to get you (and once in a while, there really are such people), you can list them as as a non-preferred reviewer and an ethical editor will make sure that they won’t see it. But this approach has its limits, because when someone is out to get you, you might not be aware of who they are.
There are also some other conflicts of interest that rarely get discussed, but is something we should talk about. For example, what about when you are asked to review the work of someone who you know is a malevolent or exploitative supervisor? What if you’re asked to review the manuscript of someone in your field who sexually harassed you, or someone you work with? We haven’t established any community norms for such circumstances.
Another generally awkward thing about conflicts of interest during peer review is being colleagues and/or friends with the people whose work you’re reviewing. I mean, there are a lot of people out there who I’ve worked with in some way. We’ve collaborated, or have been a student on a field course I’ve taught, or maybe on a collecting trip together, or we’ve had plenty of socializing time. What’s the difference between a professional colleague and a friend, and a friend and a close friend, and where is a line that defines where the conflict of interest lies?
I think NSF does peer review really well, and they’ve got a very unambiguous set of criteria about what relationships constitute a conflict of interest. There are specific guidelines for what constitutes a “recent collaborator,” which I think vary by directorate as well. These are pretty solid guidelines, but they still don’t address the fact that there are personal relationships that don’t fall under any of these categories.
There are some people out there who I haven’t collaborated with recently who I consider to be good friends, and who are working closely enough in my field that I might end up being asked to review their stuff. I am sure that’s a real situation for lots of senior scientists. The guideline, “If you feel that it’s a conflict, then it’s a conflict” goes a long way. But if we all were asked to not review the work of people who we personally know well and have positive feelings for, then the cogs of peer review would get jammed up pretty quickly.
I don’t think we have a massive peer review crisis in science — and I don’t think I’m giving preferential reviews to anybody on the basis of prior relationships. But I am frustrated that there is essentially an uneven playing field — that once you’ve been in the game for a while, then your professional network is comprised of a lot of people who have good feelings about you and are may be more likely to give you the benefit of the doubt in the review process. Meanwhile, the review process doesn’t have such an environment for people shifting into new fields, nor for junior scholars who are without the safeguard of a senior author with lots of buddies.
There’s definitely the old boys’ network in some sub-subdisciplines, that keeps new people out. But even in the absence of that kind of situation, there still is a network of men and women who’ve been around for a while. The guidelines for conflicts of interest don’t have anything to say about this, and I don’t see a simple way forward.
One proactive step would be use to use double-blind review. After all, most scientists think that double-blind is best. While reviewers often claim that they know who the authors are, there is a big difference between guessing and knowing. And when they do guess, they’re as wrong as often as they are right. In some subfields, it’s only a rarity that the reviewers could identify the authors. Single-blind review benefits people with seniority, after all.
[By the way, I should add that I haven’t had any recent review experience that caused me to write this post. It’s just been a set of ideas in my queue for a long while, and I’ve finally got around to writing the post.]
Or solved by signed reviews. We pretend this is difficult or disadvantageous to new faculty, but there’s no evidence, and it just takes Editors who act properly, as Editors.
Signing a review doesn’t make it any less favorable.
Nice post. I’m not surprised people prefer double-blind reviewing, but would be interested to hear any counter-argument. Does anyone say it’s better for reviewers to see the authors’ names? I can see that in some cases it might be, for example because someone’s reputation and track record (e.g. yours) could boost the plausibility of a daring new hypothesis or paradigm shifter. Would universal double-blind reviewing result in more timid work?
“Does anyone say it’s better for reviewers to see the authors’ names? I can see that in some cases it might be, for example because someone’s reputation and track record (e.g. yours) could boost the plausibility of a daring new hypothesis or paradigm shifter.”
1) If someone wants to advertise their new theory and support it with their scientific weight, there are already plenty of avenues outside research papers to do it. Anyway, the anecdotal evidence suggests that paradigm shifters more often come from less known or new people and do get jammed in peer review.
2) The only case when it may be better to the authors’ names is when the authors have a well-known track record of doing bad science. But then again, there are plenty of cases when this is done by prominent scientists and no one dares to openly object them or say anything.
3) It may be fun to also make peer-review options when the identity of authors is hidden but the identity of reviewers is known.
“you can list them as as a non-preferred reviewer and an ethical editor will make sure that they won’t see it.”
I’ve heard editors say that they absolutely WILL send manuscripts to non-preferred reviewers, “to see what’s going on there.”
In my opinion it kind of depends. If I write a paper massively critical of somebody else’s work, listing them as non-preferred shouldn’t get me off the hook for them having a chance to poke holes in my work. But the editors should take such reviews with a critical eye. I speak from experience. I wrote several papers critical of neutral theory very early on. I’ve always assumed Hubbell was a reviewer of some of those (and they get all got at least one very negative review and got published anyway). And Hubbell should have been a reviewer. And the editor got what they wanted and made their own independent judgment. It all worked pretty well. And science was better off for it.
On the other hand if I ask for somebody as non-preferred and state the reason is that there is a long history of conflict at more than a scientific level, then it shouldn’t be sent to them “just to see what’s going on there”.
In Terry’s original mention of this it was more of the latter case than the former and I agree that is not ethical.
In a majority of cases that I am aware of (about a dozen), editors have intentionally sent the manuscript to “non-preferred reviewers” for review. I guess the goal is to get a good critical review, or something of that nature.
That’s interesting, and if true a bit disturbing.
As an associate editor for three journals a and a couple of hundred papers, and now as an editor-in-chief for thousands of papers, my experience is in the vast majority of cases non-preferred don’t receive the paper. And the exception is usually (although not always) when the author is in a certain sense “gaming the system” by trying to exclude an opinion purely because they know it will disagree scientifically. Deeper personal conflicts are usually (almost always) respected, and when not the review is usually treated with a very healthy dose of skepticism.
But I’m sure practices vary across journals and across fields.
Ya, the reason the authors list for designating someone as a ‘non-preferred reviewer’ (which many journals ask for now) seems important. If the authors say ‘so-and-so is biased against viewpoint X’, then I think the editor has the right to make their own judgement, and in many cases it may be appropriate to go with that reviewer (as Brian’s case with possibly-Steve-Hubbell exemplifies well). But if the author says something to the effect of ‘so and so has made it clear that they have a personal beef with me’, they should not review. Glad to hear that that is mostly standard practice.