Preprints are not a standard practice in biology. Nowadays, most papers that get published in peer-reviewed journals were not uploaded to a public preprint server.
Maybe this is changing? It looks like preprints are starting to take off. It’s not clear if this is a wave that will sweep the culture of the field, or just a growing practice among a small subset. Nowadays, the preprint site bioRχiv is now getting more than 1000 papers per month. (I imagine that a lot of these will end up in peer-reviewed venues at some point.)
If you’re not familiar with preprints, here’s the deal, at least from my perspective: Some folks want to share their scientific papers with the world without having to wait for formalized peer review. There are a bunch of organizations that have cropped up to help facilitate this. For physics and some related fields, arΧiv has been doing this for a long time, so that it’s often standard practice to upload a paper there before it goes to review. (By the way, the Χ is the chi symbol, so it’s pronounced “archive.” Cute, eh?) There are a lot of ways that publishing in physics is really different than biology, so it might make sense to me why preprints have long been a thing for them, and not for us, though of course some of these differences are purely cultural.
Preprints are part of the holy hexagon in the faith of “Open Science.” That means there’s no shortage of folks who are glad to evangelize about the wonderfulness of preprints. So I don’t need to rehash this for you in any detail as you can readily find a more detailed case. But a short list might include: letting you share your stuff without having to wait for peer review, allowing good-natured peers (and anybody else) to provide feedback outside of the traditional peer review process, increasing visibility and maybe citations, and establishing priority to prevent scooping (though “open science” folks have often told me that scooping isn’t a substantive issue, when it comes to public data archival). I’m sure I’m missing out on other positives.
Not everybody is jazzed about preprints. Some of this is hashed out in a recent Wired article. I don’t think it’s a bad thing for folks to want to upload their preprints (as long as they aren’t going around telling other people what to do, which, frankly, is not unexpected when it comes to advocates for this kind of stuff). Let me tell you why I’m not bothering with preprints.
- I don’t mind waiting a few months or a couple years for people to see my paper. If there’s anybody who I want to find out about my stuff right away, then I’ll send it to them. I don’t think I’m making the world suffer by waiting a little longer than they’re already waiting.
- I don’t expect that I’ll get more or better peer review by using preprints than using other approaches.
- I want my papers to have a single (and better) version available to the public. Let’s say I have manuscript that I’m ready to submit to a journal. And so I upload the preprint and submit it at the same time. Then, the peer reviews from the journal come back with editorial remarks. And I then make some changes that improve the paper. Then that one will get published. So now, the world has access to this paper that’s not as good? Why would I want this? Can I avoid this by not posting a preprint? Yay!
- I’d prefer a paper to splash than trickle out. If I have a paper that’s interesting and exciting, and then I want to be able to share this once the legit peer-reviewed version comes out.
- I might have fucked up in a paper, and maybe I don’t want to outsource the detection of this fuckup to the entire world? And I might not want to expose my student coauthors to this possibility?
- I think preprints exacerbate the malady of careerism that makes it difficult for junior scientists to treat science like a normal job. To get ahead in academia, you gotta do MORE! FASTER! BIGGER! and it feels like the epitome of this is the need to magically publish your papers before they get published. I think preprint advocates might think that they are a prescription to fix the rat race, but I think it just puts the rats in a more convoluted maze. Po-tay-to, po-tah-to.
- If preprints do become the norm, then there’ll be a ton of crappy preprints with unsubstantiated results and highly flawed experimental designs that wouldn’t get past the editorial process of a legit journal. But I’m sure every single one of you reads every scientific paper from start to finish and comprehensively decides whether the methods were sound and the analyses were robust before citing it.
- If I were to create a list of 100 things that we should be changing about how we do and publish science, preprints wouldn’t be on my list. So until it becomes the norm, I’ll fuss over things that matter more to me.
So, this is why I’m not bothering with it. Maybe I’ll change my mind in a few months or a few years.
Well, I bet you’re in for some interesting scolding from advocates… but I agree with most of your points. Am I bothered if other folks post preprints? Not at all. Am I convinced it’s the wave of the future? Not yet. Am I going to post them myself? Haven’t yet seen the case.
Agree x 8.
I’m in a physical science field where posting to arXiv is standard practice. There’s a bit of an experimental-theoretical divide in the philosophy, but as an experimentalist the standard for me and my colleagues is NOT to post to arXiv until AFTER the paper is accepted. The moment we get the acceptance email from the journal, we submit to arXiv. Why bother at that point, you ask? Mainly because it gets out there faster without the paywall — since all papers in my field are posted to arXiv, I never have to worry that I or someone else at a small university without the big $$$ to pay for infinity journal subscriptions won’t have access to any article in the field. And since everyone reads arXiv, it’s the main way that people see my papers, and the main way that I see my colleagues’ papers, without having to subscribe to alerts from five different journals (I’m in a small field, so there really are only five, plus Science and Nature).
So, I agree with most of your points, but I think it’s weird that in biology the emerging trend is to post before acceptance — I would never want to do that, for all the reasons you mention and more. (Theorists in my field say they like to post before acceptance to pee on ideas and to get feedback from the community — sort of a way of crowd-sourcing the refereeing process.) But I wonder why there’s not a movement to post accepted bio articles to arXiv? Would you be opposed to that?
Lyra, thanks for the context! Sure, I’m all cool with doing that. (I just jailbreak my papers on my own site to serve this function.)
I don’t disagree with your thoughts Terry, the one reason I’ve started to use preprints is to get an open version of my papers out in the world for when I can’t afford to make the actual paper open access.
This does also mean I’m not posting it to the preprint server until its basically done and through peer review, just not typeset yet (and of course, only when that complies with journal rules). So for me that seems a good middle ground between some of the issues you raised, but also wanting my papers to be available to those who want to read them.
Auriel, I haven’t published in every journal, but I think every copyright transfer form said it was okay to upload a copy of the final accepted manuscript (just not typeset) on a server. (And I just jailbreak my papers when they come out on my own site, too).
I’m ‘lukewarm but still drinkable coffee’ level with preprints, myself. Mostly in favor, but with a few reservations.
I’m, as you know, 100% in favor of providing open access versions of paper that will inevitably end up behind paywalls. This is the main motivator. But my other reasons are primarily selfish but rat-race driven- and I’m not saying I agree with the rat race constraints, just that I have to survive under them.
That time I had a paper in review at a NOTORIOUSLY SLOW journal for two years- there were simple points I needed to cite from that paper in follow up studies, and those were not the points that were causing the editorial hold up. I discovered preprints midway through that process and cursed myself for not finding it sooner, but I was also too scared to try, lest I anger the editorial gods who were already putting me through the ringer on this paper (I still maintain the editor’s interpretation was WRONG and oh lord I’m going to die on this hill, aren’t I?).
For the purposes of my open science class, submitting our preprint was VERY important for me to be able to demonstrate that my approach had ‘worked’- as you know, everyone has manuscripts in prep that are basically just made of wishes and dreams and the tears of unicorns. Being able to show that my students had, in fact, written a viable paper was essential in my obtaining funding to teach the course a second time. If we’d waited for peer review, even fast peer review, I would have literally been out of a job.
Also, I’m about to submit a paper to BiorXiv in a few weeks, because this has been a year of starting a lot of new projects rather than wrapping old, so it leaves a hole in my CV without specific record of productivity- understandable but potentially problematic for someone about to start a tenure track job. Putting up the preprint allows me to show that I actually got something done during this time
I tend to think that preprints solve one set of problems while removing the answer to another. There many, sometimes large, problems with traditional peer review – that a small number of peer reviewers may not be a great proxy for paper quality, and which journal they are published in is even worse proxy. There is a need for some proxies for quality and validation, though, and some preprint utopians seem to think that these will just arise on their own, or that everyone (e.g., hiring/tenure committees) can just read all of someone’s work in order to judge the quality. I really think that when rating/review systems for preprints get formalized, they will end up looking a lot more like journals. It might be a mildly less distorted system, though.
Some interesting reactions on twitter —
Olivia Guest points out here (https://twitter.com/o_guest/status/889476487644164096) that feedback and reputation will mainly accrue to preprints from already high profile labs.
And Rubén Rellán-Álvarez (https://twitter.com/rrellanalvarez/status/889455801060012032) notes that this blog post omits any mention of the massive and rampant problems with journals and peer review.
In my field, bioinformatics & genomics, I routinely note a massive disconnect between useful tools/techniques and what gets published when; this partly manifests itself with a big latency between when a tool becomes available and when the publication about a tool becomes available. Preprints help quite a bit with this latency, and, more generally, helps with the problem where competitors slow down publication.
@Titus:
“helps with the problem where competitors slow down publication.”
That’s only an issue in some fields. In ecology, rival labs that race to scoop each other basically don’t exist.
HI Terry- thanks for your interesting perspective.
Part of my answer to your point #3 is that you can update the preprint as you get and respond to comments from peer review (or at meetings).
For me- one of the major benefits to preprints is that it removes the issues of sequencing manuscripts and the delays associated with peer review bogging down collaborators.
I also think its really important for trainees to be able to have a tangible product as they move to the next stage, instead of “I did all this great work but my grad lab is still finishing out a few experiments to get it into a higher profile journal”.
cheers
Ivan
As a recent phd grad who is doing this RIGHT NOW, I must second Ivan’s point. I have my manuscript up as a preprint. I am working through revisions to this manuscript for publication.
A) I will update the preprint if and when it comes out as an improved peer-reviewed publication. This is easy to do.
B) It is an enormous, if only abstract, relief to have a preprint that is out already and that I can send people to while I do revisions. This manuscript took 4 full years to write up, so having it out in this form is a huge benefit for my fragile emotional state.
Otherwise, I appreciate your measured “wait and see” approach to preprints.
With Terry all the way but then I started publishing back in the day when you still submitted papers (3-8 copies depending on the journal) in sturdy envelopes and sometimes with a pre-paid envelope for the Editor to return your scrawled-upon masterpieces :-)
You forgot to mention the paywall issue. For some of us without unlimited access, arXiv has been the standard way to get the top notch, state of the art papers.
And, about your claim of the quality issue: this is a problem everywhere, including predator publishers, and even recognized journals. Even more, preprints do have quality standards, you can not jus publish anything there.
Not that you’ll change your mind, but those are my two cents.
What’s up with preprints? And why I’m bothering with them. http://kamounlab.tumblr.com/post/163409024195/whats-up-with-preprints-and-why-im-bothering
Great comments here and a useful discussion for those considering posting a paper as a preprint. At risk of being repetitive, I just wanted to mention a few points:
Waiting a few months or a couple years for a paper to come out may not be a problem if you already have a job, but it can make a huge difference to a student or postdoc on the job market. Instead of taking your word that you have an awesome paper in the works (the dreaded ‘in prep’ manuscript on a CV), a prospective employer can see for his/herself what you have done. Yes – you can send manuscript drafts to potential employers, but you may not necessarily know a priori who might be interested in seeing your work.
Frankly, I don’t understand this ‘debate’ about pre-prints. If you don’t want to post pre-prints or read pre-prints – don’t do so. However, there are certain situations when pre-prints can be helpful. Let’s say you want to get your work out there ASAP (to help land a job or so a delay in publication doesn’t hold up collaborations). Also, posting pre-prints is a great way to get feedback on your study (the few times I’ve used pre-prints, I’ve received valuable suggestions on how to improve the manuscript).
I would say that nowadays about 50% of the papers I read on a weekly basis are preprints. I do this because some of the more interesting and timely papers come out as preprints well before they are published in a ‘standard’ journal. I’m not a patient person – I want to read about a cool study ASAP and preprints make this possible.
I think these are interesting points. I am also torn with preprints. What do you guys think about method papers that have no method section in the preprint or sequencing papers that share none of their sequencing data? I find that somehow rude as if everyone is allowed to accept but nobody is allowed to question/reanalyze/evaluate the data.
@noahfierer and others arguing that posting your preprints can help you on the job market because search committees don’t take “in prep” ms listings on a CV seriously, but will treat an unreviewed preprint seriously: no, sorry. That’s not how faculty search committees work, at least not mostly, and at least not at the present moment.
Search committee members typically only have time to look at a few publications of the top candidates–the 10-20ish candidates getting at least a phone/skype interview, or seriously in the discussion for an on-campus interview (there are exceptional search committee members who carefully read at least one paper from everybody in the applicant pool, but they’re really rare). Whether you are in that top group depends on many factors, of which “how many preprints you have” is not one, at least not for most search committee members. Replacing a few “in prep” lines on your CV with a few DOIs of bioArxiv preprints is not going to get you an interview you wouldn’t have gotten anyway (or cost you an interview you otherwise would’ve gotten; it won’t make a material difference one way or the other). The reason search committee members don’t care about “in prep” lines on your cv is not (just) that they don’t believe those unpublished mss exist at all. It’s that they lack the time (and in the case of many search committee members, the expertise) to evaluate those mss as a reviewer would. Well, that, and they don’t consider preprints to be accomplishments in the way that a peer-reviewed paper or a successful grant application or an award is.
And even once you’ve made the group of candidates who are competitive enough to get a phone/skype interview or be seriously considered for an on-campus interview, the content of any preprints you have is still not going to be a major factor one way or the other in whether or not you get hired. Yes, search committees care about what you’re currently working on, and what you plan to work on in future. They do want to know what your vision is for your research program and how it fits in with the broader field. And yes, they do want to be confident that you’re doing good research. But a standard application packet already gives them other, quicker ways to evaluate all that besides reading your preprints. Your research statement is a big one. Your reference letters are another. Your conference talks are a third. And if you’re interviewed on campus, your job talk is another (a very important one).
As an aside, search committees mostly don’t care about altmetrics. They mostly don’t care if your preprint (or peer-reviewed paper) has been downloaded X times, or tweeted Y times, or whatever.
Further commentary on how search committees go about their business: https://dynamicecology.wordpress.com/2013/09/19/advice-how-north-american-faculty-position-search-committees-work/
https://dynamicecology.wordpress.com/2015/09/24/guest-post-many-american-universities-use-score-sheets-to-rank-faculty-job-applicants/
My comments concern how things do work; I’m remaining silent on whether I think things should work as they currently do. And it’s of course possible that things may change in future. If things do change, it will surely be in part because of “early adopters” like Noah and others. And of course, there are many other reasons for posting preprints besides “they’ll help me land a faculty position”. My point is a narrow one: at the present moment, posting preprints won’t make a material difference to your odds of getting a faculty position, compared to merely listing “in prep” mss on my CV.
@jeremyfox – Faculty positions are just one of many types of jobs available. If you are looking at potential postdoc candidates, wouldn’t you rather see a finished manuscript instead of just an ‘in review’ paper title listed on a CV (especially if you are looking for someone with a specific skillset)? Likewise, for industry jobs, it can really be helpful to have a finished product out there in the world instead of just relying on lines on a CV to demonstrate what you know or can do.
Publication in standard journals can often take 6 months to a year and that can make a difference between a student or postdoc landing an awesome job and not even being interviewed. Likewise, I would add that in fast moving fields, 6 months to a year can be a long time (e.g. methods in microbial ecology).
@noahfierer:
“f you are looking at potential postdoc candidates, wouldn’t you rather see a finished manuscript instead of just an ‘in review’ paper title listed on a CV ”
I suppose, maybe, though frankly I’d probably lean a lot more heavily on meeting the candidate, letters of reference, etc. But here’s the thing: a candidate who hasn’t posted the preprint could just email it to me if I wanted to see it. If I’m hiring a postdoc and I want to know more about the details of what they’re currently doing than I can glean from their cv, talking to them, their references, their publications, etc., I’ll ask them for any preprints they’ve got. Isn’t that what anyone would do? That’s why many people add the line “preprints available on request” to the section of their cv’s in which they list work that’s in progress or in review.
I agree that a postdoctoral candidate who has some finished mss in review or nearly ready to go into review is a stronger candidate than one who doesn’t have those mss, all else being equal. Good postdocs are “finishers”, and having some finished mss in review or ready to go is one line of evidence (among others, such as peer-reviewed publications) that you’re a finisher. And yes, as you say, a finished ms could be used to demonstrate that you’ve mastered some technical skill. But that has nothing to do with the (non-existent) difference between a postdoctoral candidate who has mss and has posted them on arXiv, and a candidate who has mss and just emails them to me if I want to see them. I had understood the post, and your previous comments, to be about “why (or why not) post finished mss to a preprint server”, not “why it’s good to have finished mss, as opposed to merely claiming to have them on your cv even though you actually don’t”.
@noahfierer:
“If you are looking at potential postdoc candidates, wouldn’t you rather see a finished manuscript instead of just an ‘in review’ paper title listed on a CV (especially if you are looking for someone with a specific skillset)?”
But this is just the thing. A preprint is not necessarily a “finished manuscript” in the same sense that a peer-reviewed and accepted article is in a journal. It’s a manuscript uploaded to a journal’s website in the hopes of being voluntarily, speedily, and rigorously reviewed to at least the same degree as a published manuscript. There is no guarantee that any of the above has happened- that only occurs if the paper is now in “post-review”. If you’re an employer or grant agency looking through the track records of applicants, why should a preprint be viewed as anything other than a manuscript in-prep that just so happens to have a DOI number assigned to it?
For PI-level search committees I agree that it’s not usually relevant.
For postdoc candidates, I will certainly look at preprints. If the candidate has a submitted manuscript which is not in preprint, I will ask for it, but often alas the same attitude which precludes preprinting will also lead the PI of her or his lab to be reluctant to share the manuscript with me. Same story BTW for discussing with interesting students or postdocs at their poster or other opportunities at meetings. “I would love to share but my boss doesn’t want.”
And for me that it a huge advantage of preprints. I share when I’m ready (and yes I should as a researcher be able to judge this for myself), and then there are no further knots in my head or those of my collaborators about whom exactly we can share this with.
Concerning grants, whatever the attitude of the granting agency, if I as a reviewer am reviewing your grant and it include “in preparation” or “under review” or the like, I will Google Scholar it in the hope of finding a preprint. If I find one, I’ll have a look. If I don’t, I’ll dismiss the “in prep/in review/in revision” reference as meaningless.
I cited 3 of my own preprints (since all accepted) in my last big grant submission. It was a huge relief not to worry about space limitations to convince reviewers of the validity of my “preliminary results”, but to be able to put up a figure with a reference that they can check. I don’t care whether the agency took it into account officially or not, I do care to convince the reviewers and to feel that I’m showing the best science I can.
@jeremyfox I don’t think I was particularly clear in my comment, so let me try to explain.
If you already have a candidate in mind and you can email him/her to request more details on an ongoing study – of course it doesn’t make any difference whether a preprint is available or not. However, oftentimes one doesn’t have a specific candidate in mind and a pre-print gives the exposure needed to landed a job. This is not merely hypothetical – I’ve seen this happen a number of times – a PI comes across a pre-print, sees that the lead author has done something cool or has a relevant skill set – the PI then contacts the student to see if he/she wants a position. Without the preprint, the PI would never know about this person or his/her work until many months have passed and the paper comes out in a standard journal. In some respects, I don’t see why posting a preprint is that much different from giving a talk at a large conference – both are means to get exposure for one’s work – the only difference is that a preprint can include far more details about the work than could possibly be conveyed in a 15 min talk (plus pre-prints potentially reach a far broader audience).
@juvandy The question of whether a preprint counts the same as a regular manuscript is irrelevant. You can read the preprint for yourself and then decide if the work is worthwhile. If someone is just counting # of paper titles on a CV, they are probably going about the whole hiring process wrong anyways.
I just wanted to mention a couple things. First, I’m rather boggled by about 50% of the responses to this piece (here in comments, in social media, and in other blogs) which are rebutting things I never said. I’m not advising anybody else to use or not use preprints. I’m just saying, for the time being, it’s not my thing. If you want to upload and read preprints, go on ahead! More power to you. If you’re trying to change my mind, I recommend a little more honey and a little less vinegar. (Though vinegar is fine if it comes along with a serving of chips.)
The only other thing which I’d like to re-emphasize is the paywall issue. All of my papers are available without a paywall, wherever they are published. Anybody using google scholar can get a copy. So the idea that preprint servers help more people get my papers after they’re published isn’t going to change my mind.
Picking up on Terry’s recent comment that many responses to his piece are rebutting things he never said, or mistaking him as an advocate for doing as he does. I find this interesting. The debate about preprints is one of a number of online debates (in and outside of science) in which it seems to be very hard to say “here’s what I think/do and why, but that’s just me, your mileage may vary.” Anecdotally, such statements tend to get read–both by those who think/do the same, and by those who think/do differently–as advocacy. I was thinking last night about why that is. I have a bunch of non-mutually-exclusive hypotheses but no firm view as to their correctness or relative importance.
-It could be that readers of blogs/Twitter/other social media tend to read fast, or just read the opening paragraph, and so tend to miss the part where the author says “but that’s just me, your mileage may vary”.
-Really strong advocates on all sides of a debate might feel that, in order to win, they have to refute anything that might possibly aid the opposing side. So anyone who doesn’t think/do as you do needs to be refuted, even if they say their opinion is purely personal.
-Any detection system that’s tuned to be very sensitive is going to detect a lot of false positives. Advocates on all sides of any online debate may be very alert to any new piece that either supports or disagrees with their views. Causing them to often mistake non-advocacy posts for advocacy. If you’re looking for an argument, you’ll often find one even when it’s not there.
-Advocates on all sides of online debate might see posts like Terry’s as stealth advocacy, or even as disingenuous. As someone trying to argue for a position while also trying to preempt disagreement by saying “your mileage may vary”. I hope this doesn’t ever happen. I think it’s very useful for people to say “here’s what I think/do, but your mileage may vary”. It’s useful for the public sphere to contain a wide range of purely personal opinions and anecdotal experiences, for others to consider (or not) as they see fit. And saying “your mileage may vary” is a humble stance that may serve the useful purpose of reminding all sides in a heated debate that perhaps the debate needn’t be so heated, or perhaps even be a debate at all. That it’s not actually essential for everyone to Pick A Side. That it is not the case that You’re Either For Us Or Your Against Us. (There are of course also plenty of contexts in which it’s useful for people to argue that others should do as they do! I’m not saying that everybody should always and only express purely personal opinions!)
-Advocates on all sides of an issue often want to see some sort of rule other social engineering imposed to force their preferred resolution of the issue. Somebody who says “here’s what I think/do, but your mileage may vary, everybody should just do whatever they want” can be seen as arguing against any imposed rule to resolve the issue. If you think that the aggregate result of people making their own choices is collectively suboptimal for some reason (e.g., tragedy of the commons; contextual factors that unfairly give some people more/different choices than others), you’re going to see anyone who just says “everyone should make their own choices” as part of the problem.
-Advocates on all sides might be so used to arguing with others about their opposing views that they may just tend to forget that some people don’t have anything more than weak or purely-personal opinions on the issue. Causing them to misread those people. Often, our understanding of an author’s meaning isn’t based solely on the text itself, but also on our implicit understanding of “where the author is coming from”.
-It’s perhaps become so common online to use personal anecdotal experiences to buttress arguments for broader conclusions that anyone who shares their personal anecdotal experiences without trying to buttress any broader argument is apt to be misread.
-Here’s the most interesting possibility to me, because it’s a trap I sometimes fall into (such as earlier on this thread!). Even if you get that somebody has their own reasons for thinking/doing X, and you’re fine with others doing X, you might be bothered if somebody’s reasons for thinking/doing X seem to not make sense or to be based on false premises. That’s what bugged me about Noah’s suggestion above that one reason to post preprints is to help yourself on the job market. I couldn’t imagine why anyone would think that, and it bugged me! Even though I’m fine with people posting preprints if that’s what they want to do. (and thank you to Noah for clarifying what he meant; apologies for not asking for clarification first and instead jumping directly to criticism of what I thought he meant). Probably, if someone thinks/does something you’re fine with, and isn’t advocating for anyone else to think/do as they do, you should usually just let it go if their reasons for thinking/doing that thing seem inscrutable or based on false premises. Possibly, people who tend to argue about things online also tend to want both themselves and others to hold views that are internally consistent and “rational”.
I’m also interested to see that, in the cart above, the takeoff in preprint posting is all due to a takeoff at bioarXiv. I’d be curious to hear hypotheses for why. I have no idea, I don’t know enough about the various preprint repositories.