“Open Science” is an aggregation of many things. As a concept, it’s a single movement. The policy changes necessary for more Open Science, however, are a conglomerate of unrelated parts.

I appreciate, and support, the prevailing philosophy of Open Science: “the movement to make scientific research, data and dissemination accessible to all levels of an inquiring society.” Transparency is often, though not always, good.

Open Science is united by an aspirational concept. I’m not going to openly advocate for such a broad idea, because without spelling out all of the downstream consequences, I’d be asking others to take it on faith. How do all of the pieces of Open Science connect to one another in a useful manner?

Please humor this faith-based analogy: The Catholic Church has a laundry list of things that they say are good and bad. If you’re Catholic, you’re supposed to not use contraception, you should contribute money to worthy causes, you’re supposed to go to church on Sundays, you shouldn’t murder people, you shouldn’t have sex before you get married, are not supposed to eat more than one full meal nor the meat of tetrapods on certain Fridays, and not have sex with a person of the same gender. What do all those things have in common with one another? Well, those are teachings of the Catholic Church, derived from a set of cultural traditions. It’s possible to be a good Catholic without doing everything the church says. (At least I hope so, because contraceptive use is the norm among Catholics.)

Open Science is not a religion. Open Science does seem to share one property of organized religion: there are a set of recommended practices that are derived from a specific worldview. If you’re doing Open Science — at least, following the tenets of some advocates — you’re expected to archive your data publicly, perform your statistics using software that isn’t purchased from a private company, refuse to review for certain journals, sign your name to your peer reviews, not publish in journals that charge subscription fees, and not require students to pay for textbooks. All of these behavioral expectations flow from the shared doctrine of public accessibility. These are all things I’ve been asked (and sometimes even demanded) to do in the name of Open Science. There are many people who don’t follow all of these practices, but I imagine still can be viewed as good practitioners of Open Science.

A lot of people (albeit a very small minority of practicing scientists) claim to be advocates of Open Science. (I follow a lot of wonderful, non-demandy, people on twitter who mention this in their profile descriptions.) What does being an Open Science advocate mean, on an operational basis? What aspects of open science do they advocate for? All of them?

From where I am viewing science, things like choice-of-software, textbook-ordering, signing-your-reviews, journal choice and public-data-archiving all are very different from one another, and the cost/benefit tradeoffs for each manifestation of Open Science are individualized for everybody.

The wikipedia page for Open Science (which is not overtly in dispute) is comprised of six distinct “principles.” At least at the moment of this writing. Let’s consider each one. I’ll italicize my strawman jingoistic facsimile of the arguments that have been presented to me, which clearly do not represent all Open Science people.

1. Open Educational Resources

Students and the public shouldn’t have to pay to learn about science! Textbooks should be free! Professors teaching science classes shouldn’t make students pay for textbooks!

Nuance: In some disciplines, instructors can put together a great course using non-textbook resources. In other courses, it’s not practicable for students to accomplish the expected learning outcomes without a mass-market textbook. Yes, there are some free textbooks, but in many cases, you get what you pay for. And not every professor has the luxury of time of writing their own textbook. There are a lot of great reasons to not have a pricey textbook for a course, but there are often a lot of constraints and extenuating circumstances.

2. Open Access Publication

Science wants to be free! The public has a right to all scientific results immediately, particularly those funded by the public! It’s absolutely horrible that anybody should ever have to pay or be inconvenienced to get a copy of a scientific article!

Nuance: Junior scientists looking to stay employed and advance their careers are evaluated by their ability to play the prestige game in science, and some of the most prestigious places to publish your research are ones that are not openly available to the public. The funding model for academic journals is in the middle of a collapse and someone, somewhere has to pay for the publication of research. Who’s it going to be? The researchers? Subscribers? The funders? The public? Simply telling scientists to not publish in non-Open Access journals is overlooking some critical variables that affect their career and well-being.

3. Open Peer Review

Scientists have a right to know who is evaluating their work! When people and their work get unfairly trashed in review, this sucks and it harms science and people! Junior scientists must not punished by others who squash their work under the cover of anonymity!

Nuance: There are a lot of good reasons why reviewers might be wise to remain anonymous. A reviewer can experience retaliation down the road, and knowing the identity of a senior scientist that unfairly trashes the work of a junior scientist won’t magically make the situation fair and equitable. Double-blind review, while even less open than the current system, is performed by a small fraction of science journals and this seems better.

4. Open Methodology

Everything you do should be able to be replicated by anybody! Whenever you science something, not being transparent is abdicating a responsibility to the community!

Nuance: Of course sharing how and what you did is critical, but depending on what the precise methods are, there are obvious incentives for some labs to not share the details of their methods if they are proprietary and the result of a substantial investment. For example, if a lab doesn’t share a published cell line, that’s not only jerky but potentially academic misconduct or even illegal. That said, in the current rewards system in science, the motive to not share is pretty obvious. One route would be to shame folks into sharing, but another route would be to advocate for systems that facilitate sharing without ignoring the realities of the incentive system in science. Transparency about methods is a foundation of science. But when I am criticized for using JMP instead of R for some analyses because the code isn’t accessible, that indeed chaps my hide, which leads us to number five…

5. Open Source (software)

If you can’t see the code, how can you know whether the results are legit? Nobody should be expected to pay money just to run analyses on data! How can science be reproducible unless you can run the same analyses as other people for free? If you write a program to analyze your data, everybody else should be able to run it for free!

Nuance: “R is free, but it’s not cheap.” The use of R is facilitated by a community where others are using it, and many scientists are not in such a position. Much of science has been built on proprietary software, with the full approval of the academic community. Transitions take time. If there is a piece of software which is scientifically valid, and a scientist chooses to spend money on it an run an analysis through that way because it’s the most efficient way to get it done, this isn’t necessarily a bad thing.

6. Open Data

If you don’t upload your raw data to a publicly accessible database, you’re slowing down science and your work isn’t verifiable! Scientists don’t own their data, data are in the commons and want to be free! If you’re not publishing your data, you’re being selfish! Being altruistic with your data will be in your own interest when people see that you’re being altruistic! Barely anybody ever gets scooped by their own data anyway!

Nuance: Change happens slowly, and a lot of people have designed research projects with the notion that they’ll have exclusive use of their data for at least some future period of time. The distribution of the benefits and costs of open data are inequitable. I’ve said most of what I want to say about this already.

[end of rebuttal to strawman arguments]

For most of these big issues, it’s pretty murky about how to make actual constructive change.

I want all papers to be freely available. I’d love to be able to immediately download the data from every paper I read. I find it a huge problem that my low-income students are required to spend too much money on textbooks. I think bias during peer review of manuscripts and grant proposals is bigger problem than most people realize. And when an analysis can’t be transparently replicated, that doesn’t help advance science. But I’m not excited about aggregating these issues together under a single movement. Each issue, just on its own, has huge hurdles. By stacking these hurdles on top of one another, it’s not going to really encourage anybody to make the jump.

I feel that if I were to advocate for such a broad list of policies, I’d worry about overlooking the variety of tradeoffs and costs experienced by our diverse community. To me, “open science” is not the ultimate goal. I’m working toward an inclusive system in which everybody can do science, that maximizes the ability to collaborate and make new discoveries. In a world free of bias, that has well-funded science, and one in which people don’t take advantage of one another, then Open Science clearly would help advance science. But, we’re dealing with actual human beings. Which means that there is bias, and there are opportunists, and there’s not enough money going towards science. And if we just open things up, then some people are going to get screwed over more than others.

While science isn’t “broken,” our community does need to revise how we handle some major challenges, including credit for work, publication funding models, bias in peer review, mechanisms of data sharing, the development of educational resources, and software that doesn’t make you want to bash your head in with a ball-peen hammer.

What it will take to develop new journal funding models has very little to do with how we conduct peer review, which has little to do with how we make data available to the public, or what software we use.

Open Science advocates aren’t going to get much traction for Open Access journals and data archival and open review of preprints and non-anonymous peer review and all that other stuff by lumping them all together. The people who don’t want to sign reviews have separate issues from people who can’t always publish OA and are different from the people who are running analyses without sharing their code who are thinking about why they need to assign a certain textbook for their class. Changing these people requires a change in the environment. And it’s not like there’s a single thermostat that Open Science advocacy can adjust to help people sign reviews, publish in OA, drop mainstream textbooks, and so on.

When people give money to worthy causes, it’s probably not because the Catholic Church told them to. If someone goes vegetarian, it’s probably not because not eating meat is forbidden by the church at certain times. Likewise, if someone opts to publish in an open access journal, or develop their own teaching materials instead of using an expensive textbook, it’s not because they’re following an Open Science movement. They’re just making decisions that work for them in their own context.

Advocating a principle doesn’t necessarily change the environment for people in a way that helps them become practitioners. For example, when I’m deciding which journal to submit to, the welfare of my students and myself will trump the aspirational goals of Open Science.

There are a lot of Open Science advocates who are engaging in real advocacy and education. Friends of mine are training students to develop workflows so that data, code, and products are openly available as a natural consequence of doing science. (I get the opportunity to chat with such a group this week!) Other people are working with academic societies to adopt policies to reduce bias, and to identify funding models that make papers more accessible. I’d like to think that I’m pulling more than my own weight in the move toward public data archival for published papers.

So yes, advocating for policies associated with Open Science is a real thing, and advocacy makes a difference. But to advocate for change requires specific advocacy for specific changes. Arguing for “Open Science” as a scientist is like arguing for “more freedom” and “democracy” as a politician. Effective politicians might help freedom and democracy like letting people vote online and fixing absurd mandatory sentencing guidelines that put people in prison for minor drug-related offenses. Effective Open Science advocates are building tools like openstax, creating mandatory data archiving policies that allow for data embargoes with reasonable durations, and requiring gender and ethnicity bias training for faculty job search committees. There are effective Open Science advocates. And many of them have never have heard of “Open Science” as a thing.

---

As a postscript, I’d like to note that some of the moves toward “open science” might put some people at greater risk than others. A lot of us are working to diversify science and make it more inclusive (including many Open Science advocates). For scientists who are marginalized, transparency can quickly lead to marginalization. If a junior scientist who is a member of an underrepresented minority signs her peer reviews, this can have radically different consequences than for a white male scientist. If postdocs publish important papers in PLOS One — or if their PI makes this decision for them — that could torpedo a fledgling career. Just because it works for some people, it doesn’t mean everybody else is experiencing the same tradeoff. These are complex issues, and changes will have different effects on different people. Progress requires us to reckon with this complexity.