I went to a bunch of scientific conferences this summer. Four of ’em. I have a smorgasbord of reflections on the whole experience to share with you.
Novels, science, and novel scienceStandard
I was chatting with a friend in a monthly book group. A rare event happened this month: everybody in the group really liked the book. It turns out that that most of the books they read are not well-liked by the group. How does that happen? Well, this is a discriminating group, and there are lot of books on the market; many books aren’t that good.
We speculated about why so many non-good books are sold by publishers. The answer is embedded within the question: those books sell.
Let me overgeneralize and claim that there are two kinds of novels: First, there are those that were brought into the world because the author had a creative vision and really wanted to write the book. Second, there are novels that are written with the purpose of selling and making money. Of course, some visionary works of art also sell well, but many bestselling books aren’t great works of art. (No venn diagram should be required.) Some amazingly great novels don’t sell well, and weren’t created to be sold easily in the marketplace.
Most novels were never intended for greatness. The authors and the publishers know this, but have designed them to be enjoyed and to have the potential to sell well. When someone is shopping for a certain kind of book, then they’ll be able to buy that kind of book. Need a zombie farce? A spy thriller? A late-20s light-hearted romance? I have no problem with people writing and selling books that aren’t great. Books can be a commodity to be manufactured and sold, just like sandwiches or clothing. A book that is designed to sell fits easily fits into a predetermined category, and then does its best to conform to the expectations the category, to deliver to the consumer what was expected.
I think a similar phenomenon happens when we do experiments and write scientific papers.
First, some research happens because the investigators are passionately interested in the science and have a deeply pressing creative urge to solve problems and learn new things.
On the other hand, some research is designed to be sold on the scientific marketplace.
To advance in our careers, we need to sell our science well. The best way to do this, arguably, is to not aspire to do great science. We can sell science by taking the well trod path on theoretical bandwagon, instead of blazing our own paths.
If you want a guarantee that your science will sell well, you need to build your research around questions and theories that are hot at a given moment. If you do a good set of experiments on trendy topic, then you should be able to position your paper well in a well-regarded journal. If you do this a dozen times, then your scientific career is well on its way.
On the other hand, you could choose a topic that you are passionately interested in. You might think that this is an important set of questions that has the potential to be groundbreaking, but you don’t know if other people will feel the same way. You might be choosing to produce research that doesn’t test a theory-of-the-moment, but you think will be of long-term use to researchers in the field for many years to come. However, these kinds of papers might not sell well to high-profile journals.
Just like a novelist attempting to write a great novel instead of one that will sell well, if you are truly attempting to do great science, there is no guarantee that your science will sell. Just like there are all kinds of would-be-great novelists, there are some would-be-great scientists who are not pursuing established theories but are going off in more unexplored directions.
Of course, some science created for the marketplace is also great science, too. But the secrets to creating science that sells, are very different than the secrets to doing great science.
After all, most papers in Science and Nature are easily forgettable, just like the paperbacks for sale at your local chain bookstore.
Update: For the record, y’all, I’m not claiming that I am above doing science to be sold. That’s mostly what I do. I’m just owning that fact. There’s more on this in the comments.
Pretending you planned to test that hypothesis the whole timeStandard
Our scientific papers often harbor a massive silent fiction.
Papers often lead the readership into thinking that the main point of the scientific paper was the main point of the experiment when it was conducted. This is sometimes the case, but in many cases it is a falsehood.
How often is it, when we publish a paper, that we are writing up the very specific set of hypotheses and predictions that we had in mind when we set forth with the project?
Papers might state something like, “We set out to test whether X theory is supported by running this experiment…” However, in many cases, the researchers might not even have had X theory in mind when running the experiment, but were focusing on other theories at the time. In my experience in ecology, it seems to happen all the time.
Having one question, and writing a paper about another question, is perfectly normal. This non-linearity is part of how science works. But we participate in the sham of , “I always meant to conduct this experiment to test this particular question” because that’s simply the format of scientific papers.
Ideas are sold in this manner: “We have a question. We do an experiment. We get an answer.” However, that’s not the way we actually develop our questions and results.
It could be: “I ran an experiment, and I found out something entirely different and unexpected, not tied to any specific prediction of mine. Here it is.”
It somehow is unacceptable to say that you found these results that are of interest, and are sharing and explaining them. If a new finding is a groundbreaking discovery that came from nowhere (like finding a fossil where it was not expected), then you can admit that you just stumbled on it. But if it’s an interesting relationship or support for one idea over an other idea, then you are required to suggest, if not overly state, that you ran the experiment because you wanted to look at that relationship or idea in the first place. Even if it’s untrue. We don’t often lie, but we may mislead. It’s expected of us.
In some cases, the unexpected origin of a finding could be a good narrative for a paper. “I had this idea in mind, but then we found this other thing out which was entirely unrelated. And here it is!” But, we never write papers that way. Maybe it’s because most editors want to trim every word that could be seen as superfluous, but it’s probably more caused by the fact that we need to pretend to our scientific audience that our results are directly tied to our initial questions, because that’s the way that scientists are supposed to work. It would seem less professional, or overly opportunistic, to publish interesting results from an experiment that were not the topic of the experiment.
Let me give you an example from my work. As a part of my dissertation, in the past millennium, I did a big experiment in which I and my assistants collected a few thousand ant colonies, in an experimental framework. It resulted in a mountain of cool data. This is a particularly useful and cool dataset in a few ways, because it has kinds of data that most people typically cannot get, even though they can be broadly informative (There are various kinds of information you get from collecting whole ant colonies that you can’t get otherwise.) There are all kinds of questions that my dataset can be used to ask, that can’t be answered using other approaches.
For example, in one of the taxa in the dataset, the colonies have a variable number of queens. I wanted to test different ideas that might explain environmental factors shaping queen number. This was fine framework to address those questions, even though it wasn’t what I had in mind while running the experiment. But when I wrote the paper, I had to participate in the silly notion that that experiment was designed to understand queen number (the pdf is free on my website and google scholar).
When I ran that experiment, a good while ago, the whole reason was to figure out how environmental conditions shaped the success of an invasive species in its native habitat. That was the one big thing that was deep in my mind while running the experiment. Ironically, that invasive species question has yet to be published from this dataset. The last time I tried to publish that particular paper, the editor accused me of trying to milk out a publication about an invasive species even though it was obvious (to him at least) that that wasn’t even the point of the experiment.
Meanwhile, using the data from the same experiment designed to ask about invasive species, I’ve written about not just queen number, but also species-energy theory, nest movement, resource limitation, and caste theory. I also have a few more in the queue. I’m excited about them all, and they’re all good science. You could accuse me of milking an old project, but I’m also asking questions that haven’t been answered (adequately) and using the best resources available. I’m always working on a new project with new data, but just because this project on invasive species was over many years ago doesn’t mean that I’m going to ignore additional cool discoveries that are found within the same project.
Some new questions I have are best asked by opening up the spreadsheet instead of running a new experiment. Is that so wrong? To some, it sounds wrong, so we need to hide it.
You might be familiar with the chuckles that came from the bit that went around earlier this year, involving Overly Honest Methods. There was a hashtag involved. Overly honest methods are only the tip of the proverbial iceberg about what we’re hiding in our research.
It’s time for #overlyhonesthypotheses.