Experiments can sell your science, even if they’re not going to work

Standard

We typically need manipulative experiments to truly know how a biological system works.

Nevertheless, on most days, I feel that the subculture of ecology suffers from a fetish for manipulative experiments. In some cases, people design experiments that don’t entirely make sense because they know that the reviewers and the community will value that experiment more than observational research. Even if the experiment isn’t really that informative.

I remember seeing a talk from a meeting — from my gosh almost ten years ago! — that was absolutely brilliant. This talk featured a number of manipulative experiments, one of which made little sense, but served the purpose of bumping up perceived rigor. This was a project by Megan Frederickson (now faculty at U. Toronto), and when she gave the talk, the related paper was in press with Nature (though she was humble enough to not have mentioned this in her talk).

image from Alex Wild - alexanderwild.com

image of Allomerus from Alex Wild – alexanderwild.com

Frederickson made a very cool discovery. The mutualistic ants that live inside the plants of the Devil’s Gardens of the Amazon kill off all of the competing plants by poisoning them with formic acid. The ants actually cut into competing plants and inject into them, resulting in death, and over time, a big monospecific stand of the host plant. That’s super cool, but it’s also natural history. You can’t publish that in Nature. Because it has no Experiment.

So Frederickson designed simple experiments to demonstrate that this is indeed a mutualism. One of the things she wanted to show was whether the domatia (cavities that house ants) on the mutualist host plant were the cue for the defense. She wanted to test whether the ants were just killing off the non-host plants because they lacked domatia. As it was put in the Nature paper: “whether [the ant] attacks only plants that are not its host plants and whether the ant uses domatia to recognize its host.”

So how did she do this? She put artificial domatia on non-host plants that were introduced into the Devil’s Gardens. That’s what it says in the paper: “artificial domatia.” So, what are these artificial domatia? The supplementary methods says that they are “foil-covered test tubes partially filled with moist cotton.” In her talk  (which I saw ten years ago!) I recall seeing images of microfuge tubes. Which are just about the right size for these ants.

But if the experiment is testing “whether the ant uses domatia to recognize its host,” then it’s not using the presence of tubes tied to the plant. The sensory world of ants doesn’t work that way. This effectiveness of this manipulation hinges on the notion that ants would consider these domatia to be an acceptable nesting environment. It’s quite clear that these ants are attacking any plant that isn’t the host species, and an experiment was (apparently) needed to show that they weren’t being killed because there were no domatia on the host plant.

I would bet (a beer or sandwich or a gift melon) that anybody who has spent serious time with the nesting biology of ants in tropical rainforests will tell you that cutting off domatia from new plants and tacking on microfuge tubes with moist cotton won’t prevent the ants from doing their normal thing. Artificial nests can definitely work in the field, but plastic or glass tubes with cotton probably won’t do the trick. Regardless, a qualitative understanding of what these animals are doing makes this experiment a bit superfluous.

Tying little tubes onto plants is downright silly. It doesn’t just look silly, but from an experimental standpoint, it’s not an effective manipulation, and it wouldn’t be providing any useful information to those who are most familiar with the system. But — and this is a huge but — this manipulation does satisfy those who need to see the manipulative experiment in order to think that this is an important project. A reciprocal transplant experiment was critical for this project. A transplant with the domatia? That’s downright fancy. Even if it wasn’t necessary, because it’s no surprise that the tubes didn’t protect the plants from attack.

They key result in this study was that the ants kill the foreign plants, because they’re not a member of the right species. And tying on little tubes as a manipulation was the ribbon that made the package that much prettier. It’s quite correct that some reviewer, unfamiliar with the system, might ask, “but maybe they’re just attacking the foreign plants because they’re not offering domatia?” and then the author then could say, ‘Lo, behold, I have run the Experiment.” And the reviewer goes home happy.

Hats off to Frederickson for designing an impactful study. She’s done a lot of work since then and I think it’s all spectacular. As for this 1995 paper, well people have definitely seen it, and google says that it’s been cited 50 times. I think there’s a lot to be learned from looking at how people design manipulations for papers that are accepted to high-profile journals, and how these experiments can convince people that the science is important. What I think and know about ants doesn’t help sell a paper, as it’s what the editors and reviewers want to see that counts when you’re selling science.

After all, we’re not doing science to convince ourselves that the results are legit, but to convince others, especially those who don’t know our systems as well as we do.

6 thoughts on “Experiments can sell your science, even if they’re not going to work

  1. “After all, we’re not doing science to convince ourselves that the results are legit”.

    I think most of my science is exactly this, trying to convince me that my results are legit.

  2. Interesting thoughts Terry. Just to push back a bit though, I bet there are times when people do an experiment to convince/impress others, and it surprises them by coming out differently than they anticipated.

  3. I would agree with the general sentiment that there is a reification of experiment as the best way to do science. It’s certainly true of NSF as well. I’ve been trying to get funding for years of a project that would measure something literally before never observed and important to all kinds of theoretical developments in ecology, but its not an experiment so no luck funding.

  4. This does seem to come from the overwhelming bias of having to do ‘hypothesis driven research’. Noticing something just doesn’t cut it and isn’t considered “real” science. So any way you can show you’re at least trying to experiment is deemed good, even if ridiculous. In the case you cite, I can imagine a number of real experiments that might further knowledge of the system; like does the host plant produce a chemical that IDs it for the ants and they destroy everything else?

Leave a Reply to Brian McGillCancel reply