There might not be a phrase that annoys me more than “trust the science.” First of all, science is a system of inquiry that’s specifically designed to avoid the pitfalls of belief. Trusting in science is an insult to science. Science is the opposite of trust.
More than that, though, science as an industry is rife with bias and corruption and bad methodology. A large percentage of published studies are total garbage. This week’s sciku draws on one example.
First of all, the press release is titled “Team describes science-based hiccups intervention.” That phrase, “science-based,” foretells the high degree of truthiness we’re about to experience. But it’s worth understanding just how bad this is, so let’s take the time to break it down. And don’t forget this is a study in JAMA Network Open, a “peer-reviewed journal” with an impact factor of 5 (which is pretty good) under the famous JAMA brand.
It turns out that the hiccup intervention is a straw. Not just any straw, but the “HiccAway,” a special straw with its own website that you can also find on Amazon for $14.99 (available July 13th). It’s patented by one of the authors hidden in the paper’s “et al.”
The straw features a “pressure valve,” also known as a small hole, which makes it difficult to suck. Forceful suction, we’re told, induces contraction of the diaphragm, closes the epiglottis, and stimulates the phrenic and vagus nerves. Doesn’t that sound sciency?
Of course there is no mention of the fact that drinking through a 1 cent cocktail straw would do the same thing.
To study the effectiveness of the expensive straw, a team of researchers sent them to 674 participants who had joined their Kickstarter campaign, which raised $60,649, then followed up with an online survey. Of those 674 kickstarters, 249 ended up providing consent and completing the survey after receiving their straws.
This is called self-selection bias, and builds a positive outcome into the study by methodology alone. Who is going to take the time to give consent and participate in the study? The ones who thought the straw worked, of course. If you tried it and thought it was junk, would you waste more of your time on a survey, or would you just throw it away and forget about it?
Unsurprisingly, of the biased sample of participants, 92% found it to be effective. But what does “effective” mean? Here, effective is measured on a scale of 1 to 5, compared to home remedies.
Remember how participants were recruited through a Kickstarter campaign? That Kickstarter campaign video specifically calls home remedies “silly tricks” that “rarely work,” and says that HiccAway is a “guaranteed,” “fool-proof way” to get rid of hiccups created by a doctor. There’s even a cartoon image of the doctor in his white coat, in case the wording alone isn’t enough to induce the white coat effect.
So the participants are not only self-selected TWICE—once for being Kickstarter funders and again for continuing to the survey—but have already been biased to vote for the “guaranteed cure” (5) rather than the silly tricks (1).
Here’s the kicker (no pun intended): the discussion section acknowledges the limitations of the study and the need for a controlled trial comparing the effectiveness of HiccAway to a sham device, but: “The challenge is developing something that resembles [HiccAway] but doesn’t work, Dr. Seifi said.”
Yes, what makes a real trial difficult is that it’s challenging to develop a fake device that doesn’t work. Because it’s just a straw.
How does an article like this get published in JAMA Network Open?
JAMA Network Open is an open access, pay-to-play journal, so you submit your paper, and then pay $3,000 for them to publish it. Roughly one in three submitted papers are published. I’d love to see the other two.
Imagine if a poetry journal accepted 30% of submissions, and then charged the author $3,000. We would call that a vanity scam. Here, it’s just good business to be proven in JAMA, coming soon to a grocery store near you.
In the meantime, we can just invest in a pack of cocktail straws.
I don’t mean to pick on the HiccAway, or imply that this is uncommon—the trouble is, it’s not. I was just looking for something to write my Sunday haiku about, and this cracked me up last night.
we’ve finally
found the cure for—
hiccup