Sigh. When the media runs away with a science story, it really goes whole hog.
Coffee’s Bonus Side Effect: Hours Of Hallucinogenic Fun
Jezebel. I generally really like your site, and you often look at mainstream science reporting with a suitably skeptical eye. I’m going to assume this was covered ironically. It certainly wouldn’t surprise me if it was.
And, I mean, the article itself was from the Daily Mail! Oh my.
But, you know. It’s coffee. And we all love coffee. Or at least, I certainly do. And it’s HALLUCINATIONS! All CRAZY LIKE! Coffee, it’s just like LSD! Only it really, really isn’t.
Le sigh. Le twitch.
And then, I read the article itself. And I gotta put on my ranty pants. This paper was rage inducing in a way that I don’t get very often. You ready? Much capitalization for emphasis ahead.
Crowe, et al. “The effect of caffeine and stress on auditory hallucinations in a non-clinical sample” Personality and Individual Differences, 2011.
So, I know that the study was looking at the effects of high caffeine intake on auditory hallucination, using the famous “White Christmas” paradigm. This is where you play a pile of white noise at a person, and tell them that at some point they MAY hear Bing Crosby singing snatches of “White Christmas” in it. If they do, press this button. The issue is that the Bing Crosby clips are never played, so every time you hit the button, you’re wrong. If you’re hitting the button more, it probably means you’re hearing things. The paradigm itself has…problems. The biggest one is that it’s really SUGGESTIBLE. Just hearing that you might hear Bing Crosby makes you MORE LIKELY to think you hear him. It’s not really the best paradigm here, but I’ll get to that later.
But, well, it’s what they had.
The study was looking at the effects of stress and caffeine intake on auditory hallucinations in this White Christmas paradigm. Why stress? And why hallucinations? The idea here is that people who have schizophrenia exhibit auditory hallucinations most commonly, and that high stress is often the trigger for those who suffer from schizophrenia. Not only that, people with schizophrenia tend to drink a lot of coffee. I…think this is a pretty weak link right here. Yes, people with schizophrenia drink more coffee, but they also do a lot of other things, so stating a “lin kbetween high levels of stress and psychosis, and caffeine was found to correlate with hallucination proneness” seems…well. Correlation is not causation.
But hey, caffeine, stress, auditory hallucinations. I just kind of assumed that they would take a bunch of people with no history of any mental illness, give them varying doses of coffee (probably actually giving them a dose of caffeine in pill form, something which I have actually done as a participant in a study), expose them to varying amounts of stress (in the lab this is often done using math tests or public speaking, which I have also done, Sci was quite the lab rat), and then test them in the White Christmas paradigm. That way, you’d see if the caffeine doses themselves affect how people respond to stress and how THAT ends up affecting the White Christmas paradigm responses. This seems like the really logical thing to do here, so I start scrolling down to the data…
Wait. Wait. Stop right there. This is SELF REPORTED?! Both the stress and the caffeine intake were SELF-REPORTED?! And then they gave them the tests without giving the participants ANY CAFFEINE OR PLACEBO?!?
Sure enough. With sinking heart, I realized that they did NOT do the study which I had blithely assumed had been done. No. They gave questionnaires to 92 people and asked them how much caffeine they drank, and how much stress they had been under in the past YEAR. And then they just…gave them the test.
Really, this just doesn’t make much sense to me. Why would you just give people a test without controlling for how much caffeine was in the system? Or how much stress? Just because they had a stressful year doesn’t mean they are stressed NOW. How can you even conclude that CAFFEINE and STRESS are causing the effects when you didn’t even check for other things that might be going on? Not only that, one person’s “two cups” of coffee are not another’s “two cups” of coffee. Sure, Sci has two cups, but they are the size of my HEAD. And one person’s stressful year of hard college classes is not another person’s battle with cancer. There’s no control here for how MUCH stress or how MUCH coffee.
So I’m already not optimistic. They took the questionnaires and divvied them up into four groups, low stress – low caffeine, low stress – high caffeine, high stress – low caffeine, and high stress – high caffeine. They they look at how many times the participants hit false positives in the White Christmas paradigm:
Ok. On the far left you have low caffeine, low stress, on the far right you have high caffeine, high stress, etc. It doesn’t look like much, but if you turn your head and squint it looks like the high caffeine, high stress group has a larger number of erroneous responses than any of the other conditions, which the authors suggest is because stress and caffeine have a combined effect on auditory hallucinations. I went to the results section to look at the stats, figuring maybe the gigantic confidence intervals there were balanced out by stats or something.
Significant at p=0.046. I mean, that’s…significant. But barely. VERY barely. This is probably a result of the high variability, which is more than partially due to the fact that they had low numbers in the high caffeine conditions. If I had a result like this, with n=16ish in my high caffeine groups and n=35ish in my low caffeine groups, I’d be out recruiting in coffee shops on the double. Get those error bars down, and double your high caffeine numbers.
But…maybe funding ran out or something. Stuff happens. I thought maybe they’d taken more data while they were at it, there was another graph.
Wait. WAIT. The ONLY other graph is one on “estimated marginal means”?! YOUR ONLY OTHER GRAPH IS “ESTIMATED”?! I know it’s modeling based on correlations with personality factors taken during the questionnaire, but…ok I have no excuse for this.
This was the point where Sci started ripping her hair out.
All they showed was a CORRELATION between caffeine use and responses in the White Christmas study. And correlation is not causation. And it wasn’t even a particularly large correlation, hampered by low caffeine-condition numbers, and the fact that there was NO CONTROLLED CAFFEINE DOSE. Or a controlled stress dose! They didn’t even state that they’d conducted the studies at the same time in the morning, or asked if everyone had just taken their daily dose!
So here’s the deal. There MIGHT be an increase in auditory hallucinations in response to high dose caffeine. Or there might not. But this study did not address that. And it certainly didn’t adequately address the issues they spent their whole introduction on: the issues of schizophrenia. People with schizophrenia DO often experience auditory hallucinations, but there are no studies on how this is affected by caffeine in these individuals. Further, the White Christmas paradigm has other issues. I mentioned before that it’s highly open to suggestion. But also, the people are pressing a button. There are strong ties between caffeine dose and anxiety, and if you give someone a high dose of caffeine, they may well end up shaky and “trigger happy” on the button. Even Sci gets a little squirrelly after 5 cups! Of course, this would be easier to see if they took measures of anxiety and, I don’t know, gave the people a measured dose of caffeine before issuing the test. There’s ALSO a high correlation between STRESS and anxiety, which could produce the same effects.
Problems with the White Christmas paradigm aside, there’s another test they could have done, not related to hallucinations, which could have evaluated whether people under high doses of caffeine are experiencing symptoms related to schizophrenia. That test is called prepulse inhibition, and it is easy and safe to perform it on humans. You put a person in a quiet room, hooked up to some leads to monitor movement. Have them sit still. Play a REALLY LOUD tone at them suddenly. They’ll leap out of their skin. But, if you play a softer tone FIRST, before you play the louder tone, most people will not jump as much. This is called prepulse inhibition. And schizophrenics have impairments in it. Even in the presence of the softer tone, they will often jump just as hard for the loud one. This would be a relatively easy, certainly non-invasive experiment, and could confirm whether the high dose caffeine was eliciting symptoms that are schizophrenia-like.
I could go into it further (they looked at social desirability and creativity measures and other factors, which I could probably wax just as snarky about), but I think this is enough. If you hear tell of people having heard on the internet that caffeine causes hallucinations, I think you know enough now to tell them…probably not. And if it does, we’re going to need another study to prove it.
S.F. Crowe, J. Barota, S. Caldowa, J. D’Aspromontea, J. Dell’Orsoa, A. Di Clementea, K. Hansona, M. Kelletta, S. Makhlotaa, B. McIvora, L. McKenziea, R. Normana, A. Thirua, M. Twyeroulda and S. Sapegaa (2011). The effect of caffeine and stress on auditory hallucinations in a non-clinical sample Personality and Individual Differences, 50 (5), 626-630 : 10.1016/j.paid.2010.12.007
