Via Robert VerBruggen, I learn today of a new study about Naloxone, the anti-overdose drug being used to combat the opioid crisis. The authors look at overdose deaths in states before and after they adopted laws making Naloxone more easily available, and they conclude that Naloxone doesn’t help. In fact, it might even increase overdose deaths. The mechanism is quite plausible: if you know that Naloxone is around, you’re more likely to take opioids without worrying about overdosing. So Naloxone increases the number of people using opioids, but then saves some of the ones who overdose. In the end, it’s a wash—or maybe worse.
But I’m skeptical of these findings. To see why, take a look at this scatterplot of opioid deaths in the Midwest:
I’ve removed all the labels so you can take a look at it without any biases. Do you see any trends? Or any breaks in the data? I can’t say that I do. The data is all over the place. You can draw a regression line through it, of course, because you can draw a regression line through anything. But it sure doesn’t look like it would mean much.
Now take a look at the original version of the chart:
The authors draw a red line at the point where a Naloxone law was passed, and then draw regression lines separately for before and after. The “before” line is … something. I can’t even tell what. A cubic? The “after” line is … a line. But come on. Technically, the “after” trendline might start high and go down, but do you really believe that? There are only 12 data points and they vary wildly from month to month. This data just doesn’t mean anything.
I chose their scatterplot for the Midwest because it was especially noticeable that the separate trendlines were wholly artificial. But the other charts aren’t much different. Here’s one for ER admissions:
You’ll have to use your imagination to remove the labels, but I think it’s pretty easy to see that there’s nothing here. In fact, the error bars make this crystal clear: in the “after” half of the chart, they’re nearly as big as the entire chart! My eyeball estimate is that this simply shows opioid-related ER visits steadily increasing through the entire period. And yet the authors say this:
We find that Naloxone laws increased the quarterly number of visits by 266 per 100,000 residents (15%, p < 0.05). This effect is large and consistent with the hypothesis that Naloxone access increases the abuse of opioid drugs.
This is large? The only thing that looks large to me is the error bars.
Based on the data the authors collected, I’m perfectly happy to say that Naloxone laws, on average, have no effect on overdose deaths. Or at least no provable effect yet. But I’d sure be hesitant to say anything more than that.
