The Reinhart/Rogoff Episode Shows Us Why Data Openness is So Important

For indispensable reporting on the coronavirus crisis, the election, and more, subscribe to the Mother Jones Daily newsletter.


What’s the most important takeaway from this week’s kerfuffle over the errors in the Reinhart/Rogoff dataset? Obviously there are public policy implications, since R&R concluded that debt levels above 90 percent of GDP lead to a sharp dropoff in economic growth. This provided important intellectual support during the past few years for the belief that it’s more important to focus on deficit reduction than on unemployment and fiscal stimulus.

In the end, though, although R&R’s results were important, they probably weren’t crucial. As Jared Bernstein notes, deficit hawks were “using research findings the way a drunk uses a lamppost: for support, not for illumination. If the R&R lamppost turns out to be wobbly, the austerions [] will find another one. In this town, I’m sorry to say, you can pretty much go think-tank shopping to buy the result you seek.”

I agree. R&R’s results were convenient to have, but not crucial. People who are dedicated to cutting spending, either for political or ideological reasons, would simply have trotted out other justifications if R&R’s study hadn’t been available.

More important, then, may be the light this shines on the fact that an awful lot of research is based on datasets that are kept private. Megan McArdle pinpoints both the reason for this and the dangers involved:

There are a lot of private data sets out there these days, and a lot of work being produced off of them. Why can’t more of us see it?

Mostly, I suspect, because of the economics of the thing. Assembling a nice private data set is a huge amount of work. You want to be able to mine that work for publishable insights. Very little professional credit accrues to the guy who built a great dataset which everyone else uses to generate elegant new findings. The credit goes to the authors of the elegant new findings. Which means that once you’ve built a dataset, you want to keep the thing to yourself as long as possible.

….Unfortunately, there’s always the possibility that “I want to hold onto it as long as possible so I can publish” can be a cover for “I need to hold onto it as long as possible because if anyone else sees it, it will rapidly become obvious that my results aren’t very robust.” Or, in some cases, for “There are no results because I made the whole thing up.”

This was an issue that was front and center during Climategate. Climate skeptics were unhappy that the raw data collected by various research groups (mostly using public money) wasn’t made available to them, and they made the reasonable point that if your analysis is correct, you shouldn’t be afraid to share the underlying data. Professional researchers, for their part, were reluctant to make their data available because they knew it would generate an enormous flurry of amateur debunkings that they’d have to waste their time picking apart. In the end, though, releasing the data didn’t make much difference because it turned out the professionals were interpreting it correctly.

This is why the R&R incident might be more important for the cause of openness. It’s not just that critics found errors in the dataset once they got hold of it. Even more important might be the fact that once the raw data was available, they were motivated to do certain types of analysis that R&R didn’t do. For example, they discovered that R&R’s results were quite fragile, changing dramatically with the addition or subtraction of just a few observations. They also found that, far from showing a sharp drop in growth when debt passes 90 percent of GDP, the data actually showed a sharp drop between 0-30 percent of GDP and only a rather leisurely decline above 70 percent. Finally, and perhaps most important, UMass economist Arindrajit Dube took a closer look at what the data said about causality, and produced persuasive evidence that R&R’s own data suggested not that high debt leads to low growth, but that low growth leads to high debt. R&R could have performed the same analysis, but for some reason they weren’t interested in it.

As long as the dataset was private, nobody would ever have known any of these things. We could have guessed them—and a lot of people did—but we wouldn’t have had any evidence. Now we do. Unfortunately, as McArdle says, “the incentives are all wrong for data openness.” This episode suggests why it might be worthwhile to try to change those incentives.

DOES IT FEEL LIKE POLITICS IS AT A BREAKING POINT?

Headshot of Editor in Chief of Mother Jones, Clara Jeffery

It sure feels that way to me, and here at Mother Jones, we’ve been thinking a lot about what journalism needs to do differently, and how we can have the biggest impact.

We kept coming back to one word: corruption. Democracy and the rule of law being undermined by those with wealth and power for their own gain. So we're launching an ambitious Mother Jones Corruption Project to do deep, time-intensive reporting on systemic corruption, and asking the MoJo community to help crowdfund it.

We aim to hire, build a team, and give them the time and space needed to understand how we got here and how we might get out. We want to dig into the forces and decisions that have allowed massive conflicts of interest, influence peddling, and win-at-all-costs politics to flourish.

It's unlike anything we've done, and we have seed funding to get started, but we're looking to raise $500,000 from readers by July when we'll be making key budgeting decisions—and the more resources we have by then, the deeper we can dig. If our plan sounds good to you, please help kickstart it with a tax-deductible donation today.

Thanks for reading—whether or not you can pitch in today, or ever, I'm glad you're with us.

Signed by Clara Jeffery

Clara Jeffery, Editor-in-Chief

We Recommend

Latest