Paul Light has gotten a lot of attention for his recent study showing that “government failures” are on the rise. I’ve seen several criticisms of his study, but it seems to me that basic methodology is really the main problem with it. First off, his dataset is a list of “41 important past government failures (between 2001 – 2014) from a search of news stories listed in the Pew Research Center’s News Interest Index.” Is that really a good way of determining the frequency of government failures? A list of headlines might be a 
good way of determining public interest, but it hardly seems like even a remotely good proxy for cataloging government failure in general.
For example, 2007 appears to be an epically bad year for government failure. But among the failures are “wounded soldiers,” “food safety recalls,” and “consumer product recalls.” Those all seem a bit amorphous to count as distinct failures.
This methodology also mushes up timeframes. Fast & Furious is counted as a government failure in 2011, but that’s just the year it made headlines. The operation itself ran from 2006-11. Likewise, the “postal service financing crisis” is hardly unique to 2011. It’s been ongoing for years.
Some of the items don’t even appear to be proper government failures. Was the Gulf oil spill in 2010 a government failure? Or the Southwest airline groundings? In both cases, you can argue—as Light does—that they exposed lax government oversight. But this basically puts you in the position of arguing that any failure in a regulated industry is a government failure. I’m not sure I buy that.
Finally, on the flip side, there are the things that don’t show up. The government shutdown in 2013? The fiscal cliff? The debt ceiling standoffs? The collapse of the Copenhagen conference? Allowing Osama bin Laden to escape from Tora Bora? The scandalous demotion of Pluto to non-planet status?
Maybe I’m just picking nits here. But given the weakness of the core methodology; the small number of incidents; the problems of categorization; and the overall vagueness of what “failure” means, I’m just not sure this study tells us much. I’d take it with a big shaker of salt for the moment. It seems more like clickbait than a serious analysis of how well or poorly government has done over the past decade.
