Congressional Data Mining: Coming Soon?

How a little-noticed provision in a House spending bill could revolutionize access to congressional information.

Photo by flickr user <a href="" target="new">I am I.A.M.</a> used under a Creative Commons license.

Let our journalists help you make sense of the noise: Subscribe to the Mother Jones Daily newsletter and get a recap of news that matters.

By slipping a simple, three-sentence provision into the gargantuan spending bill passed by the House of Representatives last week, a congressman from Silicon Valley is trying to nudge Congress into the 21st Century. Rep. Mike Honda (D-Calif.) placed a measure in the bill directing Congress and its affiliated organs—including the Library of Congress and the Government Printing Office—to make its data available to the public in raw form. This will enable members of the public and watchdog groups to craft websites and databases showcasing government data that are more user-friendly than the government’s own.

If the Senate passes the bill with the provision intact, citizens seeking information about Congress’ activities—such as bill names and numbers, amendments, votes, and committee reports—won’t have to rely on government websites, which often filter information, are incomplete, or are difficult to use. Instead, the underlying data will be available to anyone who wants to build a superior site or tool to sift through it. “The language is groundbreaking in that it supports providing unfiltered legislative information to the public,” says Honda’s online communications director, Rob Pierson. “Instead of silo-ing the information, and only allowing access through a limited web form, access to the raw data will make it easier for people to learn what their government is doing.”

Successful, privately-created websites that provide the public with information about Congress’ actions already exist.,,, and all make legislative data available to the public in ways that are easier to navigate than Congress’ primary web portal, a system called Thomas. Those sites currently get their data through techies who “scrape” Thomas and other government websites, which means they use bots to process the HTML and gather what is valuable. The process is labor-intensive and imprecise. “It’s difficult to keep the data up to date, in some cases impossible, and occasionally there are errors in the data,” says Josh Tauberer, the 26-year-old who runs and does lots of the “scraping” that others use. “This could all be fixed by a bulk data download.”

Tauberer expects that the availability of additional and easier-to-use congressional data will spur innovation. “You can expect to see other sites spring up doing new and interesting things with the information.” He anticipates charts, graphs, and maps that represent congressional goings-on visually—”ways of visualizing the congressional process that we couldn’t yet imagine.” Honda, with his Silicon Valley roots, expects that developers and coders will quickly outpace the government’s efforts to date. “We hope that we can learn from the wisdom of crowds,” says Pierson.

There are government agencies that already provide massive amounts of data via databases. The Census Bureau provides huge amounts of information in raw form, allowing academics, statisticians, and think tank scholars to comb through it in any way they please. The Federal Elections Commission publishes unedited data on campaign contributions, giving rise to sites like, which allows the public to see who is donating to whom, and allows journalists and watchdogs to investigate the influence of money in politics.

“In our Web 2.0 world, we can empower the public by providing them with raw data that they can remix and reuse in new and innovative ways,” Honda told Mother Jones in a statement. (Disclosure: In the summer of 2002, I briefly worked as an intern in Honda’s district office.) Honda’s provision, however, pertains only to legislative data. Federal departments like the Environmental Protection Agency, the Food and Drug Administration, and the Department of Energy have reams of data that political scientists, economists, and researchers of all stripes would love to get their hands on. Many who work at the intersection of technology, politics, and transparency believe that the key player in broadening Honda’s effort to include the executive branch will be Vivek Kundra, the former Chief Technology Officer of the District of the Columbia who was named Obama’s Chief Information Officer on Thursday. According to the National Journal‘s Tech Daily Dose, Kundra “told reporters Thursday he will launch, a Web site intended to ‘democratize data’ by giving the public raw feeds of information from a range of agencies.”

John Wonderlich, the policy director at the Sunlight Foundation, which has created or funded several tools that make government data easier to analyze, is holding out hope that the president’s Open Government Directive, which is due at the end of May, will further address the issue of data availability. He applauds Honda for putting Congress, at least, on the right track. “Without Honda’s attention to this issue, congressional level attention to bulk data access would be unlikely,” he says. “We’re happy to see this first step.”


Headshot of Editor in Chief of Mother Jones, Clara Jeffery

It sure feels that way to me, and here at Mother Jones, we’ve been thinking a lot about what journalism needs to do differently, and how we can have the biggest impact.

We kept coming back to one word: corruption. Democracy and the rule of law being undermined by those with wealth and power for their own gain. So we're launching an ambitious Mother Jones Corruption Project to do deep, time-intensive reporting on systemic corruption, and asking the MoJo community to help crowdfund it.

We aim to hire, build a team, and give them the time and space needed to understand how we got here and how we might get out. We want to dig into the forces and decisions that have allowed massive conflicts of interest, influence peddling, and win-at-all-costs politics to flourish.

It's unlike anything we've done, and we have seed funding to get started, but we're looking to raise $500,000 from readers by July when we'll be making key budgeting decisions—and the more resources we have by then, the deeper we can dig. If our plan sounds good to you, please help kickstart it with a tax-deductible donation today.

Thanks for reading—whether or not you can pitch in today, or ever, I'm glad you're with us.

Signed by Clara Jeffery

Clara Jeffery, Editor-in-Chief

payment methods

We Recommend