Professional Documents
Culture Documents
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also
available for most titles (http://my.safaribooksonline.com). For more information, contact our corporate/
institutional sales department: (800) 998-9938 or corporate@oreilly.com.
Printing History:
January 2010: First Edition.
Nutshell Handbook, the Nutshell Handbook logo, and the O’Reilly logo are registered trademarks of O’Reilly
Media, Inc. Open Government and related trade dress are trademarks of O’Reilly Media, Inc.
Many of the designations used by manufacturers and sellers to distinguish their products are claimed as
trademarks. Where those designations appear in this book, and O’Reilly Media, Inc. was aware of a trademark
claim, the designations have been printed in caps or initial caps.
While every precaution has been taken in the preparation of this book, the publisher and authors assume no
responsibility for errors or omissions, or for damages resulting from the use of the information contained
herein.
ISBN: 978-0-596-80435-0
1252004795
A SAMPLE OF THE CONTENTS OF OPEN
GOVERNMENT
Washington’s golden rule is different from the one we all learned growing up: “Do unto others
as you would have them do unto you.” In fact, Washington’s golden rule—“He who has the
gold, rules”—works in opposite fashion.
That’s not news. The fact that big money drives government decisions, that it has created a
mercenary culture in which nearly everything appears to be for sale, has been true of our
nation since its founding. Whether it’s information, access to lawmakers and elected officials,
legislation, or government spending, an exclusive group of moneyed insiders have outsized
influence. There are, of course, many channels for money to influence outcomes, most notably
campaign contributions and lobbying expenditures. There are also a multitude of ways this
group of insiders gets rewarded—contracts to consulting firms, special earmarks for
government spending, targeted tax breaks, and corporate subsidies. But the result is the same:
those who give, get. Ordinary people—“outsiders”—are excluded from this cozy little game.
But now there is a new challenge to this very old way of doing things.
With the rise of the Internet and the social Web, the outsiders are becoming “insiders”—or, to
be clearer, the barriers to entry are falling, the gatekeepers are losing their power to control
access, and thus the golden rule is being disrupted. Thomas Jefferson once remarked,
“Information is power.” In large part, the highly paid “insider” lobbyists in Washington work
to help their clients not just gain access to lawmakers, but perhaps as important, to shape,
obtain, and make sense of crucial government information. Lobbyists are the ones who can
get their hands on copies of proposed legislation hot off the printing press before anybody else.
1
They can help craft language for an earmark funding a pet project and make sure it gets
sponsored by a lawmaker and dropped into some massive spending bill. They can interpret the
minutiae of some government agency’s contracting rules and shepherd a client through the
thicket. Indeed, the need for this kind of assistance has become so de rigueur that even state
and local governments sometimes have taken to hiring highly paid lobbyists to help them
negotiate the mysteries of Washington. With a government opaque to all but the “insiders,”
outsiders—read, ordinary people—rarely have a chance to engage.
In a generation that is growing up with the Internet, however, the “outsiders” have a different
kind of expectation. They expect information to be fully available 24/7, and they expect
technology to allow them to engage with their friends, communities, and elected officials. If
you can sit thousands of miles away from Washington, D.C., in a coffee shop with free WiFi
that you found via a few clicks on Google Maps; if you can then do simple searches about
particular healthcare statistics where you live, such as the number of people who lack
healthcare insurance and how much cash local hospitals and clinics are getting from Medicaid
and Medicare; if you can dig around to see how much campaign cash your senator and
representative have taken from the healthcare industry and how they have voted on key
healthcare issues—well then, you have essentially become your own lobbyist, gathering the
information you need to make your case to your elected representatives. If your lawmaker is
on Twitter or Facebook or whatever the next revolution in the social Web will be, you can
communicate directly with your representatives and hold them accountable for their actions.
This information shift works in both directions, by the way. Thanks to emerging technology,
lawmakers and government officials will have access to increasingly sophisticated tools that
help them aggregate and analyze the views of their constituents and connect directly to you.
They also will not need to rely as much on intermediaries for information on what people care
about. A systemic change in how Washington works is now possible.
We are only just beginning to see the potential benefits of this new age. James Madison, father
of the U.S. Constitution, wrote, “A popular Government, without popular information, or the
means of acquiring it, is but a prologue to a farce or a tragedy; or, perhaps both. Knowledge
will forever govern ignorance; and a people who mean to be their own governors must arm
themselves with the power which knowledge gives.” A more transparent government will not
be the panacea for all that ails us. Our democracy will remain as messy as the Founders
expected and ensured it would be. But in this revolution there is finally the potential to subvert
the “golden rule” of Washington—to turn government inside out.
2 CHAPTER ONE
it was like planning a trip across the country by stagecoach and train, or how impossible it was
to get an idea to a faraway audience before the printing press. In the time of Twitter, Facebook,
YouTube, Google, and more, it’s easy to forget the really bad old days of truly opaque
government.
It helps to take a time machine back via the Sunlight Foundation’s Transparency Timeline
(http://www.sunlightfoundation.com/projects/transparency-timeline/). Much of the openness about
Congress’s doings that we now take for granted was hard to come by.
Of course, it’s not just Congress that has specialized in opacity. Myriad government agencies,
at taxpayer expense, collect and produce dizzying amounts of data about our economy, food
and drug safety, and the environment—indeed, every aspect of our lives. Yet in the past, most
of this information remained piled up in dusty docket rooms deep inside cement edifices.
Taking this data and making it available at a high price for those that could pay became a highly
lucrative business.
4 CHAPTER ONE
interfaces for remixing, contextualizing, and participating with the audio/video media assets
of our government. As a result, it’s now possible for anyone to find, annotate, tag, clip, and
display a snippet of video from the floor of Congress of lawmakers speaking on a particular bill
or topic.
Providing this kind of information isn’t just an exercise in entertainment. It helps citizens
become more involved and hold government accountable. In 2005, a coalition of bloggers
known as the “Porkbusters” was behind efforts to help expose Alaska’s so-called “Bridge to
Nowhere.” This transportation project in Alaska to connect the tiny town of Ketchikan
(population 8,900) to the even tinier Island of Gravina (population 50) cost some $320 million
and was funded through three separate earmarks in a highway bill. The same group helped
expose which senator—Sen. Ted Stevens of Alaska—had put a secret hold on a bill creating a
federal database of government spending, cosponsored by none other than then-Sen. Barack
Obama (D-Ill.) and Sen. Tom Coburn (R-Okla.). Recently, the Sunlight Foundation launched
Transparency Corps, where people can volunteer small amounts of time to help enhance the
transparency of government data. The first project underway is helping to digitize earmark
data, which lawmakers are making available but only in awkward formats. Armed with easily
searchable data, citizens will be better equipped to track government spending on these
projects.
OpenCongress.org is another example of making information more available so that citizens
can digest and act on it. Through this site, which provides baseline information about federal
legislation along with social networking features, users can sign up for tracking alerts on a bill,
a vote, or a lawmaker and link up with other people who are interested in monitoring the same
topics, monitor and comment on legislation, and contact their members of Congress. In 2008,
more than 45,000 people posted comments on legislation extending unemployment benefits;
first they used the OpenCongress platform as a way to press their representative to vote for the
legislation. Then, once it was enacted, they turned their comment thread into a de facto self-
help group for people looking for advice on how to get their state unemployment agency to
release their personal benefits. (Who needs lobbyists when you have the power of many?) In
spring 2009, the OpenCongress wiki launched, providing web searchers an entry on every
congressional lawmaker and candidate for Congress by pulling together their full biographical
and investigative record. And that’s open for anyone to edit.
We’re starting to see change from without become change for within, as government starts to
move toward a more modern, twenty-first-century understanding of its obligations to provide
up-to-date, searchable online information to the public. For example, FedSpending.org was
the first publicly available database on all government spending, created by the nonprofit OMB
Watch with support from Sunlight. Through it, citizens can find out not only how much money
individual contractors get, but also what percentage of those contracts have been competitively
bid. The database has been searched more than 15 million times since its inception in fall 2006.
Its creation helped prompt the passage of the Coburn-Obama bill mandating that the U.S. Office
of Management and Budget (OMB) create a similar database. But instead of spending $14
6 CHAPTER ONE
government work has a better chance. Citizens can help watchdog and cut down on wasteful
spending. People can find out about traffic fatalities in their neighborhoods, government-
sponsored clinical drug trials, or whether there’s been a safety complaint about the toy they
were planning to buy their kid. The barrier for entry into policy debates will be much lower.
Sure, we will always need experts who have deep experience to help explain what information
means, to give it context. But in the future, it will be a lot easier for journalists, academics,
public interest advocates, bloggers, and citizens to conduct these analyses themselves. That will
mean a healthier debate and, as a result, a fairer and more vibrant democracy.
“The old paternalism said the world was way too complex, and that we should trust the elders
who have got the credentials to make the right decisions,” said David Weinberger, author of
Everything Is Miscellaneous: The Power of the New Digital Disorder, at the 2009 Personal Democracy
Forum conference. “But we’re beyond a paper-based democracy now. The facts that are being
given to us are intended to keep us unsettled, because in the hyper-linked world of difference,
being unsettled, existing in chaos and constructive difference and never-ending argument, is
a far better approximation of reality than the paper-based world could ever give us…
Transparency is the new objectivity.”
The old paternalism is dying, but there is more work to be done, because it’s to the benefit of
big money interests to try to get around transparency efforts and work outside of public view.
Transparency alone will not create a democratic nirvana. But there is no denying that the
outsiders are becoming the new insiders, with the potential to rattle the status quo in
fundamental ways. In the immortal words of the venerable Yoda, “Always in motion is the
future.”
Crime rates in local communities. Campaign donations. Testimony before Congress. Open
government connotes open data. The Obama administration has acted on this premise and
produced a series of websites that will function as repositories for government data, at both
national and state levels. The next step for public engagement will be to make sense of this
data. Visualization can help.
Visualization is a key medium for communication in a data-rich world. It can have a catalytic
effect on data “storytelling” and collective analysis. We have seen examples of that power in
Many Eyes,* a public website we launched where anyone can upload and visualize data. The
site fosters a social style of data analysis that empowers users to engage with public data through
discussion and collaboration. Political debate, citizen activism, religious conversations, game
playing, and educational exchanges are all happening on Many Eyes. The public nature of these
visualizations provides users with a transformative path to information literacy.
Policy
Citizens are starting to realize the power of interactive visualization to help make sense of the
political world around them at both the national and local levels. In this section, we will
* http://www.many-eyes.com
9
FIGURE 2-1. ProPublica treemap visualization of the federal stimulus bill of February 2009; rectangle size corresponds to amount
of money allocated
illustrate how people have been using visualization to think and talk about policy, the
economy, the health of their communities, and their expectations for government.
Looking closely at one’s own backyard can be quite revealing. This is what Jon Udell, a
prominent blogger, did when he created a series of Many Eyes visualizations of crime statistics
in his hometown of Keene, New Hampshire. Udell wanted to understand whether the facts
supported rumors of a crime wave in the area. After looking at the graphs and comparing
historical, national, and local trends, Jon concluded the perception of a local crime surge was
not warranted. He then created a screencast documenting his motivating questions, the data
collection process, the visualizations he created, and how playing with the visualizations helped
him deduce that perception was harsher than reality. His blog post on this screencast generated
a healthy number of comments, some of them from people who were hoping to do the same
kind of analysis in their own communities.
In addition to individual citizens, institutions have also been making use of Many Eyes
visualizations to monitor the economic and political world around them. ProPublica, a
nonprofit newsroom for investigative journalism, has used Many Eyes to cover a range of
issues, from unemployment insurance to weatherization projects in the United States. One of
the most popular visualizations ProPublica created is a treemap of the February 2009 federal
stimulus bill (see Figure 2-1). ProPublica placed the interactive visualization on its website, and
the visualization became one of a series of charts ProPublica created to follow the bill as it
passed both the House and Congress deliberations.
10 CHAPTER TWO
Another example shows both the power of visualization to make an argument engaging, and
the potential for web-based visualizations to spread to new sites and audiences. The Sunlight
Foundation used data on congressional “earmarks” to create Many Eyes bubble charts, a new
visualization technique that represents a set of numbers by circles whose areas are proportional
to the underlying numbers (see Figure 2-2). A number of blogs picked up the visually striking
results. We then saw one of these charts appear in a video created by law professor and reformer
Lawrence Lessig, who used it as evidence of the favoritism that permeates the lawmaking
process.
Visualization can function as an accessible way to engage with intimidating amounts of textual
data as well as numeric data. In March 2009, President Obama invited citizens to ask him
questions about the economy in the first-ever Online Town Hall Meeting. More than 71,000
people submitted questions to the White House website. Such a collection can be hard to parse,
and the Obama team combed the collection to select the questions the president should address.
But what about the entire collection of questions? As a whole, they could represent the
concerns of a nation. The collection was publicly available, but vast, unstructured, and
unwieldy. Shortly after the question-and-answer session, Many Eyes users busily began
visualizing the entire set of submitted questions. One phrase net, a visualization technique
introduced on the Many Eyes site, mapped all the questions on education, revealing islands of
subjects: “schools and teachers,” “science and math,” and “college tuition” (see Figure 2-3).
12 CHAPTER TWO
FIGURE 2-3. A visualization of questions on education, from among the collection of questions that were submitted to the White
House for President Obama to address in the Online Town Hall Meeting in March 2009
politician’s handlers did not plan for. We saw examples of this search for meanings on Many
Eyes. One user, for instance, created a comparison tag cloud, showing John McCain’s blog
contrasted with the blog of his 23-year-old daughter Meghan, who was appearing with him
on the campaign trail (see Figure 2-5).
In this visualization, the words in orange are taken from Meghan’s blog, and the blue words
are from John’s. The size of each word tells how frequently it occurs, and the words are sorted
from more frequent use in Meghan’s blog (top) to more frequent use in John’s (bottom). The
comparison ends up being a kind of filter, with common and clichéd words in the center
(time, things, senator). At the bottom, however, we see the three words military, angry, and
FIGURE 2-5. Comparative tag cloud of two McCain blogs: John’s (in blue) and Meghan’s (in orange)
American. At a time when McCain’s campaign was trying to project a softer image, it is
interesting to see “anger” take a prominent place.
In some cases, a politician may not be merely spinning, but actively evading an issue. A recent
notorious case was the 2007 testimony of then-Attorney General Alberto Gonzales, regarding
the firing of U.S. lawyers. After one of the Many Eyes team members put up a word tree
visualization of his testimony, showing the prevalence of the phrase “I don’t recall” (see
Figure 2-6), another user on the site quickly followed with an analogous visualization of Bill
Clinton’s words in another famous piece of testimony (see Figure 2-7). In this case, the creation
of the visualization may be seen as a kind of debate statement in itself—not about policy, but
making the point that evasive testimony crosses party lines.
Visual Literacy
How broadly accessible are these sometimes esoteric visualizations? There’s no doubt that some
of the visualization activity on Many Eyes (and other sites) is created by, and plays to, an early-
14 CHAPTER TWO
FIGURE 2-6. Word tree of Alberto Gonzales’s 2007 testimony before Congress
adopter audience that enjoys engaging with data for its own sake. But at the same time there
is evidence that both creators and viewers of the visualizations are a diverse group. In
interviews with Many Eyes users, we learned that some of the most active users had never
worked with data before—in at least one case, had never used a spreadsheet. We have also
seen more than a dozen different classes use Many Eyes for class assignments, indicating that
some teachers are putting an emphasis on teaching visual literacy.
Indeed, new and unusual visualization types seem to have the power to pique readers’ interest.
Part of what drew bloggers to the earmark bubble chart, for instance, may have been its striking
appearance. We certainly see this happening elsewhere as well. Alluring charts and graphs
from The New York Times and CNN, for instance, have become national conversation pieces.
CNN launched an interactive wall that visualized the evolution of voting patterns during the
last presidential election. The New York Times has used interactive visualizations to cover a
variety of subjects, ranging from the war in Iraq to how Congress questions Supreme Court
nominees.
Conclusion
Our experiences with Many Eyes suggest three principles for how visualization can help with
open government.
First, statistical graphics ground debate in reality. For readers, they are effective at
communicating basic aspects of an issue. But just as important is the fact that graphs and charts
16 CHAPTER TWO