Apr 24

Apr 24 Do People Really Understand What “Open Data” Means?

Big Data, Definitions, Market Research, Metrics, Open Data, Pew, Statistics, Surveys, Transparency

According to the Pew Research Center’s report Americans’ Views on Open Government Data,

Many hope that more transparency and data sharing will help journalists, make officials accountable and improve decisions. But very few think agencies are doing a great job of providing useful data.

While the term “open data” does not appear in Pew’s topline questionnaire describing what questions were asked survey respondents, Pew’s writeup of this highly informative and well documented survey does use the term “open data” in reporting some the results. It’s therefore possible to read the report and conclude that what are commonly thought of as “open data” programs have relatively little impact on the lives of citizens.

One issue with the Pew report – and with the open data “movement” occasionally – might be that people sometimes confuse providing data with providing a service. As I’ve discussed before, government sponsored open data programs to be really useful need to make sure that the content and manner in which data are provided to the public actually support the government’s goals and objectives. Being “open” isn’t enough. Program designers also need to make sure that the openness serves a purpose.

Definitions of the term “open data” are readily available online. A good one is available from the Open Data Institute. But, asking a member of the “general public” what open data means will probably generate a blank stare in return. Digging deeper with those who have some experience with the term will also generate a variety of responses with overlapping references to several concepts including:

Transparency
Accountability
Civic engagement
Big data
Open-source
Standardization
Reusability

The reality is that one can have several different perspectives on what constitutes open data. Some perspectives are influenced by technology. Others relate more to the purposes for which the openness and reusability of data are being sought.

This reminds me of the old term “Web 2.0” that caused so much confusion. No matter how hard we tried back then that term refused to die and is probably still finding uses today given that technologies and communication styles vary in adoption rates across different communities.

One of my favorite descriptions of open data is used by the Data Transparency Coalition to describe its mission:

We believe governments should adopt non-proprietary data standards for the information they generate or collect, and publish such information as machine-readable data, especially with regard to their spending, regulatory, legislative, and judicial activity.
The transformation of public information generated and received by governments from disconnected documents into interoperable open data will:
Strengthen democratic accountability by making public data more accessible for citizens and watchdogs;
Enhance government management by improving data sharing and analytics capabilities;
Reduce compliance costs by automating reporting processes.
Data transparency also stimulates tech-sector innovation and creates jobs.

The above touches on many (but not all) of the interests common to grassroots civic organizers, government bureaucrats, technology vendors, and consultants like me. But I’m not going to hold my breath till members of the general public conjure something like the DTC statement when asked about “open data” by a survey research company. That’s not going to happen. In fact, as suggested in Is Your Organization Ready for the Third Age of Open Data? use of the term may already be unnecessary in some cases because what organizations such as DTC and ODI seek is taken for granted and already being incorporated into “open by default” government programs.

Another possible issue with the term “open data” may be its current use with respect to standards and interoperability by “big data” software vendors. This is exemplified by the Open Data Platform which is currently a consortium of Apache Hadoop focused companies seeking to standardize on a core big data platform in order to “… help minimize the fragmentation and duplication of effort within the industry.”

While the goals and objectives of open data and big data may intersect, the technology infrastructure focus of the ODP’s “open big data” members appear to be less concerned with the government and citizen focus of organization such as ODI, vendors such as Socrata and OpenDataSoft, and consulting firms such as Balefire Global (with which I’m affiliated).

At the end of the day, why should the public care about such distinctions? Aren’t such terminology concerns really “inside baseball” and of much more interest to people and organizations with policy or technology axes to grind? Once you move away from those of us who have a “dog in the open data and big data hunts,” do issues like interoperability, standards, solution sharing, and government transparency really matter to the day-to-day lives of individuals and families? I think so, of course, but that’s the business I’m in.

Ultimately the most important issue has to come down to deciding what level of data literacy citizens need. As the production and consumption of goods and services become more data-dependent in both developed and developing countries, it is reasonable to ask how much understanding of data and data related decisions people really need. I’ve referred to this elsewhere as data management literacy. Maybe we also need to consider data consumption literacy. After all, if people don’t understand or appreciate the services we’re providing, no amount of standardization, interoperability, or transparency is going to make any difference.

Related reading: