Government Data and the Commercial Cloud: Who Governs, Who Pays, and TANSTAAFL
In Government data, commercial cloud: Will public access suffer?, Mariel Borowitz in AAAS’ February 2018 issue of Science reviews issues that arise when governments rely on the private sector to provide public access to data generated by taxpayer-funded government programs. These include:
Inconsistent policies across agencies in how public access programs are governed when private sector (e.g., cloud vendors) are involved as an intermediary between the agency and the public.
Significant differences in how much data government control and the much smaller portion of that data that are currently accessible to the public. NOAA’s large volume of “nonpublic” data is given as an example of this ongoing disparity.
Variations in whether the government or the public pays cloud vendors for access to government data when cloud vendors serve as the intermediary.
Variations in potentially costly ancillary services provided by cloud vendors, for example, whether only “raw” data are provided for download of public data, or whether analysis tools and other related services are also provided.
Differences across agencies in cost recovery policies in general (e.g., has Congress in enabling legislation for a program been specific about whether a particular program can charge fees to compensate for operating costs?)
Advocates of “open data” are familiar with these issues. This is especially true for those who come to open data with experience in the commercial data management world with an understanding of real-world costs and the practical realities of data governance.
In 2014 in Challenges of Public-Private Interfaces in Open Data and Big Data Partnerships I wrote:
Moving government agencies from performing basic services to becoming “data providers” may require some difficult changes for agencies and programs that have not traditionally performed the functions necessary to support open data access. Agencies and programs differ widely in how they approach data management. This may require, for example, much more attention be paid to data and metadata standards and quality control than they are accustomed to providing. I would not be surprised if government agencies find themselves experimenting with different organizational and governance structures as they begin to take open data programs more seriously.
The Science article certainly bears this out. The most important part of effective data governance, I’ve found, is the ability to link what is done with data to the goals and objectives of those originating the data:
When government enlists the private sector in making data available to the public, it is incumbent on the government to ensure that the originating program goals are being promoted. It is also incumbent on the private sector vendor to ensure its own business goals are supported in how it supports access to government data.
Aligning these two perspectives requires both honesty and transparency about how costs and data governance are managed. That can be complicated when the two parties involved (it’s rarely just two parties given the diverse constituencies and stakeholders associated with government programs) have potentially competing objectives and differing philosophies about what the role of government is/should be.
My questions at this stage include:
Has the government defined a consistent and detailed set of policies and processes about how agencies should interact with the private sector in providing access to government data?
Does the government have a formal and transparent mechanism through which it can help individual agencies work through an efficient planning process to engage with the private sector in providing public data access?
Do government agencies understands what it costs to develop and run an open data program? (See various definitions of the term “TANSTAAFL.”)
Has funding and an organizational infrastructure — with clout — been provided to ensure that (1), (2), and (3) happen even when data governance activities must cross organizational and jurisdictional boundaries?
Copyright (c) 2019 by Dennis D. McDonald