Do Organizations Need Data Governance as a Service (DGaaS)?

December 14, 2017 Dennis D. McDonald

Too much tech?

Once upon a time (i.e., last year) I was concerned that, when it came to improving their data governance, some organizations were paying too much attention to the interests of IT and “data scientists” and not enough to how data governance efforts need to align with the business.

Business goals ---> data governance

That may be changing, at least when it comes to what people are writing about. Examples are the following:

Andrew Joss’ The Value of Intelligent Data Governance in the Informatica blog.
Adam Famularo’s Data Governance 2.0, Data is Everyone’s Business in insideBIGDATA.
Lowell Fryman’s MDM Requires These Data Governance Activities in The Data Administration Newsletter.

All three authors emphasize the need for business goals, not technology, to drive data governance strategies. They also discuss involvement of business and operations staff – not just IT -- in prioritizing and managing data governance activities including decisions related to master and metadata management.

None of this should come as a surprise. The more we understand how dependent digital transformation and innovation are on good data, the more we realize how important efficient and effective data management really is.

Efficiency as a starting point

Naturally a lot will depend on why improvements in data governance are trying to bring about. The most obvious and probably the easiest to document are efficiency benefits. Over time, better data governance can reduce errors, misunderstandings, and miscommunication -- along with the costs associated with them. If you have ever tried to combine two customer systems that were based on two fundamentally different approaches to defining customer addresses you will know what I mean.

Of course, improved data quality starting at the data life cycle’s source has long been seen as a way to avoid rework. At the same time, an important question has always been, “When is better data quality worth the cost?” This is the simple application of the traditional concept of cost benefit analysis but to do this well you must first understand the costs you’ll be avoiding through improved data quality.

But, efficiency is not enough

Understanding what these costs will be, especially in large or complex organizations, can take time and effort. Realistically, though, improved operating efficiency and reduced costs through improved data quality can only get you so far. For example, cost avoidance may not translate directly to profitability. This may not appeal to some in management.

You will also need to understand business impacts of improved data governance that extend beyond efficiency to the organization’s relationship to suppliers, vendors, and ultimately the organization’s customers.

DG isn’t just IT’s responsibility

This is where IT staff and data scientists can’t go it alone. As suggested by authors such as those cited above, IT and data science teams must engage with the business in the planning and improvement of data governance, regardless of which of the following data application categories they are pursuing:

Using data to help understand what was (e.g., provide historical context)
Using data to help understand what is (e.g., get a better handle on what’s happening now)
Using data to help understand what will happen (e.g., compare and contrast different options and scenarios)

Anyone who has helped develop a data warehouse to improve data analysis and reporting will understand how challenging just the first two data applications are. Modern business intelligence tools provide impressive analytical, visualization, and collaboration power to end-users. Previous generations’ weekly stacks of report “printouts” providing mind-numbing aggregated and disaggregated sales reports are long gone. Thankfully they have been replaced by phone and tablet based apps that providing not only canned reports but powerful interactive analysis and display features.

Data power needs data literacy

These powerful features can prove challenging given the wide variations in data literacy that currently exist in many organizations. Training and adoption challenges are amplified when we add the third application area – predicting “what will happen” – to the mix of available services that improved data governance must support. The tools now available for data analysis and their ability to incorporate nontraditional table-based data sources such as textual and cloud based data pose not only unfamiliar technical but unfamiliar business challenges as well.

Data Governance as a Service (DGaaS)

In such a complex environment, data governance must be approached as a service (DGaaS) that supports the entire organization. Some sophisticated users will be able to operate on their own. Others will need training and hand-holding. Still others will prefer data services to be provided to them.

The implications for managing data quality, privacy, context, security, lineage, stewardship, and currency – all areas that are relevant to data governance – will vary for these different user groups, even when master or core data supporting all three are the same. Hence the need exists to integrate both business and IT in both the planning and management of data governance and the services that rely on data.

How DGaaS will be managed will depend on the organization. One consideration will be how it addresses the issue of a centralized versus decentralized operation. Another will be the degree to which standardization is needed and what costs and benefits might emerge from improved data standards.

Needed: a data inventory management process

Ultimately, in order for a data governance service to operate effectively it must be able to understand what data the organization controls and what data it needs to operate to support three application areas described above. Given the variety of data types flowing through most organizations, the creation of a single unified database encompassing all data is unrealistic and unnecessary. Instead, what is needed for data governance is a data inventory management process that provides a comprehensive view of the organization’s data and associated resources.

Tools: necessary but not sufficient

This is where improved metadata management and semantic analysis come in. Tools to support such a comprehensive view are becoming available (e.g., see Solution to Manage the Provenance & Governance of Big Data Analytics).

However, regardless of the tools used to inventory and model the organization’s data, IT and data scientists must work with management so that data-dependent services provided to the organization deliver value in support of the decisions and priorities that really matter.