Improving Data Program Management: Where to Start?
Management wants to upgrade how you manage and analyze your organization's data assets. Where do you start?
A good starting place
Are your efforts to harness and analyze your organization's data failing due to faulty planning or faulty execution? If so, how can you avoid such pitfalls in the future?
A good starting place is to examine why you got started with upgrading your data handling and analysis efforts in the first place:
- Perhaps the demand for a change came from the top. Maybe one of your top executives came back from a "big data" conference and, having heard about what a competitor was doing with predictive modeling of market demand, told your IT department, "I want one of those!"
- Perhaps one of your junior software engineers came back from a weekend hackathon devoted to building visualization apps for publicly distributed municipal data. She then excitedly floated the idea to management that something similar could be done with your organization's own data.
- Perhaps your own marketing staff has become "fed up" with how long it takes to get standardized reports from your highly structured data warehouse and decided to launch their own analytics with support from outside consultants.
- Perhaps you "started too small" by going after too much "low hanging data fruit" which resulted in your focusing on easy-to-solve but unimportant problems.
- Perhaps you have suffered loss or embarrassment by being unable to reconcile contradictory reports out of datasets that should be but aren't telling you the same thing.
While you may have had some early success with generating new insights, maybe later on you ran into unanticipated problems when you tried to extend your initial approach to other more important areas where you encountered both organizational and data quality roadblocks.
Maybe members of your staff have become enamored of new tools and you found yourself facing the possible need to build a parallel and potentially expensive data management infrastructure alongside your existing operations.
Another possibility is that, after investing time and money in new tools and techniques, you're faced with previously hidden data incompatibilities that your older more "siloed" data management operations had been able to accommodate; these incompatibilities and other data problems started to emerge when you began to look at managing your data assets in a more unified and strategic way.
Needed: a strategic approach
If any of the above sounds familiar, welcome to the club. You might be running into the inevitable result of having planned data handling and analysis initiatives without recognizing the need for a more strategic approach to governing your data. This can happen even when first steps focus on delivering short-term benefits without too much alteration of existing data handling methods and systems.
Data and metadata are like that. Managing them effectively and efficiently as organizational assets requires attention to both the "big picture" and the detail. This means understanding and directing how data are organized, managed, and used.
My colleagues and I refer to such questions as being related to “data governance.”
One thing that makes data such a challenge to govern is that, like "The Force" in Star Wars, data permeate and flow through all things in the organization. Both people and machines need to communicate. Data and metadata underlie the languages they both use to communicate.
If data are confused, misdirected, inconsistent, overly siloed, or sloppily defined, miscommunication and eventual problems may occur, sometimes with grave and/or expensive consequences.
The real world
Organizations that have grown up in the digital era are more likely to understand data complexities. They may have already solved or evolved methods for managing how data flow and taking advantage of their significance. For them, "digital transformation" will be less stressful or problematic.
For the rest of us, our operations may depend on systems and procedures built over many years and are interconnected with fragile software and translation methods that need constant updating. Changes in language, incompatibilities in data sources, and even minor variations in how things are are defined semantically can wreak havoc when we’re attempting to make sense quickly from data in response to a query from management. Adding the need to store and make sense from constantly increasing stores of transactional data puts additional stress on our more traditional approaches to data storage, management, and analysis.
Our approach
My colleagues and I call our approach to improving performance through better data management Data Program Management. As described in An Introduction to Data Program Management (DPM), DPM has three related components:
- Data Strategy addresses how data programs are planned and executed.
- Data Architecture addresses how the organization’s current and future data and metadata are defined and organized.
- Data Governance addresses how data, metadata, and their associated processes are managed.
Some of what we do will be viewed as traditional in the IT consulting world. Developing "Current State” and “Future State” assessments -- but with a focus on data -- is one example. Knowing where you're starting from will be critical to helping you decide what needs to be done in order to get to where you want to be.
Making better use of data, especially when improvements in predictive and descriptive analytics are being contemplated, requires management not just through use of technology and tools but also in terms of how technologies and tools are managed and aligned with and supportive of the organization’s goals and objectives.
Making better use of data requires a strategic perspective and an understanding of how the organization works overall, not just a technical perspective that focuses only on traditional data management tools.
Balancing act
Despite the importance of strategy, you also need to start somewhere and deliver benefits to the project sponsor as soon as possible in order to gain support for the work. It's important to balance the desire to comprehensively realign how data are governed -- which can take a long time -- with convincing and useful solutions to specific and important analytical or data management problems.
Important questions
Our recommended approach is incremental and starts with "deep dive" into the data and metadata associated with a particular problem, function, or domain. This targeted focus helps control and define our scope of work while helping to make visible the connection between data governance and the success of the organization.
The manner in which this initial deep dive is conducted supports both (a) delivering value quickly (e.g., through improved analytical reporting or predictive modeling for the problem of interest) as well as (b) providing a foundation for future growth (e.g., by introducing a data governance process that collaboratively involves both technical and business stakeholders).
For the selected area we recommend addressing questions such as these:
- What data and metadata are required to support this domain or solve this problem?
- How are these data and metadata currently defined and managed (or not managed)?
- What systems and processes are connected to these data and metadata?
- Are there problems or challenges that are being negatively impacted by the way these data and metadata are defined or managed?
- Who currently has responsibility for the systems and processes that generate and depend on these data?
- What current systems and processes would be positively -- or negatively -- be impacted by a change in how data and metadata are managed?
- Are there unmet needs or even "stretch goals" that could be addressed were data and metadata managed differently?
- What needs to be done both technology- and process-wise so the data and metadata are better managed and utilized?
- What are the actual words used by humans and machines in communicating about areas of interest?
- How are these words related to each other and to the systems and processes they touch?
- For the chosen focus area or problem, how will we measure progress based on improvements in data governance?
- Does the organization already have a collaborative model for how cross-departmental or cross-functional governing bodies operate?
- Can we address the above questions in an agile fashion without getting tangled up with unnecessarily structured processes and procedures?
Future topics
In future documents and presentations we will provide more detail about our approach to helping clients answer the above questions through a combined approach to data strategy, data governance, and data architecture related initiatives. Topics to be covered in more detail will include:
- Data inventory. Do you know what data you have?
- Data provenance and process ownership. Who is responsible for how your data are used, regardless of where the data originate?
- Metadata repository. What are the terms, concepts, and process connections related to your data that you need to document and control?
- Data governance and stewardship. How do you efficiently manage your data as your organization and its data requirements continue to evolve?
Copyright (c) 2017 by Dennis D. McDonald. For more about Data Program Management go here. Contact Dennis by email at ddmcd@outlook.com or by phone at 703-402-7382. Check out his curated Managing Data collection on Google+.
