"I'll Take 'DataOps' for 500, Alex"
On behalf of my client Dimensional Concepts LLC (DCL) I attended the recent webinar sponsored by Data Science Central titled How To Structure A Modern Data Ops Team. Michele Goetz from Forrester talked about what that company is finding in its research about modern corporate data governance practices. Will Davis of Trifacta discussed that topic from the perspective of a tool vendor that helps clients implement state of the art data management and analytics platforms.
My client’s interest is straightforward: in the business of supporting government programs for both a civilian and defense agencies, DCL is researching potential sub and prime contractors for program support contracts that include data management and analytics. Influencing this is my own interest in how to manage such programs when transitioning from traditional data architectures to distributed and cloud-based approaches involving large or diverse data volumes.
The webinar presentations expanded on what I have learned in my own research and consulting related to data governance and data program management, for example:
There’s always more to data governance than selecting the right tools.
Traditional roles and organizational structures must change to address opportunities generated by rapidly evolving data management and analytics capabilities.
People tend to resist changes to how they have worked in the past.
How the transition is managed can make or break an organization’s ability to make the most of its data.
I have already addressed some of these issues—especially the last—in my own work (for example, see An Introduction to Data Program Management (DPM)). Still, the webinar’s presentations were enlightening since they addressed the practical definition of different roles and responsibilities, for example, what a “data analyst” does, what a “data scientist” does, and -- arguably most important -- how roles and responsibilities must include sharing information as well as how data contribute to the organization’s bottom line.
My only (albeit minor) disappointment with the webinar -- admittedly a lot was packed into an hour -- was the lack of discussion of how management approaches must be adapted to the modern organization’s data environment. Still, the intelligent discussion of roles and responsibilities is extremely valuable, as is the emphasis on sharing, collaboration, and targeted use cases to drive development.
In my own work I have found that implementing an appropriate data governance infrastructure depends on several factors including:
How sophisticated the organization already is regarding management and use of data. For example, when is self service appropriate versus when is an internal “service bureau” operation needed?
How disciplined is the organization is regarding how it manages projects and processes that cross departmental and functional lines? Does it have a centralized PMO type operation to support both administrative and operations management, or are project managers “on their own”?
How prepared is the organization to recognize the necessary alignment between front end ETL, quality control, and data transformation practices with downstream value delivery to those in the organization’s executive suite?
This last item will not be a surprise to anyone with experience with organizational change efforts. The number one requirement for a successful project, for example, has always been and will continue to be the maintaining of management support through the ability to contribute demonstrably to the organization’s bottom line. So it is with transitioning to a more mature “data ops” approach.
Building and maintaining an understanding of how data governance -- starting with gathering and processing “clean” data -- aligns with delivering value can a major challenge. It requires not only technology but management sophistication as well.
Based on what I heard in the Data Science Central webinar we’re making great progress — and we still have a long way to go.
Copyright © 2018 by Dennis D. McDonald