Risk and Uncertainty in Prioritizing Health Data Analysis Initiatives
By Dennis D. McDonald
Once upon a time I was part of a consulting company team brought in to help an insurance company CIO develop a process for selecting IT projects to fund. What I remember most about my experience was the difficulty of handling non-quantifiable benefits once a prospective initiative had been profiled in terms of its cost savings, revenue impact, and multiyear calculations of ROI and TCO. Inevitably, a governing group would gather, review the numbers, and -- as clients are wont to do -- assign their own numbers and priorities, starting with the inevitable priority of all pet projects, "must-have!"
I was reminded of this experience by the recent conference report Population Health: Organizational and Data Governance, and Analytics Strategies that summarizes the 2016 SI Chief Data and Analytics Officers Summit.
Among the main conference findings was the need for a robust and companywide method for prioritizing health data projects. The prioritization steps recommended by the conference attendees are fairly generic:
- Starting with the top, engage all stakeholders around a common vision.
- Have a common understanding of organizational needs, capabilities, and readiness.
- Use a consistent improvement methodology, align incentives, and balance polarities.
- Focus: practice disciplined decision-making to prioritize, find, organize, and sustain improvements.
The conference report goes on to discuss perennially important issues whenever "big data" approaches to health data analytics are discussed, i.e., managing privacy when data are shared across systems, security, and data quality. Managing these challenges can require significant resources, especially when we're thinking about health data and statistics. Also addressed by the conference was the need to incorporate behavioral data and the need to track patients and treatments longitudinally; these add additional complexity to the project selection process.
None of these issues will be a surprise to anyone reasonably familiar with health data and data analytics policy. Of particular importance, though, is the conference’s discussion about the increasing need for an analytical focus on health improvements data versus fee-for-service data. These two clusters of metrics make, potentially, somewhat different data management lifecycle infrastructure demands. One focuses on the mechanisms that support payment for health care services rendered. The other focuses on the outcomes ("performance") of the healthcare.
Obviously we need to focus on both but the reality is that, given the heavy involvement of the private sector in U.S. healthcare, the financial aspects of treatment have in many cases been much better managed data-wise than the follow-up and outcome side of treatment.
Accordingly, an important question that might be addressed if we're really interested in intelligently prioritizing health data analytics projects is this:
- How much do we need to spend in improved data infrastructure (gathering, importing, cleanup, standardization, storage, etc.) before we get meaningful and useful analytical results?
This is always a tough question answer if what we're asking of the data is new, untried, or unknown, or if we're using new, unfamiliar, or experimental analysis techniques. Also, how do we know what we’re going to find from our analysis before we do it?
Some of our proposed projects, for example, might be "reporting" oriented where it’s possible to analyze existing data to address a well-defined cost or time-saving initiative. Existing infrastructure and data quality might be sufficient to support such a project.
Other initiatives might involve more front end investment in data quality or data population completeness before we can even think of using new visualization or analytical techniques.
This is where risk and uncertainty come in as part of the prioritization process along with more traditional estimates of cost and benefits. Ideally, the process by which health and the analytics projects are defined and prioritized will be comprehensive across the organization, will demonstrate alignment with organizational priorities, and will give program and project managers a clear indication of why the initiative has been prioritized and recommended.
At the same time management needs to include for serious consideration projects that involve risk and uncertainty, for in that direction may lie the best opportunities for innovation.
Copyright © 2017 by Dennis D. McDonald
