Dec 31

Dec 31 Developing a Basic Model for Data Analytics Project Selection

BDPM, Big Data, Business Models, Cloud Computing, Cost Analysis, Customer Support, Data, Data Access, Data Lake, Internet of Things, Project Management, Data Program Management

By Dennis D. McDonald. Follow me on Linkedin, Twitter, and Google+.

Introduction

The US Government’s Social and Behavioral Sciences Team (SBST) was formed to apply the findings of behavioral and social sciences to improve how people interact with government services. Its description of how it identifies opportunities where it can work best is described on its web site:

How does SBST identify where to apply behavioral insights? The most promising opportunities have three features:
Policy Goal: Successful projects require well-defined policy goals with defined outcomes of interest - in part because SBST itself does not set policy.
Individual Behavior: A link between a policy goal and the decisions or actions of individuals is the essence of an SBST project. Where Federal policy outcomes do not depend on behavior, applications of behavioral insights will have little relevance, no matter how worthwhile the goal.
Program Touchpoint: A point of direct interaction between the Federal government and individuals, such as an application process or form, a website that offers program choices, or the structure of incentives (financial and nonfinancial).

A thought experiment

Let’s adapt these three points to help guide an imaginary consultant who is helping an organization, whose management is interested in “big data,” decide where to apply improved data analytics to the organization’s operations.

Right now the organization might not be interested in enterprise wide changes to its data management processes and systems but would like to see practical examples of improvements based on better uses of its internal and external data. Based on this experience the organization might then try to expand its data analysis efforts and/or to develop a more strategic approach to data management and governance, possibly involving a move to “big data” approaches. Picking the initial applications will be an important planning exercise.

I’m of the opinion that it’s always possible to do a better job of making use of data to guide planning or management. As the number of options have grown for making both structured and unstructured data useful, so has the need to do a better job “at the front end” of deciding where to put the organization’s resources in terms of improving how data are used. Attempting too much will inevitably lead to disappointment. On the other hand, attempting something trivial won’t make a positive impression or generate trust and enthusiasm.

Here is my first cut at modifying the above three points to help guide the consultant, along with my comments:

Policy Goal: Successful projects require well-defined policy goals with defined outcomes of interest.

This requires little modification from what SBST suggests. You have to be able to define whether or not a project is successful. Having target metrics in mind is an obvious requirement if you intend to actually measure performance or want to impact decisions or behaviors. Based on my own experience with statistical and financial data projects, however, linking data use to actual measurable outcomes is easier said than done — which brings us to the second point.

Individual Behavior: A link between a policy goal and the decisions or actions of individuals is the essence of a successful data analysis project. Where outcomes do not depend on behavior, applications of behavioral insights will have little relevance, no matter how worthwhile the goal.

Remember, SBST is addressing behavioral impacts. Not everyone will be in a position to make this restriction. Many uses of data may not link directly to behavioral outcomes but to intermediate rules or processes that only indirectly impact behaviors, some of which can be automated. That said, most organizations are attempting to impact the behaviors of individuals either inside or outside the organization. Improving how employees work is one example. Influencing client, user, or customer behaviors is another, even in cases where a “B2B” (business-to-business) relationship is the focus. In many if not most cases better data or data analytics by themselves will not be what ultimately impacts a target user group’s behavior, which brings us to the third point.

Program Touchpoint: There must be a point of direct interaction between the organization and individuals, such as a person, callcenter, application, process, form, or website that offers program choices, benefits, or incentives that are viewed as useful or beneficial.

I’ve modified this one to make it a bit more generic. All the customer or user “touchpoints” mentioned here can be directly or indirectly influenced by data. Interactions with these touchpoints can have measurable impacts on the behavior of individual users or customers whether these individuals are self-servicing via a website or app or are availing themselves of the help provided by a callcenter rep using better data to guide the interaction process.

Note that the beneficiary of improved services may have no idea that better data have had a role in an improved outcome; that’s an inevitable result of how data are used. This ”invisibility of data” is one of the reasons it is sometimes a challenge to convince management to spend money on improving data-dependent programs. Explicitly connecting better data services and analytics to measurable policies and goals via a touchpoint that can be monitored and controlled will help the project planning and justification process.

Discussion

One of the benefits of focusing on behavioral outcomes as a way of assessing the effectiveness or usefulness of improved data analytics is that behavioral outcomes are potentially measurable. Measurement of data-impacted behavioral outcomes may not be easy as many factors can influence how the user interacts with the touchpoint. Making an interaction with a system or process faster and more efficient may not actually make that program more effective in the long run if, for example, the benefits of the products or services thus obtained are not substantial, not what was intended, are unsustainable, or are outweighed by their costs.

Even when focusing on intermediate goals such as process efficiency, the cost of using data to increase such efficiency is potentially measurable and is something that can be assessed objectively. Performing a process more efficiently means that a program can provide more services to more people at less cost, something that is in most cases viewed as a positive outcome.

This does presume that cost and sustainability are a considerations in selecting and managing data analytics projects. This may not always be the case initially since technical feasibility and user acceptance may be more important. Eventually, however, cost and sustainability will need to be addressed along with technical feasibility and user acceptance, especially if potentially expensive changes to data governance systems and processes are considered and made part of a business case based on the “lessons learned” in initial data analytics projects.

Caveat

So far I have not addressed the types of data analytics projects we’ll be attempting but have been addressing the context in which improved data analytics will be used. Assuming our initial forays into better analytics will be in terms of “bite sized chunks” of work that focus on individual cases —- an approach recommended by Curtis Franklin Jr. in Intuit CIO: Changing The Way Projects Are Managed —- we will also need to be considering the approaches we’re taking to make data more useful, for example:

Are we looking to make existing data more useful and understandable?
Are we attempting to better represent current operations or transactions through near-real-time data visualizations?
Are we building a predictive model based on past experience to help anticipate resource demands?
Are we trying to understand the relationship between passively collected remote sensing data and the actual behavior of our customers?

Given the different technical and resource implications of project type I’ll be addressing this topic in future posts.

Summary

In this paper I have adapted a simple model to help select projects that involve the development and provision of improved data analytics. With additional work such a model can be further refined to help in assessing and comparing different opportunities throughout an organization that is considering “testing the waters” for applying modern data science and big data management techniques.

While much can be learned by performing a series of quick projects to help with various high priority operations throughout the organization, it is important to realize that eventually improved data management and governance may have to be addressed at the enterprise level. Realizing this right from the start while focusing on measurable impacts linked to important organizational objectives will improve the likelihood of success both initially and in the long run.