By Dennis D. McDonald

Bernard Marr’s Where Big Data Projects Fail goes over some familiar ground about project failure that’s worth repeating. As common causes of failure he lists the following:

• Not starting with clear business objectives.
• Not making a good business case.
• Management failure.
• Poor communication.
• Not having the right skills for the job.

Anyone who practices project management for a living will recognize this list. It’s not unique to big data analytics project. It is reasonable to ask whether “big data” projects are unique in some way that exacerbates the probability of failure.

One factor might be how you approach managing risk and uncertainty. It’s the nature of many big data efforts that you don’t really know what’s going to come out at the other end. Looking for patterns and trends in large collections of structured and unstructured data — patterns that may not have been recognized before — is by definition an exploratory process. You can’t always predict what you’re going to find.

The better and more experienced your analysts are the better able they will be at working intelligently through a sequence of exploratory steps to locate previously undetected “nuggets” of meaning.

They might also resist providing estimates to management of what they will find and how long it will take them to find it. Management will need to understand this and will have to adjust its oversight and reporting processes accordingly.

Personally I see communication being the most powerful tool the project manager has for reducing the likelihood of failure. This isn’t a unique observation. When entering into projects involving big data, analytics, and uncertainty, we really need to pay attention to how communication takes place and among whom. Communication across both professional and organizational boundaries is especially important so that expectations, risk, and goals are understood by all.

This suggests that a more collaborative approach to project management is what is needed where information is shared across participants and not just in a top-down orbottom-up fashion. You can’t manage a big data analytics project the same way you would a construction project since you won’t always know what you’ll find. Instead, you have to assume that both your work, how you measure success, and how you manage may have to evolve over time.

Copyright (c) 2015 by Dennis D. McDonald. For articles like this scroll down. For information about my consulting go here.