By Dennis D. McDonald.

This following is the Introduction to a special 30-page compilation of blog posts about big data project management. To download a free .pdf version of this compilation click or tap here.

Purpose

To download a free 30-page compilation of blog posts dealing with Big Data Project Management click or tap the above image.I began researching “big data project management” when I started seeing publications and online discussions concerning big data project “failures” being attributed to the classic reasons for project failure such as scope creep, poor stakeholder engagement, and inadequately understood requirements.

“What is it about “big data” projects that invites failure?” I wondered, thinking that rapidly evolving data management technologies and increasing interest in better analytics might be putting strain on “traditional” project management approaches.

Having long been connected as a consultant or employee with projects and organizations involved in data management, distribution, or access, I decided to research the topic of “what’s different” about big data projects, especially those that involve new data management architectures or more sophisticated analysis requirements.

Much of what I’ve found from interviewing colleagues and experts I’ve published on my own website.  Some of the most relevant articles are included in this document.

Acknowledgements

Thanks are due the following for sharing their thoughts with me about big data project management: Aldo Bello, Kirk Borne, Clive Boulton, Doug Brockway, Bob Davis, Ana Ferreras, Keith Gates, Douglas Glenn, Jennifer Goodwin, Jason Hare, Christina Ho, Randy Howard, Catherine Ives, Ian Kalin, Michael Kaplan, Jim Lola, David McClure, Jim McLennan, Trevor Monroe, Brian Pagels, John Parkinson, Dan Ruggles, Nelson Searles, Sankar Subramanian, and Tom Suder.

I’ve benefited greatly from the diversity of experience and opinions of this group. Naturally, any errors or misunderstandings are my own.

Next Steps

I’ve been a project manager and management consultant for several decades and have moved from an initial focus on statistical and quantitative research to database publishing, systems integration, and IT strategy work. While I’ve found that “big data” projects are making use of newer technologies that are undergoing a variety of different learning and adoption curves, the processes of how one manages a big data “project” or develops a big data “program” may not be that different from managing other tech-related projects.

What is different, I’m finding, is the planning that needs to take place at the front end, balanced with the need to deliver results rapidly while maintaining a reasonable focus on enterprise level data governance.

I’m now turning my attention to understanding how this planning process can be implemented so that the projects that emerge from the planning process are effective, efficient, and successful. Please contact me if you need help doing this.

Dennis D. McDonald, Ph.D. (email ddmcd@outlook.com phone 703-402-7382) is a management consultant based in Alexandria, Virginia. His experience includes consulting company ownership and management, database publishing and data transformation, managing the integration of large systems and databases, corporate technology strategy, social media adoption, statistical research, and IT cost analysis. His clients have included the U.S. Department of Veterans Affairs, the U.S. Environmental Protection Agency, the National Academy of Engineering, General Electric, AIG, the World Bank, Whirlpool, and the National Library of Medicine. He has worked as a project manager, analyst, and researcher throughout the U.S. and in Europe, Egypt, and Hong Kong.