What Drives Data Science Initiatives Failures?

In the age of AI and ML hype, a specific category of headlines that get good attention are the ones that describe failures of AI and ML algorithms in the real world. These articles on data science initiative failures are generally written for a generic audience and sometimes may give an impression that the failures result from AI algorithm itself. Remember that an algorithm performs the way it was designed to perform. And we, humans, designed it. However, despite you hearing all the time that AI can think like a human, or outperform humans, the fact is that these algorithms are designed by humans. And humans are prone to errors and mistakes.

The Four Ds of Data Science Initiatives

In my opinion, a data science initiative has four critical stages and the failures result from mistakes made in mostly three of these phases. Those four stages are:

• Discover: In this phase, you identify the problem, dissecting it from the noise of a broad strategic or operational problem.
• Decode: In this stage, you translate the problem into a data science solution, incorporating the real-world nuances.
• Develop: This is the stage where the solution is developed in terms of its infrastructure, data pipelines, and algorithms.
• Disseminate: Making sure that the solution goes into successful production and penetrates the trust factor of stakeholders who will put it into production and the user base

The Three Ds That Often Lead to Failures

As you can decipher from these phases, these phases also pertain to unique skills. Often, data science failures stem from mistakes made in the three Ds of discover, decode and disseminate. And this is where the skill association with phases becomes very important. An ideal data science initiative should be staffed in a way such that it has strong skillsets in all the four key areas identified here. The video below explains these stages in detail and highlights which stages often are ignored in the real world. https://vimeo.com/718892475