Tips to make Your Next Data Science Project A Success

Posted by admin updated on 04 Oct, 2018

We live in the era of data. It is creating a transformative impact on performance and results across multiple sectors. Politicians use data to win elections, CEOs to increase profitability, genetic scientists to improve human health and athletes to win trophies. The Economist describes data as the fuel of the future, and if data is a fuel, then data science tools – like artificial intelligence and machine learning – are the engines that use it to improve our lives. It’s no surprise, then, that the data science industry is booming – it is estimated that the global analytics market will be worth almost $67 billion by 2019.

Despite the potential, most data science projects do not deliver on their promise. According to one estimate, nearly 85% of data science projects fail. This happens for a variety of reasons, ranging from suboptimal use cases to having unrealistic expectations to poor project management. Having been involved in the data science field for 17 years (and still going strong!), I have had front-row seats to a wide variety of data science projects. I have seen projects that have delighted even the harshest critics and have witnessed some spectacular crash-and-burn scenarios. Over the years, I have picked up some common approaches to adopt and pitfalls to avoid.

In this blog, I am sharing these so that your next data science project isn’t a gamble.

Estimate The ROI Before Starting

In many cases, the first mistake teams make is to start a data science project without estimating its impact and comparing against other options. The best way to approach this is like any other investment decision – consider all options, evaluate costs and expected benefits, estimate success likelihood and choose the most attractive option.

One key factor to consider is the potential for impact. For example, if you are considering whether to focus on customer churn or on better prospect targeting, think about the potential upside. What percentage of improvement is each project likely to drive? What does each percentage point of reduction in cost of acquisition mean for your bottom line? How about 1% reduction in customer churn? Simple back-of-the-envelope calculations work great for this exercise.

Start Small Wherever Possible

With data science projects, there can be a temptation to over-invest and make sweeping changes. This risk is further compounded by technology vendors that try to sell expensive “transformation” projects. This can often backfire by causing disruptions with no real return on investment (ROI). Every data science project need not require a complete overhaul. Smaller projects can be delivered within existing systems or, with cloud computing, using temporary systems.

However, small projects do not mean unimportant projects – many can generate compelling ROIs and impactful insights. My rule of thumb for a “beginner” data science project is that it should generate at least 10 times the ROI and take three to four months to execute. For example, developing a predictive model to rank order prospects or designing a smarter credit scoring algorithm that reduces loan losses. These are examples of important projects that don’t require an overhaul of existing systems.

Plan Implementation

Even if a project is well-conceptualized, another threat to its success is execution. In its essence, data science generates and uses data to improve decision making. This is a two-part process that involves generating insights and then acting on these insights. Companies often make the mistake of ignoring the latter. No matter how illuminating an insight or an algorithm may be, it still needs to be acted upon. I recommend planning backwards from day one. Assume that you have the final outcome ready with you. How will it be implemented? What specific decisions will be made differently? What tools and technology platforms will be used? What are the constraints of these platforms? What data feeds will be required? Think through these questions upfront and you will have an easy ride later on.

Manage Stakeholders, Measure Outcomes

Executing successful data science projects requires stakeholder involvement. Hear out all perspectives, including those from the naysayers. Clarify bandwidth expectations and get a commitment from managers whose teams will assist you. Outline the expected outcome and lay down a clear measurement strategy. To achieve this, and I can’t stress enough, it’s importance to set up well-designed tests. Using a control-test setup, you can measure the improvement arising out of the project, which can help silence critics and get buy-in for the future.

Focus On Project Management

A continuous project management approach is essential, but data scientists usually don’t recognize the importance. Share regular updates with stakeholders (I suggest weekly). An open line of communication can be effective, as long as it covers the following: overall status, expected end date, risks, dependencies and current workstreams. While creating the project plan, remember that data will be messier than you think, resources will periodically get pulled into other initiatives, and people will have differing opinions on final usage. Plan accordingly. Escalate issues before they snowball into show-stoppers and manage expectations by not announcing kick-off until all pre-requisites are in place.

Conduct Due Diligence

One last tip is to conduct due diligence. If there is vendor involvement, then it is critical to research these vendors. Given the growth of the sector, there is now a plethora of vendors to select from. Big industry names can be attractive but may not be a guarantee of success. In fact, specialized players may bring unique expertise and create value. I suggest interviewing the actual team members that will work on your project and asking potential vendors for client references. Even when data science projects are in-house, a similar approach – where project team members are carefully selected – should be adopted.

These are a few tips that can help in ensuring success when you begin your next data science project. Data science may seem complex, but the management of such projects need not be as complex. On its own, data science is not the solution but it’s an important tool. And, like every tool, it needs to be used with care, precision and rigor. Good Luck!

This article was originally published on, you can read the original article here.