How to get the most from your advanced analytics projects
Successful data-driven businesses focus most of their attention and effort on asking and answering actionable, valuable business questions. This all sounds straight forward and driven by common sense, but many organizations starting on their advanced analytics journey struggle with identifying and prioritizing these questions. This article will share our framework for defining, refining, and assessing the value of business questions that are candidates for advanced analytics projects.
What is a Valuable Business Question?
You hear a lot about “actionable insights” but not a lot about how you get to that insight except for vague statements like “use our tool” or “work with us”. So, how do you really gain the insight you need? By making sure you are asking a valuable question.
How do you know if you’re asking a valuable business question? Our favourite way to recognize a valuable business question is that, when answered, a specific action can be taken which provides a measurable result in line with strategic business objectives.
In advanced analytics, an action can be accomplished through:
- Modifying one or more processes based on the actionable insight OR
- Operationalizing a data product (a classic example would be a recommendations engine)
To make sure measurable results are in line with your strategic business objectives, you should invest time in documenting and prioritizing your key business drivers. Then, use those drivers to help shape the business questions being asked. A workshop with executive and senior business stakeholders is often the best approach for this important first step to ensure you capture them and reach consensus on their relative priority.
Finding Valuable Business Questions
Use the approach pictured above to:
- identify candidate business questions
- refine them to make sure they meet the definition of a valuable business question and
- evaluate the candidate business questions to determine just how valuable they are, if answered
Let’s look at each of the three phases in more detail:
You’ve documented the business drivers and strategic objectives. Now, the focus changes to identifying potential questions to ask. This can be done through a Business Value Workshop involving business subject matter experts. These workshops need strong facilitation to make them effective. To set up for success, you must address three things before the workshop happens:
- Who to invite?
Business end users and subject matter experts – those who know the processes and data best. We often find only senior resources attend (especially if this is the first foray into advanced analytics). It’s great if they are there for decision making and oversight, but you really need the end users too – they are the VIPs in the room that day.
- What do they need to before they arrive?
Share the business drivers and strategic objectives if they were not involved in capturing them.
- What should they bring?
Lots of available background material. Things like:
- Process maps for the processes involved, if available (if you are targeting process change, you need to know what the process is)
- Data dictionaries, if available
- Sample data that would be needed
- Sample reports/dashboards/KPIs that are in place today for the processes, and related calculations
- Known gaps or inconsistencies
If the materials don’t exist, a gap analysis is done to determine where you need to invest time and money to get the information you need to answer the questions. Asking for the information ahead of time will, typically, improve the quality of the workshop results.
The first hour of the workshop should be spent brainstorming ideas and capturing business questions. We often like to group the results of the brainstorming session into these categories:
Non-starters are drowning in vagueness and meet none of the attributes of a Valuable Business Question – actionable, measurable result, aligned with strategic business objectives. Some examples:
- Can we show value with tool X?
- Data science helps businesses – how can it help ours?
Preconceived questions are, by far, the most dangerous. They sound like you will be building something valuable and are relatively easily sold to decision makers because they sound good and seem forward thinking and cutting edge. Some examples:
- We need to use deep learning to do some stuff…
- Produce an algorithm that can be used, season to season, to set prices when new products become available in the market from us or competitors
Often these questions arise because participants are focused on technology first, rather than on business problems. These need to be nipped in the bud quickly or they can lead to wasted dollars with no real results.
These are questions that are looking promising but to meet our definition of Valuable Business Question, you need to do some more work before making that decision. Ideally you would like to come out of the workshop with at least 5 of these. Some examples:
- Who are the biggest users of healthcare in X field?
- How can we save money on our preventive maintenance?
Refining is all about taking those “Requires Refinement” questions and digging deeper, asking probing questions to turn them into candidate business questions. We call them “candidate” business questions because their value hasn’t been proven – yet! That is what the Evaluating stage is all about.
Refine the questions and turn them into candidate business questions by asking the 5 Ws and determining some level of feasibility. Questions like “Why does it matter?”, “Who matters?”, “What is the definition of …?”, “Do we have that metric or that data?”
Through the course of questions and responses, the goal is to get to a point where the original question can be rephrased to become a good candidate business question. As an example, the requirements refinement question was “We want a machine learning model that predicts customer churn four weeks in advance of the event, with an accuracy of at least 80% and lift curves graphed to demonstrate why you picked this model.” True story! Through the refining process it was rephrased as “Can we predict which of our high-value customers will leave in two months?”
Evaluate & Prioritize
With a set of refined candidate business questions, it is time to evaluate and prioritize them. Often it is effective to evaluate the relative importance of the candidate business questions using three prioritization criteria:
- What does it mean to the business if you can answer this question?
- Do you have, or can you acquire the data, skills, and tools needed to answer the question?
- How well does the candidate business question align with your business drivers and objectives?
For each of these criteria, we look at two parameters – a score and a timeframe.
The Value Score is a measure of what it will mean to your business if the question is answered and ideally this is an estimate in dollars such as additional revenue generated or dollars saved. The Value Timeframe can be relatively simple. As an example, if answering the question will give you a competitive advantage, assign a weighting of 10. If you have to do it because everyone else is (table stakes), assign a weighting of five, or if you are already too late for it to have an impact, assign a zero or one.
Feasibility Score is even simpler. It is answering the question “Do you have or can you acquire the data, skills, and tools you need to answer the question?” If the answer to any of these is no, assign a zero, otherwise assign full score. Feasibility Timeframe is an evaluation of the length of time it will take to acquire the resources. If they are already in place, assign the maximum value. This parameter is usually a relevant ranking across the questions being evaluated.
The Alignment Score is all about how well the question aligns with your business drivers and strategic objectives. It involves mapping each question to the business objectives and calculating a weighted score based on the objectives’ priority. The Alignment Timeframe is a measure of the timeframe within which meeting the objective will be effective, such as Q3 or fiscal year end, etc.
Assigning the score and timeframe values for each of these criteria allows you to plot them on a spider graph, as shown below. We like to use a scale of one through 10, but the scale is your choice. This is used to help understand which questions are worth investing in and their relative value – those with the largest area are most valuable. Use score and timeframe values to set priorities for your analytics road map.
Identifying valuable business questions is a lot easier once there is a clear definition of what counts as valuable in your business. It is this critical step that will ensure your data-driven business teams (whether through data science or business intelligence) are focused on the right work.
Now… ask away!