Analytics & Big Data
Mar 28, 2016

Balancing speed to insight and accuracy of insight

March 28, 2016 - Suppose we had a vast amount of time to undertake an analysis. What would the analytical approach include? It would likely consider a wide set of data, examine alternate hypotheses, functional forms, and methodologies. It could potentially include comparisons to other research and contrast against prevailing opinions.

Alternatively, what if we had limited time to undertake the same analysis? In such a situation, we would need to prioritize the research to the most important components, issues, and questions. We may have to make assumptions to accelerate the computations or cut corners that may violate best practice.

In a third scenario, what if we have an extraordinarily and even unrealistically short amount of time to undertake the same analysis? We would potentially have to reinvent the approach altogether, build the analysis from a combination of select data points and expertise in the problem space. Rather than simply cutting corners, the analysis would likely require a far greater degree of art than science.

These are common situations. Hardened analysts are often frustrated when timelines do not allow for full-blown, unbiased analytics. However, pressures from the business, competition, and marketplace require insights immediately. In fact, some analytics, as powerful as they are, are only valuable when delivered in real time. Given the lag in acquiring some data and the time required to conduct some analytics, the outputs can be too dated to drive real value. Hence, analytics has a problem:

  • The most accurate, advanced, and holistic analytics could be less timely to share with the stakeholder and could therefore hold less value
  • The less accurate, more assumption-based analytics could be faster to share with the stakeholder and could be more valuable, yet less accurate

The business impact of these contrasting situations is not, however, clear. Given this common reality, there are several questions analytics stakeholders need to consider when scoping any analytical project:

  • How quickly do they need the results?
  • How much inaccuracy can be tolerated?
  • Which of the previous two questions/answers is more important? What is the optimal balance between speed and accuracy (see figure 1)?

Figure 1: The balance of analytics approaches


Many companies invest heavily in a variety of real-time management dashboards that provide holistic, yet simplistic, views of the environment. Many companies also invest heavily in analytical teams and tools that supplement the dashboards with customized and more accurate analytics. Yet the analytic teams deliver insights slower than dashboards, while dashboards deliver less accurate insights than the analysts but do so in real time. As analytics, data, and technology evolve, the trade-off changes between fast and slow, empirical results and assumptions, and accurate versus imprecise outputs.

The chasm between the competing needs of speed versus accuracy is closing as big data becomes ubiquitous, advanced analytics powers automated analytical engines, and smarter auto-learning analytics procedures, such as machine learning, are adopted. Advances in analytics and the supporting infrastructure allow for advanced analytics to be performed more quickly than ever before:

  • Analytics that used to take weeks now take days. In the future it should take hours
  • Projects that used to take months now take weeks. In the future it should take days

Most dashboards are powered by cross tabulations and simple analytics. For this to become a reality, businesses need to:

  • Invest not only in dashboards for speed and advanced analytics for accuracy, but also in powering dashboards with advanced analytics
  • Operationalize custom analytics that is currently done in an ad hoc manner
  • Invest in technologies and processes that reduce the bottlenecks of human intervention, data manipulation, and data cleansing

Competitive advantage is not typically a function of product or price, the size of the marketing or sales budget, or even customer loyalty. Instead, competitive advantage is a function of how accurately companies assess their data and market, and how quickly they act on that information.

It is the company that simultaneously assesses its data most accurately and does so in real time or near real time that has the greatest advantage.

About the author

David Hauser, PhD

David Hauser, PhD

Chief Science Officer, CPG and Specialty Analytics

Follow David Hauser, PhD on LinkedIn