How to optimise your Business Intelligence

These days you would be hard-pressed to find someone to argue against the importance of Business Intelligence (BI) to company success, as evidenced by the fact that a new report by Dimensional Research found that 98 percent of companies use some sort of BI today. The insights derived from data analytics are pivotal in constructing a comprehensive and holistic overview of business operations, while they also help enterprises spot emerging trends and drive better business decisions. Whether this is helping to streamline operations by identifying and boosting efficiencies, or delivering actionable reports straight to senior management, these decisions drive revenues and are a crucial factor in maintaining a company’s competitive edge.

It will be worrying to many, therefore, that the same report by Dimensional Research suggests that although BI is clearly well-resourced, investments in BI may well be going to waste.

So where are businesses going wrong?

To obtain actionable insights, the first thing businesses need is data, and lots of it. Luckily, digital transformation has led to a huge influx of information across enterprises. And in a post-pandemic landscape, this trend shows no sign of slowing down. In fact, having access to reliable, recent data sources is now more crucial than ever, as data and models from even just a few months ago are often no longer relevant today.

Once businesses have this information, they need teams ready to derive and leverage insights from the data sources. Data analysts are increasingly recognised as an important resource in unlocking this meaning. This is all as it should be; data analytics delivers business value and therefore the analysts themselves are central to the continued innovation of forward-thinking companies. In fact, Dimensional Research’s research found that 71 percent of companies plan to hire more data analysts this year.

However, the same research suggests that hiring more data analysts may not help organisations extract maximum value from their data. This is because these analysts, although a vital resource, are critically underutilised.

The report, conducted in the Spring of 2020, surveyed approximately 500 data professionals across five continents, and provided a startling insight into the data analyst role itself: data analysts only spend 50 percent of their time actually analysing data. Data analysts are instrumental in unlocking the insights that influence strategic business decisions, but when only half their time is spent on the actual analytics, suddenly it becomes clear that companies are far from unlocking the full potential of this key resource.

Constant delays in extracting data makes things difficult

In the report, data analysts identified data access as one of the top challenges they encounter within their day-to-day workings that stop them from performing their role

effectively. Analysts spend as much as a third of every single workday trying to access the data they need, with data sources cited as being unreliable, broken, and intermittently accessible. The vast majority of analysts (90 percent) said their work had been slowed by frequent unreliable data sources over the past year, slowing them down and clogging up the process. Plus, it only takes one data source being unavailable to create further widespread delays.

Another factor contributing to this delay relates to the fact that data schemas, the blueprints for the way in which data sources are constructed, are in a constant state of flux, as businesses focus and refocus their operations, and re-cut data to unlock new opportunities. This is an essential part of increasing the accuracy of reporting and thus of the decision-making process in a business, and as a result 60 percent of analysts surveyed said they are required to update the schemas every month. However, for the analyst, the constant changing of focus and new data sets being added leads to more work and further delays.

What makes this even more difficult is that the engineering resources to support these updates are not always readily available. More than 60 percent of data analysts surveyed said they wasted time waiting for engineering resources several times each month, meaning the analysts often find themselves performing tasks extraneous to their job description – such as creating reports in Excel because they cannot get to data via a dedicated dashboard – to keep projects moving.

Out of date data is unhelpful and even misleading

Lack of accessibility to data inevitably leads to delays in delivering actionable BI to the wider company. This leads to a significant problem: lags in the process can mean analysts have to produce reports based on information that is out of date.

Out-of-date data cannot contribute to helpful business intelligence. It is therefore practically useless to data analysts, as well as wasting the time it took them to extract the data in the first place. Yet 86 percent of companies struggle with working with out-of-date data. As for companies using their data for AI, using out-of-date data to train AI models is completely out of the question.

Moreover, using out-of-date information to inform business decisions can have a misleading impact on the direction of the business as a whole. In view of the constant uncertainty and rapid upheaval occurring across industries and markets as a result of Covid-19, the fact that 41 percent of data analysts had used data that was two months old or older is a worrying statistic for organisational leaders acting on BI. In a rapidly changing economic landscape such as now, any decisions made based on insights derived from ‘old’ data are likely no longer the appropriate courses to take.

Re-imagine data transit to optimise BI

The true extent of the impact of these inefficiencies on innovation becomes clear in the report, as more than two in three of the analysts surveyed told researchers that they have profit-driving ideas, but not enough time to implement them. This problem will persist even after the hiring of more data analysts. The issue can never be solved this way because each analyst will continue to be slowed down by the same inefficiencies. Data analysts need to be allowed to spend more time analysing, and less time ‘finding’ the data. In other words, if data is going to be used to optimise decision-making (particularly in a quickly evolving economic landscape), data analysts have to be able to extract data ready for analysis in real time.

Automated data integration boosts efficiencies and decision-making capabilities

Companies should rethink data transit with a focus on enabling analysts to carry out their primary role instead of spending precious time finding, fixing and stabilising the data. By automating certain processes – such as if a change in business direction necessitates an altered schema – organisations can remove most or all of the hurdles data analysts are facing today.

With an automated data integration process, data analysts could add new data sources as fast as they need to and extract data from multiple sources – including multiple cloud-based applications in real time – running analytics without wasting their time on engineering busywork, or waiting for data to become available. With 100 percent of data analysts’ time being expended on analytics instead of ETL, businesses can make the best possible use of corporate data to inform business decisions and drive revenue