54M Data Workers Not Happy with Data Tools, Knowledge

We live in a data-centric world. Many digital transformation initiatives are focused on maximizing the data advantage.

Yet, a new study showed that 54 million data workers around the world face common challenges. They include complexity, diversity and scale. But their companies are not offering the support they need.

According to the IDC Infobrief, The State of Data Science and Analytics, which was commissioned by Alteryx, inefficiencies, ineffectiveness and wasted time continue to plague DX efforts.

The report noted that 80% of organizations now leverage data across multiple organizational processes. While it is spurring innovation, data workers still waste 44% of their time each week because they are unsuccessful in their activities.

For example, data workers spend more than 40% of their time searching for and preparing data instead of gleaning insights. On average, they use four to seven different tools to perform data activities, adding to the complexity of the data and analytics process.

The top frustrations cited by data workers in the survey are indicative of root causes that are responsible for inefficiencies and ineffectiveness. For example, more than 30% of data workers say they spend too much time in data preparation, a task that can often be automated.

Eighty-eight percent of data workers use spreadsheets in their data activities. Spreadsheet functions are often used as a proxy for data preparation, analytics and data application development tools but are error-prone and expose the organization to compliance and trust issues.

“Data is at the core of digital transformation, but until organization leaders address these inefficiencies to improve effectiveness, their digital transformation initiatives can only get so far,” said Stewart Bond, director of data integration and integrity software research at IDC. “Consolidating platforms and looking for tools that address the needs of any data worker, whether a trained data scientist or an analyst in the line of business, can help reduce the friction that many organizations experience on their path to becoming data-driven.”

The survey found that data workers are unsuccessful for a variety of reasons. They include lack of collaboration, knowledge gaps and resistance to change. Survey participants also reported the lack of creative and analytic thinking, analytic and statistical skills, and data preparation skills as the highest-ranked skills gaps responsible for productivity issues. It shows the pervasive talent gap that exists between data scientists and data workers in the line of business. 

“Collecting data alone won’t digitally transform a business and the answer is not as easy as hiring a leader, a few data scientists or over-investing in disparate technologies. The key is to empower all users, many of whom are currently stuck in spreadsheets, to analyze data effectively to drive real, business-changing results,“ said Alan Jacobson, chief data and analytics officer (CDAO) of Alteryx. “As the data landscape becomes more complex, this survey exposes the tip of the iceberg when it comes to the sheer volume of workers needing to conduct analysis on a daily basis and the untapped potential for them to drive meaningful business impact.”