CIOs struggle to define AI value

New research from revenue intelligence leader Gong reveals widely varying viewpoints among CIOs and other tech leaders over how to evaluate the success of AI projects. Surveying over 500 CIOs and heads of IT across the UK and US, the findings illustrate the challenge many businesses are facing when it comes to strategically implementing AI, and the uncertainty in measuring whether those AI investments are paying off.

While over half of CIOs (53 percent) prioritise productivity gains, an equal proportion focus on revenue growth as their key success metrics, with worker satisfaction trailing closely behind (46 percent). This divergence underscores a broader challenge: confusion about where AI can deliver the most business value and a well-defined approach for evaluation.

Key insights from the study include:

• Revenue Growth vs. Time Savings: 61 percent of global CIOs believe increased revenue alone justifies AI costs, while 60 percent say that time savings alone will justify costs. Yet, only 32 percent actively measure both, suggesting that many companies still don’t have systems in place to measure and assess the impact on the variables that they say matter most.

• A Growing Interest in Predictive AI: It turns out that while generative AI attracts much of the buzz around the technology, it is not the clear leader among CIOs in terms of driving value. 54 percent of tech leaders report they prioritise generative AI, 51 percent prioritise automation, followed by 31 percent prioritising predictive AI. To capitalize on this discord and deliver value across a broad spectrum, AI models must be tuned to support workflow automation and predictive analytics.

• Adoption of Domain-Specific Solutions: While nearly three-quarters of tech leaders rely on off-the-shelf Large Language Models (LLMs) as part of their AI investments, 58 percent are utilising domain-specific solutions. These AI tools are trained on industry- and function-specific data to deliver more precise and measurable results.

• Security is a Key Obstacle…: Security remains a top priority for 68 percent of tech leaders, but 28 percent admit this is where their AI projects most often fall short.

• …As is Data Integration: Data integration challenges also threaten project success, with 36 percent of CIOs likely to pause initiatives if implementation complexities arise. Without the right underlying data, AI outputs are at risk of delivering little value, or worse – biased or inaccurate results.

• AI’s Long-Term Value Persists: Despite mixed measurement strategies, only a small fraction (under 20 percent) cited a lack of provable ROI as a reason to abandon AI initiatives, indicating most companies continue to explore its potential and long-term value.

• Smaller companies are more eager to prove ROI: Smaller US firms (250-500 employees) are more ROI-focused, with 40 percent willing to halt projects lacking clear ROI, compared to just 19 percent of larger companies. This suggests that while smaller US firms see the value in making investments in AI, they need to focus on initiatives that deliver measurable and immediate returns and have less budget for experimentation, whereas larger companies might have more capacity to invest in long-term projects without immediate ROI.

“Over the last two years, the AI hype and pace of innovation has created incredible excitement and confusion for CIOs and tech leaders about its potential and where to focus,” said Eilon Reshef, co-founder and Chief Product Officer, Gong. “But one thing is clear: eaders are pursuing value and exploring different areas across the business where AI can have a transformative impact.”