The great AI investment paradox.

A report by KPMG noted that only 2% of Canadian companies are seeing a clear return on their generative AI investments. At the same time, an MIT study that’s been widely cited by The Economist and others found that about 95% of enterprise AI pilots fail to generate measurable value.

And yet, we’re living through the frothiest AI build-out in history. Global AI-related data centre and infrastructure spend is already in the hundreds of billions of dollars per year, with estimates that it could reach $3-4 trillion by 2030. Capital is pouring into chips, data centres, and model providers faster than most organisations can write a proper AI business case.

So what’s going on? How can ROI be so anaemic while investment is so extreme?

It's not just people and process… but they still matter

When I work with large companies on tech-enabled transformations, the failure modes are depressingly consistent:

• People: Teams are excited by the demo but unsure how AI changes their day-to-day, incentives aren't aligned, and capability building is an afterthought.
• Process: AI is bolted onto legacy workflows rather than used to redesign them. We automate fragments instead of re-architecting how work actually flows end-to-end.
• Framing: The initiative is treated as a project (“we implemented the tool”) instead of a program (new operating model, new metrics, continuous tuning).

If you only fix the tech, the transformation still fails.

But there’s an uncomfortable truth we don’t talk about enough. Sometimes it really is the technology.

Generic AI often underdelivers in specialized domains

We tend to talk about “AI” as if it’s a singular tool in the box. Buy an enterprise licence, connect your data, and surely one of the big models will be “good enough” for whatever you’re trying to do … right?

That assumption quickly breaks down in specialized, high-stakes workflows.

We put this to the test in one of our own businesses, an AI-first staffing and recruiting agency. We initially evaluated off-the-shelf, general-purpose LLMs for matching potential candidates to job roles at scale. They were impressive as conversational tools, but when we asked them to:

• Interpret messy, inconsistent job descriptions.
• Reconcile fragmented candidate histories.
• Rank thousands of possible pairings by actual hiring signals.

They simply weren’t very good. They hallucinated skills, over-indexed on keywords, and struggled to consistently separate “great” from “good enough” candidates—a pattern that independent reviews of off-the-shelf matching engines have also highlighted.

So our Engineering Teams went a different route.

The hidden constraints of today's LLM architecture

Most of today’s leading AI systems are built on a transformer architecture, a breakthrough model design that enables LLMs to read enormous volumes of text, understand relationships between words, and generate human-like responses. Transformers are extraordinarily good at language: predicting the next word, summarizing documents, drafting emails, or reasoning through text-heavy questions.

But that strength is also their constraint. Transformers are optimised for linguistic prediction, not for specialized, structured, or domain-specific decision-making. And when enterprises assume a chat-optimized transformer can automatically solve complex operational problems, this architectural mismatch becomes quickly visible.

That’s exactly what we encountered.

Instead of treating a chat model as a Swiss Army knife, the solution was to create a fit-for-purpose model designed for one mission-critical workflow: evaluating and ranking candidates against thousands of open roles.

. . . success didn't come from “using AI.” It came from using the right kind of AI . . .

To do that, our Engineering Teams:

• Re-architected the underlying transformer models to ingest both structured and unstructured data—skills taxonomies, work histories, rate cards, and job attributes, not just free-form text.
• Changed the learning objective from “predict the next word” to “predict the quality of a match,” with real hiring outcomes serving as the training signal.
• Injected deep domain knowledge about how creative, marketing, tech, and emerging professional roles are actually filled, rather than relying on patterns scraped from generic internet text.
• The result: our proprietary AI models consistently outperform general-purpose LLMs and basic keyword search in identifying the right candidate for the right role.

In other words, success didn’t come from “using AI.” It came from using the right kind of AI, embedded in a redesigned process, with people, data, and operating models aligned around it.

What this means for leaders staring at that 2% ROI number

If you’re a CEO, CHRO, or CMO looking at AI budgets and wondering whether you’re in the 2% or the 95%, a few questions are worth asking:

• Is this a one-off implementation or an ongoing program?
  Are we funding change management, capability building, and continuous model tuning, or just the software licence?
  
• Are we using generic tools for specialized jobs?
  Where are we relying on off-the-shelf, general-purpose models for deeply domain-specific tasks (risk, pricing, workforce planning, matching), and what evidence do we have that they're actually good at those tasks?
  
• Have we re-designed the workflow, or just added a bot?
  Are we rearchitecting how work happens across people, process, and tech—or asking employees to “sprinkle AI” on top of what they were already doing?

AI isn’t failing because the technology is inherently over-hyped. It’s failing because we’re often deploying the wrong class of technology, in unchanged workflows, with unprepared people.

People and process will absolutely break your transformation if you get them wrong.

But as we learned first-hand building an AI-powered staffing business, sometimes the answer really is: you need different technology … purpose-built models, tuned on your domain, integrated into how your organization actually works.

That’s where the next 2% of ROI is going to come from.

Leadership & Management (126)
Engineering & Technology (52)
Innovation & Emerging Tech (68)