scripod.com

6 in 10 Enterprises Can't Find the Root Cause When Their AI Workloads Fail | Paul Appleby, Virtana

Eye On A.I.

Shownote

Companies are spending billions building AI factories, but most of them can't tell you why their AI workloads are failing, whether their GPUs are actually being used, or what their infrastructure is going to cost them when agents start running at scale. Paul Appleby, CEO of Virtana, joins Craig Smith to discuss the findings of their AI Factory Reality Check study, a research report that reveals a striking and underappreciated gap between the pace of AI infrastructure investment and the governance needed to run it safely and efficiently. Six in ten enterprises, the study found, cannot automatically identify root cause when an AI workload fails, a problem that compounds fast once you're running critical services on AI infrastructure at scale. The conversation covers the mechanics of Virtana's observability platform, capturing 20,000 metrics per second across the entire AI stack, correlating them in real time, and increasingly using agentic capabilities to remediate failures automatically, but its most important insights are structural. Appleby makes a sharp observation that cuts through a lot of AI optimism: token costs are falling, but token consumption is exploding, meaning the total cost of running agentic AI systems is still going up even as the per-unit price drops. He also tracks a cultural shift inside enterprises - IT resilience reporting that used to happen annually now happens weekly - as evidence that technology risk has become a board-level conversation in a way it simply wasn't before. The result is a conversation that's less about the promise of AI and more about what it actually takes to make it work at production scale. Subscribe to Eye on A.I. for weekly conversations with the people building and deploying the future of AI.

Highlights

In this episode, Paul Appleby, CEO of Virtana, discusses the critical gap between massive AI infrastructure investments and the governance needed to manage them effectively. Drawing from Virtana's AI Factory Reality Check study, the conversation reveals that most enterprises lack the ability to automatically diagnose AI workload failures, leading to significant operational risks. Appleby explains how his company's observability platform captures thousands of metrics per second to provide real-time insights, and he offers a sobering perspective on the true costs of scaling AI systems.
07:54
60% of enterprises cannot automatically identify root causes.
28:24
Falling token costs lead to higher total spending
40:12
Edge AI and physical AI will develop in parallel.

Chapters

Why 60% of AI Workload Failures Remain a Mystery
00:00
The Hidden Cost of Cheaper Tokens: Why Your AI Bill is Still Going Up
24:46
Inside the AI Factory: Observability Across a Hybrid, Heterogeneous Stack
36:10

Transcript

Craig S. Smith: Six in ten enterprises cannot automatically identify the root cause across AI infrastructure domains when AI workloads fail. Why a root cause is so much harder in an AI factory than in traditional enterprise IT to discover? What you're sayi...