Why People Are Paying 10x More for AI | Sid Sheth, d-Matrix

Eye On A.I.

1 DAYS AGO

Overview Shownote Highlights Transcript Chapters Pins

In this episode, d-Matrix CEO Sid Sheth joins Craig Smith to challenge the assumption that NVIDIA's dominance defines the entire AI chip market. He argues that the industry is splitting into two distinct tiers, with a rapidly growing segment focused on low-latency inference that GPU-based systems are structurally unable to serve. The conversation explores this 'premium token economy,' where speed and interactivity are the product, and also offers a candid look at how AI is reshaping executive decision-making, from generating complex M&A plans in minutes to acting as a debate partner that pushes back on ideas.

Sid Sheth explains that the AI chip market is bifurcating into general-purpose GPUs and specialized low-latency chips needed for interactive, agentic AI applications. This demand for instant responses is creating a 'premium token economy' where users pay significantly more for speed, a segment d-Matrix's architecture is designed to capture. To scale, d-Matrix acquired GigaIO and launched the Proteus rack, recognizing that racks are now the fundamental unit of compute deployment. Sheth also discusses the evolution of AI from an echo chamber to a genuine debate partner, citing its ability to generate comprehensive M&A integration plans in 15 minutes and its growing willingness to disagree. He predicts a shift toward 'organizational AI,' where teams of agents run entire company functions. The conversation also covers the return of AI workloads to private data centers for large enterprises, the strategic choice to avoid bleeding-edge processes and HBM technology for manufacturing advantages, and the vision of AI as a tool for broad societal benefit.

03:25

Low-latency inference creates a premium token economy.

27:06

AI assistants have evolved from echo chambers to debate partners.

29:47

AI serves as an overlay on existing infrastructure