AI Engineering with Chip Huyen
The Pragmatic Engineer
2025/02/05
AI Engineering with Chip Huyen
AI Engineering with Chip Huyen

The Pragmatic Engineer
2025/02/05
In this episode of The Pragmatic Engineer, host interviews Chip Huyen, a computer scientist and author of the new O’Reilly book *AI Engineering*. Chip brings a wealth of experience from her time at Netflix, NVIDIA, and as a co-founder of Claypot AI, along with teaching machine learning at Stanford. The conversation offers a deep dive into the evolving field of AI engineering, exploring practical applications, common pitfalls, and how professionals can navigate this fast-moving domain effectively.
The discussion outlines the distinction between AI engineering and traditional machine learning engineering, emphasizing a product-first approach and API integration over model-building. Chip highlights the importance of starting simple—using techniques like BM25 for retrieval before jumping to complex methods like fine-tuning. She stresses the need to focus on user needs rather than adopting GenAI blindly, citing cases where simpler solutions outperform AI. Evaluation remains a major challenge, especially as AI systems become more advanced and harder to assess manually. The conversation also covers effective learning strategies, advocating for a balance between project-based and structured learning. Despite rapid advancements, the core of engineering—problem-solving—remains human-driven. Finally, Chip expresses optimism about AI’s potential in education and entertainment, while cautioning against over-reliance on tools without understanding their limitations.
00:00
00:00
Building AI applications now has a lower entry barrier.
04:00
04:00
The book includes analysis of over 900 GitHub repositories showing how they've changed over time.
05:01
05:01
Chip Huyen started writing her AI engineering book after an existential crisis triggered by ChatGPT.
15:31
15:31
AI engineering emphasizes product-first development using APIs before building custom models.
19:30
19:30
LinkedIn's job-fit assessments highlight the need for clear AI guidelines and evaluation
22:55
22:55
Extracting keywords or metadata can help manage document chunking challenges.
23:44
23:44
Hybrid search combines term-based and semantic retrieval for better results
31:00
31:00
Start with a human-in-the-loop approach when deploying GenAI chatbots
33:47
33:47
Many teams build with GenAI not because it's needed, but due to fear of missing out.
37:02
37:02
Smarter AI systems are harder for humans to evaluate effectively.
38:25
38:25
AI evaluation should not rely solely on human performance benchmarks.
41:31
41:31
Comparative evaluations are promising but require hard work over tool reliance.
49:04
49:04
Many companies abandon GenAI due to poor implementation and unnecessary complexity
52:17
52:17
Changes in prompts can lead to unpredictable performance changes
56:01
56:01
Tutorials often lead to mindless copying without understanding the 'why'.
1:01:09
1:01:09
Coding is a means, not the end goal; problem-solving is the core of software engineering.
1:03:11
1:03:11
AI enables software engineers to handle more complex software while still requiring professionals for business use cases.
1:10:28
1:10:28
Built a custom AI tool to gather paper information rapidly.