scripod.com
AI Engineering with Chip Huyen

Overview

Shownote

Highlights

Transcript

Chapters

Pins

AI Engineering with Chip Huyen

The Pragmatic Engineer

2025/02/05
The Pragmatic Engineer

The Pragmatic Engineer

2025/02/05
OverviewShownoteHighlightsTranscriptChaptersPins

Shownote

Supported by Our Partners • Swarmia — The engineering intelligence platform for modern software organizations. • Graphite — The AI developer productivity platform.  • Vanta — Automate compliance and simplify security with Vanta. — On today’s episode o...

Highlights

In this episode of The Pragmatic Engineer, host interviews Chip Huyen, a computer scientist and author of the new O’Reilly book *AI Engineering*. Chip brings a wealth of experience from her time at Netflix, NVIDIA, and as a co-founder of Claypot AI, along with teaching machine learning at Stanford. The conversation offers a deep dive into the evolving field of AI engineering, exploring practical applications, common pitfalls, and how professionals can navigate this fast-moving domain effectively.
00:00
Building AI applications now has a lower entry barrier.
04:00
The book includes analysis of over 900 GitHub repositories showing how they've changed over time.
05:01
Chip Huyen started writing her AI engineering book after an existential crisis triggered by ChatGPT.
15:31
AI engineering emphasizes product-first development using APIs before building custom models.
19:30
LinkedIn's job-fit assessments highlight the need for clear AI guidelines and evaluation
22:55
Extracting keywords or metadata can help manage document chunking challenges.
23:44
Hybrid search combines term-based and semantic retrieval for better results
31:00
Start with a human-in-the-loop approach when deploying GenAI chatbots
33:47
Many teams build with GenAI not because it's needed, but due to fear of missing out.
37:02
Smarter AI systems are harder for humans to evaluate effectively.
38:25
AI evaluation should not rely solely on human performance benchmarks.
41:31
Comparative evaluations are promising but require hard work over tool reliance.
49:04
Many companies abandon GenAI due to poor implementation and unnecessary complexity
52:17
Changes in prompts can lead to unpredictable performance changes
56:01
Tutorials often lead to mindless copying without understanding the 'why'.
1:01:09
Coding is a means, not the end goal; problem-solving is the core of software engineering.
1:03:11
AI enables software engineers to handle more complex software while still requiring professionals for business use cases.
1:10:28
Built a custom AI tool to gather paper information rapidly.

Chapters

Intro
00:00
A quick overview of AI Engineering
01:31
How Chip ensured her book stays current amidst the rapid advancements in AI
05:00
A definition of AI Engineering and how it differs from Machine Learning Engineering
09:50
Simple first steps in building AI applications
16:30
An explanation of BM25 (retrieval system)
22:53
The problems associated with fine-tuning
23:43
Simple customer support solutions for rolling out AI thoughtfully
27:55
Chip’s thoughts on staying focused on the problem
33:44
The challenge in evaluating AI systems
35:19
Use cases in evaluating AI
38:18
The importance of prioritizing users’ needs and experience
41:24
Common mistakes made with Gen AI
46:24
A case for systematic problem solving
52:12
Project-based learning vs. structured learning
53:13
Why AI is not the end of engineering
58:32
How AI is helping education and the future use cases we might see
1:03:11
Rapid fire round
1:07:13

Transcript

Gergely Orosz: how would you define ai engineer or ai engineering? Chip Huyen: Yeah, so before when you wanted to build a machine learning applications. You need to build your own models, so that means that you need your own data and you need expertise. H...