scripod.com
AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt)

Overview

Shownote

Highlights

Transcript

Chapters

Pins

AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt)

Lenny's Podcast: Product | Career | Growth

2025/06/19
Lenny's Podcast: Product | Career | Growth

Lenny's Podcast: Product | Career | Growth

2025/06/19
OverviewShownoteHighlightsTranscriptChaptersPins

Shownote

Sander Schulhoff is the OG prompt engineer. He created the very first prompt engineering guide on the internet (two months before ChatGPT’s release) and recently wrote the most comprehensive study of prompt engineering ever conducted (co-authored with Open...

Highlights

This podcast episode features Sander Schulhoff, a leading expert in prompt engineering and AI red teaming. With contributions to OpenAI, Microsoft, Google, and esteemed institutions like Princeton and Stanford, Sander delves into the intricacies of prompt engineering techniques and their applications. The discussion covers both basic and advanced methods, emphasizing the importance of secure and effective prompting in AI systems.
06:30
Prompt engineering improved medical coding accuracy by 70%.
10:54
Trial-and-error is recommended for improving prompting skills.
26:55
Breaking down a car return inquiry improves chatbot efficiency.
35:10
Four basic techniques in prompt engineering are few-shot prompting, decomposition, self-criticism, and providing additional information.
48:44
Five AI prompt engineering techniques are listed: few-shot prompting, decomposition, self-criticism, additional context, and ensembling.
49:49
Product-focused prompt engineering provides the biggest performance boost.
50:48
AI red teaming involves getting AIs to do or say bad things
52:29
Red teaming collects 600,000 prompt injection techniques to improve AI security.
57:47
AI red-teaming competitions can result in harmful outcomes like 'weapons'.
1:02:47
Typos and obfuscation are still effective for prompt injection in certain cases
1:11:16
Guardrails against AI prompt injection often fail due to the intelligence gap.
1:21:31
An AI SDR tool went beyond ethical boundaries when contacting a CEO
1:25:03
ChatGPT has life-saving potential in the medical field.

Chapters

Introduction to Sander Schulhoff
00:00
The importance of prompt engineering
04:29
Real-world applications and examples
06:30
Basic prompt engineering techniques
10:54
Advanced prompt engineering techniques
23:46
The role of context and additional information
29:00
Ensembling techniques and thought generation
39:24
Conversational techniques for better results
49:48
Introduction to prompt injection
50:46
AI red teaming and competitions
52:27
The growing importance of AI security
54:23
Techniques to bypass AI safeguards
1:02:45
Challenges in AI security and future outlook
1:05:21
Misalignment and AI's potential risks
1:18:33
Final thoughts and lightning round
1:25:03

Transcript

Lenny Rachitsky: Is prompt engineering a thing you need to spend your time on? Sander Schulhoff: Studies have shown that using bad prompts can get you down to, like, 0% on a problem, and good prompts can boost you up to 90%. People will kind of always be ...