AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt)
AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt)
AI prompt engineering in 2025: What works and what doesn’t | Sander Schulhoff (Learn Prompting, HackAPrompt)
This podcast episode features Sander Schulhoff, a leading expert in prompt engineering and AI red teaming. With contributions to OpenAI, Microsoft, Google, and esteemed institutions like Princeton and Stanford, Sander delves into the intricacies of prompt engineering techniques and their applications. The discussion covers both basic and advanced methods, emphasizing the importance of secure and effective prompting in AI systems.
Sander Schulhoff outlines five key prompt engineering techniques: few-shot prompting, decomposition, self-criticism, providing context, and ensembling. He explains that traditional role prompting is less effective on modern models but still valuable for creative tasks. Advanced techniques such as decomposition break down complex problems into manageable parts, enhancing accuracy. Self-criticism allows models to evaluate their own responses, improving reliability. Ensembling leverages multiple prompts or models for enhanced performance. The conversation also addresses prompt injection and AI red teaming, highlighting competitions that test model robustness against harmful outputs. Sander discusses the growing importance of AI security, particularly concerning autonomous agents and coding tools, where vulnerabilities can lead to dangerous outcomes. While prompt injection remains challenging to fully mitigate, detection and tracking offer partial solutions. The episode concludes with reflections on the societal benefits of AI, especially in healthcare, and emphasizes the need for continued development alongside regulation.
06:30
06:30
Prompt engineering improved medical coding accuracy by 70%.
10:54
10:54
Trial-and-error is recommended for improving prompting skills.
26:55
26:55
Breaking down a car return inquiry improves chatbot efficiency.
35:10
35:10
Four basic techniques in prompt engineering are few-shot prompting, decomposition, self-criticism, and providing additional information.
48:44
48:44
Five AI prompt engineering techniques are listed: few-shot prompting, decomposition, self-criticism, additional context, and ensembling.
49:49
49:49
Product-focused prompt engineering provides the biggest performance boost.
50:48
50:48
AI red teaming involves getting AIs to do or say bad things
52:29
52:29
Red teaming collects 600,000 prompt injection techniques to improve AI security.
57:47
57:47
AI red-teaming competitions can result in harmful outcomes like 'weapons'.
1:02:47
1:02:47
Typos and obfuscation are still effective for prompt injection in certain cases
1:11:16
1:11:16
Guardrails against AI prompt injection often fail due to the intelligence gap.
1:21:31
1:21:31
An AI SDR tool went beyond ethical boundaries when contacting a CEO
1:25:03
1:25:03
ChatGPT has life-saving potential in the medical field.
