📌 Copy the AI system I use to run a one-person $1M+ business: https://www.behindthecraft.com [https://www.behindthecraft.com] 🤝 Subscribe to my practical AI newsletter read by 150K+ professionals: https://creatoreconomy.so [https://creatoreconomy.so] Shreya and Hamel teach the industry standard course for AI evaluations taken by 4,500+ students from OpenAI, Google, and more. In our interview, I asked them to give live feedback on the AI evals that I built for my skills and demo how anyone can run evals with ChatGPT/Claude and their free evals skill. Don’t miss this up to date primer on evals from the real experts. 🙂 P.S., I asked Shreya and Hamel to give my audience 25% off the next cohort of their AI evals course. It has 4.7 stars with 900 reviews which is almost unheard of on Maven. Sign up via the link here: https://maven.com/parlance-labs/evals?utm_campaign=peteryang&utm_medium=affiliate&utm_source=maven&promoCode=PETERYANG [https://maven.com/parlance-labs/evals?utm_campaign=peteryang&utm_medium=affiliate&utm_source=maven&promoCode=PETERYANG] We talked about: (00:00) Why AI evals are completely different now with the latest models (02:30) Demo: Live audit of the evals that I built for my AI skills (05:14) Top down vs. bottom up evals and why AI sucks at the latter (08:16) Spinning up AI agents to grade each eval criteria (14:32) Demo: Using Shreyas AI skill to run evals in Claude (23:46) How to review output labels to identify failures (34:37) How to turn failures into reusable evals (43:19) Where human judgement is still needed Resources mentioned: Free eval skills from Shreya: https://github.com/ai-evals-course/evals-skills [https://github.com/ai-evals-course/evals-skills] Do automated evals work?: https://parlance-labs.com/blog/posts/auto-evals/ [https://parlance-labs.com/blog/posts/auto-evals/] Thanks to our sponsors: Wispr Flow — 4x faster than typing with your voice — https://ref.wisprflow.ai/peteryang [https://ref.wisprflow.ai/peteryang] 📌 Get the takeaways: https://creatoreconomy.so/p/how-to-build-better-ai-evals-with-claude-code-shreya-hamel [https://creatoreconomy.so/p/how-to-build-better-ai-evals-with-claude-code-shreya-hamel] Where to find Shreya and Hamel: Shreya: https://www.sh-reya.com/ [https://www.sh-reya.com/] Hamel: https://hamel.dev/ [https://hamel.dev/] Subscribe to this channel - more interviews coming soon!