scripod.com

Ep 852: Measuring AI ROI: Why you’re doing it wrong and the 7 Steps to fix it (Start Here Series Vol 11)

Shownote

Why can't most companies show their ROI on GenAI?  Because their implementation is backwards. If you're using the same digital transformation playbook that you used for the social media and cloud eras, you're in trouble.  On this 'Start Here Series' episode, we break down what your company is doing wrong and the 7 Step process to properly calculate ROI on your AI efforts.  Measuring AI ROI: Why you’re doing it wrong and the 7 Steps to fix it -- An Everyday AI Chat with Jordan Wilson Newsletter: Sign up for our free daily newsletter [https://youreverydayai.com/] More on this Episode: Episode Page [https://www.youreverydayai.com/ep-725-measuring-ai-roi-why-youre-doing-it-wrong-and-the-7-steps-to-fix-it-start-here-series-vol-11/] Join the discussion on LinkedIn: Thoughts on this? Join the convo on LinkedIn and connect with other AI leaders. [https://www.linkedin.com/posts/jordanwilson04_why-cant-most-companies-show-their-roi-on-activity-7434472648321142784-R9iP?utm_source=share&utm_medium=member_desktop&rcm=ACoAAEnbkdQBR48B1Rc3uoMWlWfVMPCScgZCUCQ] Upcoming Episodes: Check out the upcoming Everyday AI Livestream lineup [https://www.youtube.com/@EverydayAI_/streams] Website: YourEverydayAI.com [https://youreverydayai.com/?utm_source=podcast&utm_medium=podcast&utm_campaign=shownotes] Email The Show: info@youreverydayai.com [info@youreverydayai.com] Connect with Jordan on LinkedIn [https://www.linkedin.com/in/jordanwilson04/] Topics Covered in This Episode: 1. Proving and Measuring AI ROI in Companies 2. OpenAI’s GDP Benchmark: AI vs Experts 3. Debunking MIT’s Viral Zero ROI Study 4. Quantitative AI ROI Data from Business Studies 5. Invisible Productivity: AI Savings Pocketed by Workers 6. Limitations of Pre-AI Job Roles and Metrics 7. ROI Calculation: Time Saved and Cost Reduction 8. Five Main Reasons AI ROI Isn’t Measured 9. Seven-Step AI ROI Measurement Blueprint 10. Importance of Ongoing AI Model Retesting 11. AI ROI: Training, Education, and Implementation Gaps Timestamps: 00:00 "AI ROI: Fixing the Debate" 04:50 "AI ROI Debate is Pointless" 08:03 AI Implementation Yields Positive ROI 12:13 AI Reshaping Work Structures 15:08 AI ROI Challenges 20:11 "Baseline Assessment Before AI Implementation" 23:49 "AI Operating System Discussion" 24:35 "Standardized Testing for AI Models" 28:26 "Rethink AI ROI Urgency" 31:37 "Everyday AI: Subscribe & Explore" Keywords:  measuring AI ROI, AI return on investment, generative AI ROI, GenAI ROI measurement, ROI on artificial intelligence, AI productivity, time saved with AI, cost reduction with AI, AI-driven revenue increase, AI risk avoidance, AI implementation, baseline assessment of AI, BASE framework, pre-AI baseline, digital transformation and AI, quality metrics in AI, cost per task AI, error rates AI, throughput AI, AI utilization rate, AI pilot evaluation, AI business impact, AI model benchmarks,  Send Everyday AI and Jordan a text message. (We can't reply back unless you leave contact info) [https://www.buzzsprout.com/2175779/fan_mail/new]

Highlights

This episode examines why organizations often struggle to demonstrate the business impact of generative AI. It argues that the central problem is not necessarily weak technology, but outdated evaluation methods and poorly designed implementation strategies.
00:00
AI ROI fails when companies measure the wrong things.
05:00
AI can outperform experts while cutting task time
11:06
AI gains matter only when outcomes prove them.
12:13
AI ROI requires redesigning work, not just adding tools.
15:08
Measure AI by outcomes, not tool usage.
20:14
Measure AI against real-world performance
23:53
Repeatable testing requires proof, not assumptions
24:36
Measure AI ROI with disciplined retesting.
28:27
Unmeasured AI value is value organizations continue to lose.
31:41
Delayed AI education leads to lost value

Chapters

"AI ROI: Fixing the Debate"
00:00
"AI ROI Debate is Pointless"
04:50
AI Implementation Yields Positive ROI
08:03
AI Reshaping Work Structures
12:13
AI ROI Challenges
15:08
"Baseline Assessment Before AI Implementation"
20:11
"AI Operating System Discussion"
23:49
"Standardized Testing for AI Models"
24:35
"Rethink AI ROI Urgency"
28:26
"Everyday AI: Subscribe & Explore"
31:37

Transcript

Jordan Wilson: Welcome to the Everyday AI Podcast. My name is Jordan Wilson and for the past three and a half years, we've put out more than 800 episodes. Yet, one of the most common questions I get, I didn't really have an answer for. Where do I start on ...