How to Evaluate RAG Systems When Ground Truth Keeps Changing

AI Engineering Podcast
This show is your guidebook to building scalable and maintainable AI systems. You will learn how to architect AI applications, apply AI to your work, and the considerations involved in building or customizing new models. Everything that you need to know to deliver real impact and value with machine learning and artificial intelligence.
Share on social media:
Loading transcript...
Introduction to evaluating RAG in production
Ofer Mandelovich's path from ML to modern AI
Why RAG became the first practical LLM pattern
Production ingestion challenges at enterprise scale
Why RAG evaluation is so difficult
LLM as judge with Umbrella and AutoNuggetizer
Bootstrapping evals and offline versus online testing
Agentic RAG, tools, and changing retrieval design
Context engineering, permissions, and multimodal UX
Lessons learned and the future of retrieval and agents
Text to SQL, semantic layers, and AI education gaps
Closing thoughts and wrap up
Share on social media:
Listen in your favorite app:
Here are shows you might like