You built a RAG chatbot. It answers questions from your documents. You test it a few times. The answers look good. So… can you ship it? No. One good answer doesn't tell you whether your RAG system works. A RAG application has multiple moving parts. The retriever can fail. The generator can fail. They can both work individually and still fail when combined. And once the application goes live, your users will ask questions you never tested. So how do you actually evaluate a RAG system?
