How to Evaluate a RAG System: Retrieval, Grounding and Product Quality

Build a known-answer set

Create representative queries with expected source documents or passages. Without expected evidence, retrieval quality is difficult to measure.

Measure whether the correct evidence appears in the candidate set and how highly it ranks.

Check whether claims are actually supported by retrieved evidence and whether citations point to the right sources.

Latency, token usage, vector-query time, failure rate and cost per answered query are part of production quality.