IEEE/ACM CAIN 2024

LLM evaluation must be driven by real-world failures, not pre-written rubrics

Building effective evaluations for retrieval-augmented generation (RAG) systems requires a failure-driven approach rather than pre-defined rubrics. The process involves collecting and grouping actual failures observed d…