arXiv 2610.29722
Retrieval Augmented Generation Compositionality
AI26 upvotes
Authors: Ido Levy, Segev Shlomov, Leshem Choshen
Overview
The authors built LASAGNA, an open-source framework for controlled RAG evaluation, and tested 450 pipeline configurations across eight benchmarks and corpora of 5K to 200K documents. They examined how embedding, chunking, retrieval, reranking, query augmentation, context compression, and agentic verification behave when combined rather than tested in isolation.
Introduction
RAG techniques are usually evaluated one at a time, leaving unclear whether their isolated gains hold when stacked together. It is also unknown whether retrieval quality and answer faithfulness favor the same choices, or whether small-corpus conclusions transfer to large corpora.
Results
Three retrieval techniques predicted a +47% combined improvement but delivered only +17% when used together. Component gains also stack differently for retrieval and faithfulness, and the best composition changes as the corpus grows.
Conclusion
Gains measured in isolation are unreliable predictors of realistic performance, so new RAG techniques should be tested within the full pipeline, which LASAGNA is intended to simplify.