Publications

Efficient training-free query routing framework for federated retrieval-augmented generation

Scientific Reports, Article in Press, 2026.

Hosun Eoo, Yeonho Yoo, Gyeongsik Yang

Hosun Eoo, Yeonho Yoo, and Gyeongsik Yang. 2026. Efficient training-free query routing framework for federated retrieval-augmented generation. Scientific Reports. Article in Press.

Download PDF

Abstract

Retrieval-augmented generation (RAG) enhances large language models (LLMs) by grounding responses in external documents retrieved at inference time. In federated RAG (FedRAG), documents are distributed across multiple autonomous corpora that cannot be centrally aggregated due to privacy, regulatory, or organizational constraints. A key challenge in FedRAG is query routing: selecting a subset of relevant corpora for each query without broadcasting to all participants. Existing routing approaches rely on supervised learning, requiring a uniform retrieval embedding model across corpora and costly retraining whenever the federation changes. These assumptions violate corpus autonomy and limit scalability in dynamic federated environments. We propose RAGSpot , a training-free query routing framework that enables efficient corpus selection without requiring a shared local retrieval embedding model. RAGSpot represents each corpus using compact cluster-level metadata derived from its vector index and estimates corpus relevance via query-to-cluster matching. In addition, it introduces a rank-based fusion strategy that jointly leverages document-level and corpus-level rankings to support robust aggregation across heterogeneous embedding models. Experiments with corpora from the BEIR benchmark show that RAGSpot improves retrieval accuracy over the strongest training-based router by 1.23–1.48× in nDCG and 1.28–1.66× in Recall, and over the strongest LLM-based router by 1.63–2.01× in nDCG and 1.93–2.34× in Recall, without any further training. Also, RAGSpot achieves query latency that is comparable to, or even lower than, that of SOTA methods on average.

Paper summary

Find the right collections before retrieving documents

Independent document collections can answer a query together without retraining a router whenever a new collection joins.

Federated RAG retrieves evidence from independently managed document collections. Sending every query to every collection increases communication, while trained routers require updates as collections change. RAGSpot separates the routing decision from local retrieval so each collection can keep its own retrieval model.

Route through a shared collection map

Each collection uses a common routing encoder to group document vectors into clusters, then publishes their centers and radii. For a new query, the router compares its distance to nearby cluster centers relative to each cluster’s radius. This selects collections without requiring their local retrieval encoders to match.

The shared representation decides where to look; each collection decides how to search its own documents.

Selected collections return their top documents. RAGSpot combines each document’s local rank with its collection’s relevance rank. A new collection joins by registering its routing metadata, without retraining the router.

Cluster metadata guides corpus selection, then rank-based fusion combines locally retrieved documents.View full-size figure
Cluster metadata guides corpus selection, then rank-based fusion combines locally retrieved documents.

What the retrieval experiments show

The authors test six BEIR collections using 3,726 single-corpus queries and 3,464 synthetic two-corpus queries formed by concatenation. The table compares RAGSpot with the trained RAGRoute baseline. nDCG measures ranking quality; Recall measures coverage of relevant documents. Both are reported at ten retrieved results.

RAGSpot improves both ranking and coverage in these comparisons without router training. The composite workload uses concatenated queries to test routing to two known sources.

Retrieval quality on the six-corpus benchmark
Workload / methodnDCG@10Recall@10
Single / RAGRoute0.280.39
Single / RAGSpot0.350.51
Composite / RAGRoute0.110.10
Composite / RAGSpot0.160.16

Key takeaways

  1. Federated routing need not standardize local retrievers. Shared cluster metadata provides a common routing view while collections retain their own retrieval models.
  2. Source selection and evidence ranking operate at different levels. Combining corpus and document ranks connects collection relevance to individual results.
  3. A new collection can join through metadata. This avoids retraining a supervised router whenever the federation changes.