DLLG: Dynamic Logit-Level Gating of LLM Experts
Researchers have introduced DLLG, a novel framework for dynamically integrating multiple specialized Large Language Models (LLMs). This approach learns to fuse expert LLMs at the logit level on a token-by-token basis, using only sparse response-level supervision. DLLG consistently outperforms existing methods like routing, heuristic ensembling, and parameter merging across various benchmarks and model scales. AI
IMPACT Introduces a new method for combining specialized LLMs, potentially improving performance and adaptability in complex tasks.