Dynamic Structural Pruning

New ToMoE method converts dense LLMs to Mixture-of-Experts

Researchers have developed a method called ToMoE that converts dense large language models into Mixture-of-Experts (MoE) architectures. This technique uses differentiable dynamic pruning to reduce computational and memo…