Qwen1.5-MoE
PulseAugur coverage of Qwen1.5-MoE — every cluster mentioning Qwen1.5-MoE across labs, papers, and developer communities, ranked by signal.
1 day(s) with sentiment data
-
Lightweight fine-tuning prunes MoE models, reducing size and latency
Researchers have developed a method to prune experts in Mixture-of-Experts (MoE) models using lightweight fine-tuning techniques. By applying parameter-efficient adapters like LoRA, they can identify and remove less cri…
-
New EPnG framework enhances MoE model fine-tuning efficiency
Researchers have developed EPnG, a novel framework for parameter-efficient fine-tuning of Mixture-of-Experts (MoE) models. This method adaptively reallocates fine-tuning capacity by pruning under-utilized experts and gr…
-
JetBrains releases efficient Mellum2 MoE model; research advances MoE techniques
JetBrains has released Mellum2, an open-source 12-billion parameter Mixture-of-Experts (MoE) model optimized for efficient inference in text and code tasks. This model activates only a fraction of its parameters per tok…
-
New methods enhance LLM quantization for efficiency and accuracy
Researchers have developed several new methods to improve the efficiency and accuracy of quantizing large language models (LLMs). These techniques aim to reduce the memory footprint and computational cost of LLMs, makin…