Mount Holyoke College
PulseAugur coverage of Mount Holyoke College — every cluster mentioning Mount Holyoke College across labs, papers, and developer communities, ranked by signal.
-
New PEFT method mHC enhances Transformer finetuning when combined with LoRA
Researchers have introduced Manifold-Constrained Hyper-Connections (mHC), a novel parameter-efficient finetuning (PEFT) method for Transformer models. This approach modifies residual connections, a component typically l…
-
xHC method expands transformer streams beyond N=4 for improved LLM pre-training
Researchers have introduced xHC (Expanded Hyper-Connections), a novel method for scaling transformer models beyond the typical limit of N=4 streams. This new approach addresses bottlenecks in previous Hyper-Connections …
-
AI Techniques Explored: Memory, Inference, Fine-Tuning, and Tokens
A blog post synthesizes current and emerging AI techniques, focusing on memory, inference, fine-tuning, and tokenization. The article highlights advancements such as Manifold-Constrained Hyper-Connections (mHC) alongsid…
-
DeRes architecture improves CTR prediction with dual residual paths
Researchers have introduced DeRes, a novel architecture for Transformer-based CTR prediction models that decouples residual stability and adaptivity. This new design employs parallel identity and block attention residua…
-
DeepSeek V4 Introduces Manifold-Constrained Hyper-Connections
DeepSeek V4 is an advanced language model that builds upon its predecessor, DeepSeek V3. The V4 architecture introduces novel components such as Compressed Sparse Attention (CSA), Heavily Compressed Attention (HCA), and…
-
New mHC architecture in AI models alters attention head behavior
Researchers have investigated the impact of Manifold-Constrained Hyper-Connections (mHC), a novel architecture implemented in Deepseek v4, on model interpretability. Experiments revealed that previous token attention he…