Glue
PulseAugur coverage of Glue — every cluster mentioning Glue across labs, papers, and developer communities, ranked by signal.
5 day(s) with sentiment data
-
Temperon training method achieves SAM quality with reduced cost
Researchers have introduced Temperon, a novel training method designed to achieve the quality of Sharpness-Aware Minimization (SAM) while significantly reducing computational costs. Temperon utilizes a two-phase approac…
-
French-only BabyLM model reveals tokenizer sensitivity
Researchers have developed MéTRON-FR, a 125M parameter GPT-2 model trained exclusively on French text, achieving notable scores on French-specific benchmarks. When evaluated using a cross-lingual GLUE protocol, the mode…
-
New study benchmarks privacy risks in NLP text classifiers
A new study on arXiv evaluates the privacy risks associated with training natural language processing (NLP) text classifiers. Researchers benchmarked membership inference attacks (MIAs) on the GLUE SST-2 sentiment datas…
-
New RAPTOR framework enhances private training for MoE AI models
Researchers have developed RAPTOR, a novel framework for differentially private training of Mixture-of-Experts (MoE) models. Existing methods treat these sparse models as dense blocks, leading to issues like gradient su…
-
Looped GPT-BERT model trades parameters for computation in language modeling
Researchers have developed a novel language model called Looped GPT-BERT, which achieves comparable performance to existing models on linguistic and downstream tasks while using fewer parameters. This is accomplished by…
-
New methods enhance LoRA efficiency and stability for model adaptation
Researchers have developed two new methods to improve the efficiency and stability of Low-Rank Adaptation (LoRA) techniques used in parameter-efficient model adaptation. Normalized Low-Rank Adaptation (NoRA) normalizes …
-
New LoRA-GA^2 Algorithm Enhances Large Model Fine-Tuning
Researchers have introduced LoRA-GA^2, a novel fine-tuning algorithm designed to improve upon existing Low-Rank Adaptation (LoRA) methods for large models. This new approach utilizes multi-step gradient information, whi…
-
New FedPA-LoRA framework improves federated LLM fine-tuning
Researchers have introduced FedPA-LoRA, a novel framework designed to improve the efficiency and accuracy of federated fine-tuning for large language models. This new approach addresses challenges in aggregating local m…
-
New FedPA-LoRA framework improves federated LLM fine-tuning
Researchers have developed FedPA-LoRA, a new framework designed to improve the efficiency and accuracy of federated fine-tuning for large language models. This approach addresses the challenges of aggregating updates an…
-
New SeFoRA algorithm enhances federated fine-tuning of large neural networks
Researchers have introduced SeFoRA, a novel algorithm designed for federated parameter-efficient fine-tuning of large neural networks using low-rank adaptation (LoRA). SeFoRA addresses challenges arising from clients us…
-
UniF-MoE framework unifies adaptive MoE computation for improved efficiency
Researchers have introduced UniF-MoE, a novel framework for Mixture-of-Experts (MoE) computation that unifies various adaptive strategies. This approach decomposes experts into blocks, allowing for shared computation fi…
-
AWS AI/ML Services Beyond EC2 and S3 for Cloud Engineers
This article highlights 15 AWS AI and machine learning services that cloud and DevOps engineers should be aware of, moving beyond foundational services like EC2 and S3. It covers a range of offerings designed for variou…
-
New SiPE method enhances Transformers with syntactic structure
Researchers have developed Syntax-Informed Positional Embeddings (SiPE), a novel method to enhance Transformer models by incorporating syntactic structure. SiPE learns a syntactic prior from dependency parses during pre…
-
New research enhances Transformer positional encoding for better language understanding
Two new research papers explore advancements in positional encoding for Transformer models, aiming to improve their understanding of token order and syntactic structure. The first paper provides a comprehensive survey o…
-
New M-TTFS encoding boosts SNN energy efficiency for LLMs
Researchers have developed a new encoding method called Masked Time-to-First-Spike (M-TTFS) for spiking neural networks (SNNs) to improve energy efficiency in large language models. The M-TTFS encoding reassigns the sil…
-
Federated pre-training evaluation methods compared in new research
A new research paper explores the challenges of evaluating federated pre-training, a method for training models on distributed data without centralization. The study highlights that downstream fine-tuning on benchmarks …
-
Google Cloud's borderless Lakehouse unifies data across AWS, Databricks, Snowflake
Google Cloud has introduced a new "borderless Lakehouse" initiative aimed at eliminating the need to copy data between different cloud platforms. The service, currently in preview, allows data pipelines and agents to di…
-
New cMoLLM architecture scales LLMs via dynamic convolutions
Researchers have introduced cMoLLM, a novel approach to scaling large language models by incorporating a mixture-of-experts (MoE) style throughout the entire model pipeline, rather than just in the feed-forward networks…
-
New research explores optimized LoRA fine-tuning methods for LLMs · 4 sources tracked
Researchers are exploring new methods to optimize Low-Rank Adaptation (LoRA) for fine-tuning large language models. One approach, Unified LoRA (ULoRA), introduces a continuum of preconditioned gradient initializations t…
-
New research explores adaptive rank allocation for efficient LLM fine-tuning
Two new research papers introduce advanced methods for parameter-efficient fine-tuning (PEFT) of large language models. The first paper proposes LAARA, a framework that dynamically allocates adapter ranks to different t…