ModernBERT
PulseAugur coverage of ModernBERT — every cluster mentioning ModernBERT across labs, papers, and developer communities, ranked by signal.
10 day(s) with sentiment data
-
AI training efficiency: Gradient optimization methods benchmarked
A new research paper benchmarks five gradient optimizers and three memory strategies for AI training on constrained hardware. The study found that gradient accumulation is the most effective strategy for reducing traini…
-
Small language models streamline daily symptom tracking via conversational AI
Researchers have developed a novel method called "Scale-to-Dialogue" that uses small language models to efficiently collect daily premenstrual symptom ratings. This approach frames conversational administration as an or…
-
New model CheMatE unifies chemical structures and natural language
Researchers have developed CheMatE, a new embedding model designed to jointly represent chemical structures (SMILES) and natural language within a unified space. Built on a ModernBERT backbone, CheMatE employs a two-sta…
-
LLMs eager to facilitate online discussions, humans more cautious
A new study published on arXiv explores the tendencies of humans and Large Language Models (LLMs) in facilitating online discussions. Researchers created the PEFK corpus to standardize facilitation datasets and conducte…
-
AI system AWARE-FX quantifies FX hedging disclosures in corporate reports
Researchers have developed AWARE-FX, an AI system designed to analyze corporate annual reports and quantify foreign-exchange hedging disclosures. This system integrates a specialized lexicon, logic for negation and acco…
-
Browser extension uses single model for clickbait, leaning, and sentiment analysis
The author describes a browser extension called 'UnBlur' that analyzes news articles for clickbait, political leaning, and sentiment. Instead of using three separate models, the extension employs a single shared backbon…
-
LiquidAI releases LFM2.5 multilingual bidirectional encoders
LiquidAI has released two new multilingual bidirectional encoders, LFM2.5-Encoder-230M and LFM2.5-Encoder-350M, built on the LFM2 architecture. These models are designed for on-device efficiency and fine-tuning for vari…
-
Older BERT model outperforms newer successors in sparse retrieval tasks
Recent research indicates that older BERT models, specifically BERT-base, outperform newer encoders like ModernBERT in sparse retrieval tasks. This phenomenon is attributed to a "Vocabulary Gap," where the larger, case-…
-
Symbolic Augmentation boosts neural fact-checker robustness on scientific text
Researchers have developed a new method called Symbolic Augmentation to improve the accuracy of neural fact-checkers, particularly in handling scientific text. These models often struggle with numbers and units, leading…
-
LLMs unified for multimodal clinical prediction, matching specialized models
Researchers have developed a novel method for clinical prediction by converting all patient data, including text and structured measurements, into a single natural language sequence. This approach allows for the fine-tu…
-
AI analyzes public sentiment on Advanced Air Mobility
A new study published on arXiv analyzes public sentiment regarding Advanced Air Mobility (AAM) by examining over 300,000 texts from Reddit and Quora. Researchers evaluated seven AI sentiment analysis approaches, finding…
-
Regolo.ai's Brick LLM router optimizes costs by selecting the best model
Regolo.ai has developed Brick, an LLM routing system designed to optimize costs by intelligently selecting the most appropriate model for a given prompt. Unlike traditional cascade systems that involve retries, Brick an…
-
Prompting language models surpasses fine-tuning for legal term retrieval
A new research paper published on arXiv demonstrates that zero-shot prompting of decoder-only language models outperforms supervised fine-tuning methods for statutory term retrieval. The study compared two approaches fo…
-
New framework bridges vocabulary gap to boost AI sparse retrieval performance
Researchers have identified a "vocabulary gap" as the reason why advanced foundation models like ModernBERT underperform older models in learned sparse retrieval tasks. This gap arises because modern tokenizers use raw,…
-
ModernBERT models adapted for legal domain show significant performance gains
Researchers have investigated domain adaptation for modern BERT models within the legal sector. By further pre-training ModernBERT on a large corpus of US court opinions using masked language modeling, they achieved sig…
-
New BERTomelo model enhances Portuguese NLP tasks
Researchers have developed BERTomelo, a new monolingual encoder model specifically designed for the Portuguese language. This model utilizes the ModernBERT architecture and incorporates optimizations like FlashAttention…
-
Encoder classifiers offer cost-effective LLM safety evaluation, study finds
A new research paper explores the effectiveness of encoder classifiers, specifically from the ModernBERT family, as a cost-efficient alternative to LLM-based judges for evaluating the safety of large language model outp…
-
moBERTo: New Portuguese Language Model Enhances NLP Tasks
Researchers have introduced moBERTo, a new Portuguese language model derived from ModernBERT through continued pretraining. This model was trained on 60 billion tokens, incorporating data from FineWeb2 and filtered STEM…
-
New technique improves SPLADE retrieval models with larger encoders
Researchers have identified a performance degradation issue when using larger, more powerful pretrained encoders with SPLADE, a neural sparse retrieval model. This problem, termed a "scale mismatch" in the MLM head, can…
-
New LOCUS corpus unlocks U.S. local ordinances for AI research · 2 sources tracked
Researchers have developed LOCUS, a comprehensive corpus of U.S. local ordinances, aiming to make this critical layer of American law accessible for large-scale research and AI applications. The corpus includes codes fr…