ModernBERT
PulseAugur coverage of ModernBERT — every cluster mentioning ModernBERT across labs, papers, and developer communities, ranked by signal.
3 day(s) with sentiment data
-
Compact HTML-LM Model Achieves State-of-the-Art for Czech Web Documents
Researchers have developed HTML-LM, a compact foundation model with 154 million parameters designed to efficiently process Czech HTML documents. This model, built on a ModernBERT architecture, utilizes HTML-aware traini…
-
H Company releases NeoMME, a single-tower multimodal encoder family
H Company has introduced NeoMME, a new family of multimodal encoders designed for efficiency. These models, available in 260M and 800M parameter sizes, eliminate the need for separate vision towers and causal decoders, …
-
Hugging Face unveils efficient multimodal encoder NeoMME, study favors encoders for Indic NER
Hugging Face has introduced NeoMME, a new family of multilingual multimodal encoders designed for efficiency. Unlike many generative models, NeoMME uses a single bidirectional Transformer to process both text and image …
-
New research explores grammar's geometry in Transformer layers
A new research paper explores the geometric properties of language representations within Transformer models. The study investigates how the intrinsic dimensionality (ID) of these representations changes across layers a…
-
New GEO-Flag tool detects AI-optimized web content in search results
Researchers have developed a new method called GEO-Flag to detect and measure "Generative Engine Optimization" (GEO), a technique where web content is modified to rank higher in generative search engines. This optimizat…
-
New GEO-Flag system detects AI-optimized web content in search results · 2 sources tracked
Researchers have developed GEO-Flag, a system designed to detect and measure "Generative Engine Optimization" (GEO), a technique where web content is modified to increase its visibility in generative search engines. Thi…
-
AI training efficiency: Gradient optimization methods benchmarked
A new research paper benchmarks five gradient optimizers and three memory strategies for AI training on constrained hardware. The study found that gradient accumulation is the most effective strategy for reducing traini…
-
Small language models streamline daily symptom tracking via conversational AI
Researchers have developed a novel method called "Scale-to-Dialogue" that uses small language models to efficiently collect daily premenstrual symptom ratings. This approach frames conversational administration as an or…
-
New study benchmarks gradient optimizers for memory-efficient AI training
A new study benchmarks five gradient optimizers (SGD, Adam, Adagrad, Adadelta, and Conjugate Gradient Descent) against three memory-saving strategies (standard training, gradient checkpointing, and gradient accumulation…
-
New model CheMatE unifies chemical structures and natural language
Researchers have developed CheMatE, a new embedding model designed to jointly represent chemical structures (SMILES) and natural language within a unified space. Built on a ModernBERT backbone, CheMatE employs a two-sta…
-
LLMs eager to facilitate online discussions, humans more cautious
A new study published on arXiv explores the tendencies of humans and Large Language Models (LLMs) in facilitating online discussions. Researchers created the PEFK corpus to standardize facilitation datasets and conducte…
-
AI system AWARE-FX quantifies FX hedging disclosures in corporate reports
Researchers have developed AWARE-FX, an AI system designed to analyze corporate annual reports and quantify foreign-exchange hedging disclosures. This system integrates a specialized lexicon, logic for negation and acco…
-
Browser extension uses single model for clickbait, leaning, and sentiment analysis
The author describes a browser extension called 'UnBlur' that analyzes news articles for clickbait, political leaning, and sentiment. Instead of using three separate models, the extension employs a single shared backbon…
-
LiquidAI releases LFM2.5 multilingual bidirectional encoders
LiquidAI has released two new multilingual bidirectional encoders, LFM2.5-Encoder-230M and LFM2.5-Encoder-350M, built on the LFM2 architecture. These models are designed for on-device efficiency and fine-tuning for vari…
-
Older BERT model outperforms newer successors in sparse retrieval tasks
Recent research indicates that older BERT models, specifically BERT-base, outperform newer encoders like ModernBERT in sparse retrieval tasks. This phenomenon is attributed to a "Vocabulary Gap," where the larger, case-…
-
Symbolic Augmentation boosts neural fact-checker robustness on scientific text
Researchers have developed a new method called Symbolic Augmentation to improve the accuracy of neural fact-checkers, particularly in handling scientific text. These models often struggle with numbers and units, leading…
-
LLMs unified for multimodal clinical prediction, matching specialized models
Researchers have developed a novel method for clinical prediction by converting all patient data, including text and structured measurements, into a single natural language sequence. This approach allows for the fine-tu…
-
AI analyzes public sentiment on Advanced Air Mobility
A new study published on arXiv analyzes public sentiment regarding Advanced Air Mobility (AAM) by examining over 300,000 texts from Reddit and Quora. Researchers evaluated seven AI sentiment analysis approaches, finding…
-
Regolo.ai's Brick LLM router optimizes costs by selecting the best model
Regolo.ai has developed Brick, an LLM routing system designed to optimize costs by intelligently selecting the most appropriate model for a given prompt. Unlike traditional cascade systems that involve retries, Brick an…
-
Prompting language models surpasses fine-tuning for legal term retrieval
A new research paper published on arXiv demonstrates that zero-shot prompting of decoder-only language models outperforms supervised fine-tuning methods for statutory term retrieval. The study compared two approaches fo…