XLM-RoBERTa
PulseAugur coverage of XLM-RoBERTa — every cluster mentioning XLM-RoBERTa across labs, papers, and developer communities, ranked by signal.
3 day(s) with sentiment data
-
New benchmark PhoPunct evaluates Transformer architectures for Vietnamese punctuation restoration
Researchers have developed PhoPunct, a large-scale benchmark for evaluating Vietnamese punctuation restoration models. The benchmark, comprising 22.1 million labeled tokens from News and Novels corpora, was used to test…
-
Sarcasm hinders Turkish hate speech detection in LLMs and fine-tuned models
A new research paper introduces SarcasTraps, a dataset designed to evaluate how sarcasm impacts the detection of hate speech in Turkish. The study found that sarcasm significantly challenges both fine-tuned models like …
-
LLMs tackle knowledge conflicts, diverse perspectives, and low-resource languages · 7 sources tracked
Recent research explores how large language models handle conflicting information and diverse perspectives. One paper introduces a method for resolving disagreements between a model's internal knowledge and retrieved co…
-
RAG pipeline issue: BGE-Reranker truncates crucial data at 512 tokens
A developer encountered issues with a Retrieval-Augmented Generation (RAG) pipeline where a cross-encoder reranker, specifically BAAI/bge-reranker-base, was truncating relevant information. The reranker, designed to pro…
-
Language models compared for propaganda detection · research paper
A new research paper compares the effectiveness of masked language models (MLMs) and causal language models in detecting propaganda techniques. Using the SemEval-2020 Task 11 dataset, the study evaluated models like XLM…
-
LLMs analyze Russian social media for spontaneous value expression
Researchers have developed a novel framework to analyze human values expressed spontaneously on Russian social media, contrasting with traditional survey methods. This framework utilizes LLMs and an XLM-RoBERTa encoder …
-
New multilingual privacy tagger outperforms GLiNER2, Presidio, and OpenAI
Researchers have developed a new multilingual named-entity tagger designed for privacy redaction, outperforming existing tools like GLiNER2, Microsoft Presidio, and OpenAI Privacy Filter. This tagger, fine-tuned on a mu…
-
Domain-specific pretraining boosts Transformer performance on Arabic-English code-switching
A new study published on arXiv explores the impact of domain-specific pretraining on Transformer models for analyzing Arabic-English code-switching. The research evaluated MARBERT and XLM-RoBERTa, with BERT as a baselin…
-
AI framework incorporates annotator psychology for sexism detection
Researchers from VANGUARD have developed a multimodal framework for detecting sexism online, incorporating annotator psychology and demographics into the detection process. Their approach fuses five input modalities usi…
-
New IndicTriMix method improves language identification in code-mixed text
Researchers have developed a new method called IndicTriMix for identifying languages within code-mixed text, which is common in social media. This approach treats language identification as a sequence labeling problem a…
-
Transformers Mimic Traditional Models in Multilingual Readability Assessment
Researchers have analyzed how Transformer-based models and traditional feature-based models approach multilingual readability assessment. They found that while Transformers achieve high accuracy, their internal feature …
-
Fine-tuning LLMs: Four crucial steps before you start
The article advises against immediately fine-tuning large language models like GPT-3, Bert, T5, Roberta, and XLM-RoBERTa. It suggests performing four crucial steps before proceeding with fine-tuning to ensure better and…
-
Hugging Face unveils efficient multimodal encoder NeoMME, study favors encoders for Indic NER
Hugging Face has introduced NeoMME, a new family of multilingual multimodal encoders designed for efficiency. Unlike many generative models, NeoMME uses a single bidirectional Transformer to process both text and image …
-
New TabuLM model enhances low-resource language processing with tabular data
Researchers have developed TabuLM, a novel language model specifically pre-trained on Kinyarwanda tabular data to address the scarcity of resources for low-resource languages. This model extends KinyaBERT-large by incor…
-
Prompt compression struggles with non-English languages, study finds
A new study published on arXiv titled "Lost in Compression: A Controlled Cross-Lingual Audit of Extractive Prompt Compressors" investigates the effectiveness of prompt compression techniques across different languages. …
-
TabuLM: First Language Model Pre-trained on Kinyarwanda Tabular Data
Researchers have developed TabuLM, a novel language model specifically pre-trained on tabular data for Kinyarwanda, a low-resource Bantu language spoken in Rwanda. This model enhances KinyaBERT-large with new embeddings…
-
Hugging Face Model Selection Framework Launched Amidst 2M Model Milestone
This article provides a framework for selecting and fine-tuning models from Hugging Face, a platform that now hosts over two million models. It guides users through the decision-making process, offering insights beyond …
-
Sparse few-shot language model for Bengali achieves 90% sparsity
Researchers have developed BnBERT-iPET, a novel approach to sparse few-shot language modeling specifically for Bengali. This method utilizes lottery ticket pruning to achieve 90% sparsity, significantly reducing computa…
-
OpenAI's Privacy Filter shows mixed results in PII detection study
A new research paper introduces the first systematic evaluation of OpenAI's Privacy Filter (OPF), a 1.5 billion parameter PII detector. The study found that OPF performs well on structured PII types like emails and phon…
-
Kazakh-Russian code-switching identification bottleneck is annotation, not model
A new paper on arXiv explores the identification of code-switching between Kazakh and Russian languages, finding that the annotation boundary is more critical than the model used. Researchers developed a gold LID (Langu…