BERTScore: Evaluating text generation with BERT
PulseAugur coverage of BERTScore: Evaluating text generation with BERT — every cluster mentioning BERTScore: Evaluating text generation with BERT across labs, papers, and developer communities, ranked by signal.
6 day(s) with sentiment data
-
AI metrics fail romanized Indic languages, new study finds
A new research paper highlights a significant gap in evaluating AI models for romanized Indic languages, where automatic metrics fail to accurately assess generation quality. The study found that metrics like BGE-M3 and…
-
New LLM decoding technique ELIDE struggles with semantic preservation
Researchers have explored methods to reduce the verbosity of large language models without sacrificing semantic content. Initial attempts using an entropy-gated logits processor proved ineffective, merely truncating the…
-
New decoder unifies AI assistant's gating and hint generation
Researchers have developed a novel compact decoder architecture that integrates both gating and hint generation for proactive assistants. This single-model approach aims to improve efficiency and performance by deciding…
-
New benchmark ETC-Bench tackles multilingual transcript canonicalization
Researchers have introduced ETC-Bench, a novel multilingual benchmark designed to evaluate evaluation-side transcript canonicalization. This benchmark addresses the challenge of differing written transcripts for the sam…
-
LLM assistant improves HPC user support by querying documentation and tickets
Researchers have developed an agentic LLM assistant designed to improve user support for high-performance computing (HPC) environments. This assistant leverages retrieval-augmented generation (RAG) on live documentation…
-
Rowhammer attack amplifies LLM inference latency and cost
Researchers have developed LLVMhammer, a novel Rowhammer attack that targets the latency and cost of Large Language Model (LLM) inference on GPUs. By inducing a bit-flip in the CPU's DRAM, the attack causes the Triton J…
-
New multilingual dataset enhances AI crisis warning generation
Researchers have developed CrisiLangs, a new multilingual dataset designed to improve crisis warning generation by language models. This resource extends the existing CrisiText dataset to include warning messages for cr…
-
New MeddeX system uses LLaMA-3 and Gemini Vision for medical report analysis
Researchers have developed MeddeX, a system designed to analyze and summarize complex medical reports. MeddeX integrates a fine-tuned LLaMA-3 model with retrieval-augmented generation and a Gemini Vision layer to provid…
-
New BRUGEscore metric proposed for evaluating financial report summarization
Researchers have conducted a comparative study on various summarization methods using pre-trained transformers for long-document financial reports. The study focuses on annual reports from companies listed on the London…
-
Small Language Models Framework for SUD Patient Dialogue Generation
Researchers have developed a framework using Small Language Models (SLMs) to generate dialogue for patients with substance use disorder (SUD). This approach addresses the limitations of larger models in clinical setting…
-
AI legal assistants developed for Nepal to improve access to justice · 2 sources tracked
Two research papers introduce AI-powered legal assistants for Nepal, aiming to improve access to justice. The first, NepKANUN, utilizes a retrieval-augmented generation (RAG) framework with a fine-tuned large language m…
-
LoRA fine-tuning enhances Qwen2.5 models for control systems Q&A
Researchers have evaluated the effectiveness of LoRA fine-tuning on Qwen2.5 models for answering questions in a Linear Control Systems course. The study found that LoRA improved both textual similarity to reference answ…
-
LexFlip diagnostic tool evaluates legal meaning preservation in simplified text
Researchers have developed LexFlip, a new diagnostic tool designed to evaluate how well legal meaning is preserved when text is simplified. Current metrics often fail to distinguish between lexical overlap and actual le…
-
New PetQA benchmark evaluates AI veterinary knowledge
Researchers have developed PetQA, a new benchmark designed to evaluate the veterinary knowledge and clinical reasoning capabilities of large language models (LLMs) and large vision-language models (LVLMs). The benchmark…
-
LLMs show promise in extracting software design decisions from code commits
A preliminary study explored the ability of four Large Language Models (LLMs) to extract Architectural Design Decisions (ADDs) from source code commits. Models including Gemini 3-Pro, DeepSeek-R1, Kimi K2, and Qwen3 wer…
-
New corpus StageWell enhances AI support dialogues
Researchers have introduced StageWell, a new Chinese corpus designed for positive psychology support dialogues. This corpus, developed using the HQS protocol, organizes support into a six-stage process and includes 12,4…
-
PermitGPT uses generative AI for construction governance and safety
Researchers have developed PermitGPT, a generative AI framework designed to streamline urban construction governance. This system unifies scattered data from municipal and regulatory sources to identify safety hazards, …
-
SimpCue prompts boost multilingual text simplification modestly
Researchers have developed SimpCue, a novel prompting technique for multilingual text simplification using the Qwen3_8B language model. This method involves enriching prompts with explicit linguistic cues about sentence…
-
LLMs Compared for ASR Evaluation: Encoder vs. Generative Models
A new study published on arXiv explores the effectiveness of encoder and decoder-based Large Language Models (LLMs) for evaluating Automatic Speech Recognition (ASR) systems. The research compares metrics like BERTScore…
-
Gaze-supervised AI enhances chest X-ray diagnosis and report generation
Researchers have developed a novel two-stage multimodal framework for interpreting chest X-rays, integrating radiologist eye-tracking data to improve diagnostic accuracy and report generation. The first stage employs a …