Indic Languages
PulseAugur coverage of Indic Languages — every cluster mentioning Indic Languages across labs, papers, and developer communities, ranked by signal.
5 day(s) with sentiment data
-
AI metrics fail romanized Indic languages, new study finds
A new research paper highlights a significant gap in evaluating AI models for romanized Indic languages, where automatic metrics fail to accurately assess generation quality. The study found that metrics like BGE-M3 and…
-
Open-source EOU detector released for English and 8 Indic languages
A new open-source End-of-Turn (EOU) detection model has been released, designed to prevent voice agents from cutting callers off mid-sentence. This model is particularly beneficial for languages like those in India, whe…
-
AI news roundup: llama.cpp update, Indic language tokenization, and Krea 2 styles
A new release of llama.cpp (b11935) includes support for KleidiAI and OpenEuler, with various build configurations for different operating systems. Separately, a research paper introduces AdaptTok-Indic, a specialized t…
-
LLMs show evaluation awareness in Indic languages, impacting safety tests
A new study published on arXiv investigates how large language models (LLMs) behave when evaluated in Indic languages compared to English. The research found that LLMs exhibit "evaluation awareness," meaning they alter …
-
Hugging Face & Nvidia Scale Indic Language AI with 2.8T Tokens
Hugging Face and Nvidia have collaborated on a project to scale pretraining for Indic languages, processing an immense 2.8 trillion tokens. This initiative includes the development of an open data generation pipeline to…
-
New Tamil-first LLM, Agaram 2.5B, released with tool-calling capabilities
Researchers have developed Agaram 2.5B, a new language model specifically designed for Tamil, trained from scratch on a dataset with a majority of Tamil text. This model demonstrates improved translation capabilities co…
-
New research quantifies multilingual tokenization tax, finding it largely removable
A new research paper proposes a "token-cost ledger" to quantify the extra cost associated with processing non-English text in large language models. The study, which analyzes eight languages on the FLORES-200 dataset, f…
-
New benchmarks reveal LLM struggles with Indic languages and translation mechanics · 3 sources tracked
Researchers have developed VakyArth, a new benchmark designed to evaluate the pragmatic competence of large language models (LLMs) specifically within Indic languages like Hindi, Punjabi, Tamil, and Malayalam. Initial f…
-
Hugging Face unveils efficient multimodal encoder NeoMME, study favors encoders for Indic NER
Hugging Face has introduced NeoMME, a new family of multilingual multimodal encoders designed for efficiency. Unlike many generative models, NeoMME uses a single bidirectional Transformer to process both text and image …
-
UniLipi OCR model unifies 13 Indic scripts for historical manuscripts
Researchers have developed UniLipi, a novel unified multi-script Optical Character Recognition (OCR) model designed for historical Indic manuscripts. This single framework can process 13 different Indic scripts, address…
-
New research explores advanced tokenization for LLMs, improving efficiency and performance · 4 sources tracked
Researchers are developing new methods for tokenizing text in large language models to improve efficiency and performance. One approach, SuTRA, focuses on morphological structure for morphologically rich languages like …
-
DonorRank framework improves donor language selection for low-resource speech recognition
Researchers have developed DonorRank, a novel learning-to-rank framework designed to improve the selection of effective donor languages for low-resource cross-lingual speech recognition. This method addresses the challe…
-
Researchers pinpoint cross-lingual refusal circuit in multilingual MoE model
Researchers have identified a specific circuit within a multilingual Mixture-of-Experts (MoE) model, named sarvam, that is responsible for refusing harmful requests. This circuit's ability to refuse is language-invarian…
-
New IndicTalk Corpus Boosts Multilingual Conversational AI for Indic Languages
Researchers have introduced IndicTalk, a substantial new corpus of multilingual, code-mixed conversations designed to advance conversational AI for Indic languages. The dataset features over 1.3 million multi-turn dialo…
-
New benchmark reveals LLMs struggle with Romanized Indic-English code-mixing
A new benchmark called Indi-RomCoM has been developed to evaluate Large Language Models (LLMs) on their ability to understand and process Romanized Code Mixing (RCM), a communication style blending Indic languages with …
-
Pāninian grammar offers unified framework for Indic language NLP
A new research paper proposes a unified computational framework for Indic languages, drawing inspiration from Pāninian grammar. The authors argue that the shared morphosyntactic architecture across these languages, form…
-
New IndicGuard model enhances LLM safety for Indic languages
Researchers have developed IndicGuard, a new multilingual safety guard model and dataset specifically designed for Indic languages. This model addresses the limitations of existing English-centric safety mechanisms by c…
-
AI Frameworks Enhance Multimodal Reasoning in Healthcare
Researchers are developing advanced multi-agent frameworks to enhance AI's capabilities in specialized domains like healthcare. These systems aim to improve reasoning accuracy and address limitations in multilingual and…
-
New ASR methods tackle compute scaling and multilingual evaluation
Researchers are developing new methods to improve automatic speech recognition (ASR) systems. One approach, LARM, uses a depth-conditioned looped Transformer to allow for adjustable test-time computation, achieving perf…
-
New pipeline targets abusive comments in Indic languages
Researchers have developed a multi-stage training pipeline for detecting abusive comments in Indic languages. The proposed system utilizes language-based preprocessing and an ensemble of models to identify harmful conte…