PulseAugur
EN
LIVE 14:13:56

LLMs show language bias in code generation, study finds · 3 sources tracked

By PulseAugur Editorial · [3 sources] ·

A new study published on arXiv explores the impact of prompt language on code generation quality across different Large Language Models (LLMs). Researchers found that the language used to prompt models like GPT-4o mini, DeepSeek, and Claude can significantly affect the functional correctness and structural quality of the generated code. The study utilized a benchmark of 460 Python and Java coding tasks, with prompts translated into Chinese, Hindi, Spanish, and Italian, revealing that English prompts do not always yield the best results and that generated code can sometimes mix languages. AI

IMPACT This research highlights the need for multilingual prompt engineering to optimize code generation quality across different LLMs and programming languages.

RANK_REASON The cluster contains a research paper detailing a benchmark and study on LLM code generation.

Read on arXiv cs.AI →

AI-generated summary · Google Gemini · from 3 sources. How we write summaries →

LLMs show language bias in code generation, study finds · 3 sources tracked

COVERAGE [3]

  1. arXiv cs.AI TIER_1 English(EN) · Saima Afrin, Alessandro Midolo, Camilo Escobar-Vel\'asquez, Mario Linares-V\'asquez, Weiyuan Ding, Bowen Xu, Massimiliano Di Penta, Antonio Mastropaolo ·

    Large Language Models for Code Generation from Multilingual Prompts: A Curated Benchmark and a Study on Code Quality

    arXiv:2607.14816v1 Announce Type: cross Abstract: Large Language Models (LLMs) perform differently on identical programming tasks when prompted in different natural languages, a phenomenon known as language bias. While this behavior has been widely studied for general text genera…

  2. arXiv cs.AI TIER_1 English(EN) · Antonio Mastropaolo ·

    Large Language Models for Code Generation from Multilingual Prompts: A Curated Benchmark and a Study on Code Quality

    Large Language Models (LLMs) perform differently on identical programming tasks when prompted in different natural languages, a phenomenon known as language bias. While this behavior has been widely studied for general text generation, its impact on code generation quality and pr…

  3. Hugging Face Daily Papers TIER_1 English(EN) ·

    Large Language Models for Code Generation from Multilingual Prompts: A Curated Benchmark and a Study on Code Quality

    Large Language Models (LLMs) perform differently on identical programming tasks when prompted in different natural languages, a phenomenon known as language bias. While this behavior has been widely studied for general text generation, its impact on code generation quality and pr…

RELATED ENTITIES

RELATED TOPICS