CodeBLEU
PulseAugur coverage of CodeBLEU — every cluster mentioning CodeBLEU across labs, papers, and developer communities, ranked by signal.
-
New 128GB Corpus and Benchmark Highlight LLM Deficiencies in Scientific Code Generation
Researchers have introduced SciCodePile, a substantial 128GB corpus of scientific code, to address the limitations of existing datasets in evaluating large language models' (LLMs) ability to generate scientific code. Th…
-
New research questions LLM fine-tuning effectiveness for Dart code decompilation
A new research paper explores the effectiveness of fine-tuning large language models for the neural decompilation of Dart Ahead-of-Time (AOT) binaries. The study found that fine-tuning did not significantly improve pass…
-
Researchers evaluate code metrics for detecting software plagiarism
A new research paper investigates whether standard code evaluation metrics can effectively detect plagiarism in source code. The study compared five metrics—CodeBLEU, CrystalBLEU, RUBY, TSED, and CodeBERTScore—against s…