CodegenBench: Can LLMs Write Efficient Code Across Architectures?
Researchers have introduced CodegenBench, a new benchmark suite to evaluate the ability of large language models (LLMs) to generate efficient parallel code across diverse hardware architectures. The benchmark includes standard BLAS routines and specialized kernels for x86_64, Sunway, and Kunpeng platforms. Initial evaluations show that while LLMs perform well on common architectures, they struggle with domain-specific architectures lacking extensive public documentation and training data, indicating limitations in cross-platform generalization. AI
IMPACT Highlights limitations in LLM code generation for specialized hardware, suggesting a need for improved cross-platform generalization.