RepoProbe

New benchmark RepoProbe evaluates LLM code comprehension and Edit Bias

Researchers have introduced RepoProbe, a new benchmark designed to evaluate how well large language models (LLMs) can comprehend the architecture of software repositories. Unlike previous benchmarks that relied on GitHu…