The new benchmarks like DeepSWE now show a very big gap in proprietary models and open source
New benchmarks like DeepSWE are revealing a significant performance gap between proprietary and open-source AI models. This disparity is currently disappointing for the open-source community, which hopes to see advancements that can help it catch up. The current benchmarks indicate a substantial difference in capabilities, prompting a call for more progress in open-source AI development. AI
IMPACT Highlights the growing performance divide, potentially influencing future development priorities for open-source AI.