SWE-bench Leaderboards

Select at least one model using the checkboxes in the first columns, or click one of the following buttons for a pre-defined selection.
Each entry reports % Resolved, the percentage of task instances solved.
The original benchmark: real GitHub issues from 12 Python repositories.
The default Verified view: every model in the same mini-SWE-agent environment.
A subset curated for less costly evaluation.
A human-filtered subset of SWE-bench.
Tasks from 42 repositories across 9 programming languages.
Issues described with visual elements.
We thank the following institutions for their generous support: Open Philanthropy, AWS, Modal, Andreessen Horowitz, OpenAI, and Anthropic.