SWE-bench Verified leaderboard (Sep 2026): AI model scores
As of 1 Sep 2026, the highest reported Google score on SWE-bench Verified is Gemini 3.7 Flash (Google) at 80.8%, #23 on that leaderboard, per Vals AI. The source no longer updates this leaderboard.
What it measures
Resolving real-world GitHub issues with a code patch, checked by unit tests.
Fixing real GitHub issues in open-source Python projects; a fix counts if the project tests pass.
What the score means
Score: share of issues resolved
Share of issues resolved: the model’s patch makes the unit tests pass.
Results from published AI leaderboards. Compare models within each benchmark; scores from different tests aren’t directly comparable. How we collect these results
Scores by model
Run by Vals AI (benchmark by Princeton/OpenAI). Leaderboard dated 1 Sep 2026 (no longer updated by the source). Last checked 22 Oct 2026. Showing top 25 of 88 models listed.
Highlight a provider: Anthropic OpenAI Google xAI Meta DeepSeek Alibaba Z.ai Moonshot
Highlighting Google models. Other providers are dimmed. Show all providers
- Anthropic
- OpenAI
- xAI
- Meta
- DeepSeek
- Alibaba
- Z.ai
- Moonshot
- Other
- Score
- Share of issues resolved: the model’s patch makes the unit tests pass.
- Cost per test (USD)
- Average API cost per issue, as published.
All scores
| Rank | Model | Provider | Score | Cost per test (USD) | Harness | Price in / out ($/1M tokens) |
|---|---|---|---|---|---|---|
| 1 | Claude Opus 5 | Anthropic | 97.0% | $1.29 | Mini-SWE-agent | $5 / $25 |
| 2 | DeepSeek V4 Pro 0813 | DeepSeek | 96.4% | $0.10 | Mini-SWE-agent | $1.32 / $3.96 |
| 3 | GPT-5.6 Sol | OpenAI | 96.2% | $1.15 | Mini-SWE-agent | $4 / $20 |
| 4 | Grok 4.6 | xAI | 95.6% | $0.78 | Mini-SWE-agent | $2 / $6 |
| 5 | GPT-5.6 Terra | OpenAI | 95.4% | $0.40 | Mini-SWE-agent | $2 / $12 |
| 6 | GLM 5.3 | Z.ai | 95.4% | $0.34 | Mini-SWE-agent | $1.4 / $4.4 |
| 7 | Claude Fable 5 | Anthropic | 95.0% | $2.05 | Mini-SWE-agent | $10 / $50 |
| 8 | Kimi K3 | Moonshot | 93.4% | $0.76 | Mini-SWE-agent | $3 / $15 |
| 9 | GPT-5.6 Luna | OpenAI | 93.0% | $0.040 | Mini-SWE-agent | $0.2 / $1.2 |
| 10 | GLM 5.3 Flash | Z.ai | 92.0% | $0.020 | Mini-SWE-agent | $0.075 / $0.25 |
Show 15 more rows
| Rank | Model | Provider | Score | Cost per test (USD) | Harness | Price in / out ($/1M tokens) |
|---|---|---|---|---|---|---|
| 11 | DeepSeek V4 Flash 0731 | DeepSeek | 88.8% | $0.010 | Mini-SWE-agent | $0.44 / $1.32 |
| 12 | Claude Opus 4.8 | Anthropic | 88.6% | $1.92 | Mini-SWE-agent | $5 / $25 |
| 13 | Grok 4.5 | xAI | 86.6% | $0.54 | Mini-SWE-agent | $2 / $6 |
| 14 | Muse Spark 1.2 | Meta | 86.6% | $0.55 | Mini-SWE-agent | $1.25 / $4.25 |
| 15 | Qwen 3.8 27B | Alibaba | 86.0% | $1.08 | Mini-SWE-agent | $0.5 / $3 |
| 16 | Claude Opus 4.8 | Anthropic | 85.8% | $0.67 | Claude Code | N/A |
| 17 | Qwen 3.8 Max | Alibaba | 85.6% | $1.13 | Mini-SWE-agent | $2 / $6 |
| 18 | GLM 5.2 | Z.ai | 82.8% | $0.71 | Mini-SWE-agent | $1.4 / $4.4 |
| 19 | GPT 5.5 | OpenAI | 82.6% | $1.36 | Mini-SWE-agent | $5 / $30 |
| 20 | Inkling Small | Other | 82.2% | $0.13 | Mini-SWE-agent | $0.3 / $1.2 |
| 21 | Muse Spark 1.1 | Meta | 82.0% | $0.35 | Mini-SWE-agent | $1.25 / $4.25 |
| 22 | Claude Opus 4.7 | Anthropic | 82.0% | $2.42 | Mini-SWE-agent | $5 / $25 |
| 23 | Gemini 3.7 Flash | 80.8% | $1.44 | Mini-SWE-agent | $1.5 / $7.5 | |
| 24 | Gemini 3.8 Flash | 80.0% | $2.19 | Mini-SWE-agent | $1.5 / $7.5 | |
| 25 | Composer 2.5 | Other | 79.6% | – | Cursor CLI | $0.5 / $2.5 |
Source: Vals AI leaderboard ↗ · SWE-bench project ↗
Best-ranked Google model here: Gemini 3.7 Flash (Google). Try prompts for Gemini →
Prompts for this kind of work
Copy-ready prompts from our library that match what SWE-bench Verified tests. Run them on your own material to compare models.