Sign in
← Coding benchmarks

DeepSWE leaderboard (Sep 2026): AI model scores

As of 22 Sep 2026, the highest reported score on DeepSWE is GPT-6 Astra (xhigh) (OpenAI) at 74.12%, per Datacurve.

DeepSWE

Original, long-horizon software engineering tasks written from scratch, to keep them out of training data.

Loading chart…

Each square is one model, marked with its provider’s initial. Higher is better; further left costs less per task.

Highest reported score: GPT-6 Astra (xhigh) (OpenAI) at 74.12%.

Pass@1, highest first. Position is the source’s published rank.

  1. 1GPT-6 Astra (xhigh) OpenAI74.12%
  2. 2Gemini 3.8 Flash (high) Google73.83%
  3. 3Claude Opus 5 (max) Anthropic73.65%
  4. 4GPT-5.6 Sol (max) OpenAI72.67%
  5. 5Claude Fable 5 (xhigh) Anthropic69.91%
  6. 6GLM 5.3 (max) Z.ai68.96%
  7. 7Kimi K3 (max) Moonshot68.51%
  8. 8Grok 4.6 (medium) xAI67.48%
  9. 9GPT-5.6 Luna (max) OpenAI67.19%
  10. 10GPT-5.5 (xhigh) OpenAI67.04%
  11. 11Gemini 3.7 Flash (medium) Google65.49%
  12. 12GLM 5.3 Flash (max) Z.ai63.39%
  13. 13DeepSeek V4 Pro (max) DeepSeek62.83%
  14. 14Claude Opus 4.8 (max) Anthropic58.97%
  15. 15Qwen 3.8 Max (xhigh) Alibaba57.46%
  16. 16Muse Spark 1.2 (xhigh) Meta54.87%
  17. 17Claude Sonnet 5 (max) Anthropic53.85%
  18. 18DeepSeek V4 Flash (max) DeepSeek53.32%
  19. 19Gemini 3.6 Flash (high) Google46.68%
  20. 20GLM 5.2 (max) Z.ai43.78%
  21. 21Gemini 3.5 Flash (high) Google36.06%
DeepSWE: all published values (21 models). “–” means the source lists no value.
1GPT-6 Astra (xhigh)OpenAI74.12%±2.87$4.4329,55728.8
2Gemini 3.8 Flash (high)Google73.83%±1.42$2.36143,243166.3
3Claude Opus 5 (max)Anthropic73.65%±3.87$12117,56699
4GPT-5.6 Sol (max)OpenAI72.67%±2.83$8.3960,01461.3
5Claude Fable 5 (xhigh)Anthropic69.91%±3.24$1380,35268.4
6GLM 5.3 (max)Z.ai68.96%±3.02$3.9980,436124.5
7Kimi K3 (max)Moonshot68.51%±4.54$4.6581,50097.6
8Grok 4.6 (medium)xAI67.48%±2.28$3.4549,76470.3
9GPT-5.6 Luna (max)OpenAI67.19%±3.99$3.0373,400101.7
10GPT-5.5 (xhigh)OpenAI67.04%±6.47$7.2346,29582
11Gemini 3.7 Flash (medium)Google65.49%±3.09$2.0393,991117.4
12GLM 5.3 Flash (max)Z.ai63.39%±4.38$0.4872,830122.9
13DeepSeek V4 Pro (max)DeepSeek62.83%±6.33$0.24105,999154.7
14Claude Opus 4.8 (max)Anthropic58.97%±1.76$13135,032120
15Qwen 3.8 Max (xhigh)Alibaba57.46%±2.66$3.7395,075111.3
16Muse Spark 1.2 (xhigh)Meta54.87%±2.12$3.7099,226100.8
17Claude Sonnet 5 (max)Anthropic53.85%±4.24$26214,118268.5
18DeepSeek V4 Flash (max)DeepSeek53.32%±3.57$0.10107,687152.9
19Gemini 3.6 Flash (high)Google46.68%±3.70$4.4295,845116.7
20GLM 5.2 (max)Z.ai43.78%±1.73$3.9278,175129.1
21Gemini 3.5 Flash (high)Google36.06%±3.97$3.4575,730105.3

Source: Datacurve · leaderboard dated 22 Sep 2026 · checked 24 Oct 2026

Prompts for coding work

All prompts
ProductClaudePaste-ready Claude Code promptHanding one scoped change to a coding agent, not a whole document.View prompt ProductClaudeReview a PRD or bug reportBefore building, to find what a spec misses or conflates.View prompt ProductClaudeConsolidated PRD for a Claude Code handoffAfter a long design conversation, to turn every decision into one build-ready document.View prompt
Methodology and caveats

What it measures

Original, long-horizon software engineering tasks written from scratch, to keep them out of training data.

113 original, long-horizon engineering tasks in 91 open-source repositories across 5 languages. Datacurve wrote the tasks from scratch rather than taking them from past pull requests, to keep them out of training data (contamination-free); hand-written tests check behaviour, not a specific patch.

What the score means

Pass@1: share of attempts passing the hidden tests, averaged over 4 runs

Pass@1
Share of task attempts that pass the hidden tests, averaged over 4 full runs (± is the 95% run-to-run interval).
± margin (points)
Half-width of the 95% run-to-run interval on Pass@1, in percentage points.
Cost per task (USD)
Average API cost per task attempt at list prices, as published.
Output tokens per task
Average output tokens per task attempt, including reasoning.
Agent steps per task
Average number of agent actions per task attempt.

Run by Datacurve. Leaderboard dated 22 Sep 2026. Last checked 24 Oct 2026.

GPT-6 Astra (xhigh) has the highest reported Pass@1, 74.12%. Rounded to whole points, as DeepSWE’s own page shows them, Gemini 3.8 Flash (high, 73.83%) and Claude Opus 5 (max, 73.65%) also read 74%. Gemini 3.8 Flash (high), Claude Opus 5 (max) and GPT-5.6 Sol (max) are within its ±2.87-point run-to-run margin. DeepSWE v1.1, default view: each model’s highest-scoring reasoning-effort setting, shown in parentheses after the name. Datacurve publishes 70 settings for 28 models; 7 models it hides by default are left out here too. All runs use the mini-swe-agent harness. Claude Opus 4.8 (max) was scored on 111 of 113 tasks.

Source: Datacurve leaderboard ↗ · How DeepSWE was built (Datacurve) ↗ · DeepSWE on GitHub ↗

Results from published AI leaderboards. Compare models within each benchmark; scores from different tests aren’t directly comparable. How we collect these results

Top model here: GPT-6 Astra (xhigh) (OpenAI). Try prompts for ChatGPT →

Questions people ask

Which AI has the highest reported score on DeepSWE?

As of 22 Sep 2026, the highest reported score on DeepSWE is GPT-6 Astra (xhigh) (OpenAI) at 74.12%, per Datacurve.

What does DeepSWE measure?

Original, long-horizon software engineering tasks written from scratch, to keep them out of training data. It is run by Datacurve.

What does the DeepSWE score mean?

Pass@1: share of attempts passing the hidden tests, averaged over 4 runs. Share of task attempts that pass the hidden tests, averaged over 4 full runs (± is the 95% run-to-run interval).

How recent are these DeepSWE results?

DeepSWE: leaderboard dated 22 Sep 2026; we last checked it on 24 Oct 2026.

What does the top-scoring model cost to run on DeepSWE?

On DeepSWE, GPT-6 Astra (xhigh) costs $4.43 (cost per task), as published by Datacurve. That is API cost on the benchmark’s own tasks, not a subscription price.

Can I compare DeepSWE scores with other benchmarks?

Not directly. Each leaderboard uses its own tasks, settings and scoring, so compare models within one benchmark. ChatJudge does not run the tests; we copy the published scores with a link to the source.