Multilingual AI benchmarks
Reasoning in languages other than English.
MGSM
Grade-school maths word problems translated into 10 languages, testing reasoning outside English.
Run by Vals AI (benchmark by Google Research). Leaderboard as of 9 Jan 2026 · retrieved 21 Oct 2026. Showing top 25 of 75 models listed.
- Anthropic
- OpenAI
- Meta
- DeepSeek
- Alibaba
- Accuracy
- Share of problems answered correctly, averaged across languages.
Last updated by Vals in January 2026, so newer models are missing.
| Rank | Model | Provider | Accuracy | Price in / out ($/1M tokens) |
|---|---|---|---|---|
| 1 | Claude Opus 4.5 (Thinking) | Anthropic | 95.20% | $5 / $25 |
| 2 | Claude Opus 4.5 (Nonthinking) | Anthropic | 94.76% | $5 / $25 |
| 3 | Claude Opus 4.1 (Thinking) | Anthropic | 94.44% | $15 / $75 |
| 4 | Claude Sonnet 4.5 (Thinking) | Anthropic | 94.33% | $3 / $15 |
| 5 | Claude Opus 4.1 (Nonthinking) | Anthropic | 94.22% | $15 / $75 |
| 6 | GPT 5.2 | OpenAI | 94.00% | $1.75 / $14 |
| 7 | Gemini 3 Pro (11/25) | 93.93% | $2 / $12 | |
| 8 | Claude Opus 4 (Nonthinking) | Anthropic | 93.78% | $15 / $75 |
| 9 | o4 Mini | OpenAI | 93.42% | $1.1 / $4.4 |
| 10 | Gemini 3 Flash (12/25) | 93.31% | $0.5 / $3 |
Show 15 more rows
| Rank | Model | Provider | Accuracy | Price in / out ($/1M tokens) |
|---|---|---|---|---|
| 11 | Claude Sonnet 4 (Nonthinking) | Anthropic | 93.02% | $3 / $15 |
| 12 | GPT 5.1 | OpenAI | 92.98% | $1.25 / $10 |
| 13 | Claude 3.7 Sonnet (Thinking) | Anthropic | 92.98% | $3 / $15 |
| 14 | GPT 5 | OpenAI | 92.84% | $1.25 / $10 |
| 15 | Claude 3.5 Sonnet Latest | Anthropic | 92.58% | $3 / $15 |
| 16 | GPT 5 Mini | OpenAI | 92.58% | $0.25 / $2 |
| 17 | Qwen 3 (235B) | Alibaba | 92.47% | $0.22 / $0.88 |
| 18 | Llama 4 Maverick | Meta | 92.44% | $0.22 / $0.88 |
| 19 | Claude 3.7 Sonnet (Nonthinking) | Anthropic | 92.40% | $3 / $15 |
| 20 | DeepSeek R1 | DeepSeek | 92.25% | $1.35 / $5.4 |
| 21 | Claude Haiku 4.5 (Thinking) | Anthropic | 92.15% | $1 / $5 |
| 22 | DeepSeek V3 | DeepSeek | 92.15% | $0.9 / $0.9 |
| 23 | Qwen 3 Max Preview | Alibaba | 92.15% | $1.2 / $6 |
| 24 | GPT OSS 120B | OpenAI | 92.04% | $0.15 / $0.6 |
| 25 | Qwen 3 Max | Alibaba | 91.82% | $1.2 / $6 |
Source: Vals AI leaderboard ↗
Top model here: Claude Opus 4.5 (Thinking) (Anthropic). Try prompts for Claude →
ChatJudge does not run these tests. Scores are copied as published by each source, with a link and the date we read them, and only change when we refresh from the source. Different leaderboards use different settings, so compare models within one chart, not across charts. Spotted a mistake? Tell us.