SWE-bench Multimodal Leaderboard
coding8 models · Updated October 2026
On the SWE-bench Multimodal benchmark, OpenAI: o3 ranks #1 with a score of 36.0, while OpenAI: o4 Mini offers the best score-per-dollar at $4.40/1M output tokens. The full ranking, with cost per million tokens, is below.
💰 Best value
OpenAI: o4 Mini — score 33.9 at $4.40/1M output tokens
| # | Model | Score | Output / 1M | Context |
|---|---|---|---|---|
| 1 | OpenAI: o3OpenAI | 36.0 | $8.00 | 200K |
| 2 | Claude 4 SonnetNanoGPT | 35.6 | $14.99 | 200K |
| 3 | OpenAI: o4 MiniOpenAI | 33.9 | $4.40 | 200K |
| 4 | Claude 3.7 SonnetAnthropic | 31.3 | — | — |
| 5 | OpenAI: GPT-4.1OpenAI | 31.1 | $8.00 | 1M |
| 6 | OpenAI: GPT-4oOpenAI | 30.4 | $10.00 | 128K |
| 7 | Claude 3.5 SonnetAnthropic | 25.3 | — | — |
| 8 | Claude Sonnet 3.5Anthropic | 12.2 | $15.00 | 200K |
What does SWE-bench Multimodal test?
Evaluation framework with 2,294 real GitHub software-engineering problems.
Frequently asked questions
Pricing is indicative — confirm with the provider before production use. Updated October 2026.