Leaderboard
Market-Bench
Models write executable backtesters from plain-language descriptions of three strategies: scheduled trading on MSFT, KO/PEP pairs trading, and MSFT delta hedging. Outputs are compared with a reference implementation.
Best reported
443.2
grok-4
mean MAE
date not stated
Market-Bench paper v2 (2026-01-20)
mean MAE, lower is better. 10 models.
Updated date not stated
- grok-4443.2
- gpt-5.2969.4
- gemini-3-pro-preview1744.3
- gpt-5.1-codex-max4242.9
- deepseek-v3.24575.6
- claude-sonnet-4.55126.8
- claude-opus-4.56039.6
- command-a6562.1
- nova-premier-v17740.3
- llama-3.1-nemotron-ultra9674.2
Market-Bench paper v2 (2026-01-20)
mean MAE, lower is better. 10 models.
Updated date not stated
- 1grok-4443.2
- 2gpt-5.2969.4
- 3gemini-3-pro-preview1744.3
- 4gpt-5.1-codex-max4242.9
- 5deepseek-v3.24575.6
- 6claude-sonnet-4.55126.8
- 7claude-opus-4.56039.6
- 8command-a6562.1
- 9nova-premier-v17740.3
- 10llama-3.1-nemotron-ultra9674.2
| 1 | grok-4 | xAI | 443.2 | Default | marketbench.ai leaderboard Official board, date not stated |
|---|---|---|---|---|---|
| 2 | gpt-5.2 | OpenAI | 969.4 | Default | marketbench.ai leaderboard Official board, date not stated |
| 3 | gemini-3-pro-preview | 1744.3 | Default | marketbench.ai leaderboard Official board, date not stated | |
| 4 | gpt-5.1-codex-max | OpenAI | 4242.9 | Default | marketbench.ai leaderboard Official board, date not stated |
| 5 | deepseek-v3.2 | DeepSeek | 4575.6 | Default | marketbench.ai leaderboard Official board, date not stated |
| 6 | claude-sonnet-4.5 | Anthropic | 5126.8 | Default | marketbench.ai leaderboard Official board, date not stated |
| 7 | claude-opus-4.5 | Anthropic | 6039.6 | Default | marketbench.ai leaderboard Official board, date not stated |
| 8 | command-a | Cohere | 6562.1 | Default | marketbench.ai leaderboard Official board, date not stated |
| 9 | nova-premier-v1 | Amazon | 7740.3 | Default | marketbench.ai leaderboard Official board, date not stated |
| 10 | llama-3.1-nemotron-ultra | Meta | 9674.2 | Default | marketbench.ai leaderboard Official board, date not stated |