Leaderboard
CAIA (Crypto AI Agent Benchmark)
Time-anchored crypto analyst tasks (on-chain analysis, tokenomics and similar) in adversarial, misinformation-heavy conditions. Models are run with and without tools.
Best reported
67.4
gpt-5
accuracy % (majority vote, 5 runs)
17 Jan 2026
CAIA (Crypto AI Agent Benchmark) 0927 release; paper v2 (2026-01-17)
accuracy % (majority vote, 5 runs), higher is better. 10 models.
Updated 17 Jan 2026
- gpt-567.4
- GPT-OSS 120B62.9
- Grok 4 Fast Reasoning61.2
- Claude Opus 457.3
- Claude Sonnet 456.7
- Claude Opus 4.156.3
- GPT-o353.4
- DeepSeek V3.149.2
- Kimi K2 Instruct47.2
- GPT-4.146.6
CAIA (Crypto AI Agent Benchmark) 0927 release; paper v2 (2026-01-17)
accuracy % (majority vote, 5 runs), higher is better. 10 models.
Updated 17 Jan 2026
- 1gpt-567.4
- 2GPT-OSS 120B62.9
- 3Grok 4 Fast Reasoning61.2
- 4Claude Opus 457.3
- 5Claude Sonnet 456.7
- 6Claude Opus 4.156.3
- 7GPT-o353.4
- 8DeepSeek V3.149.2
- 9Kimi K2 Instruct47.2
- 10GPT-4.146.6
| 1 | gpt-5 | OpenAI | 67.4 | with tools | Paper Tables 2-3 Reported by the authors, 17 Jan 2026 |
|---|---|---|---|---|---|
| 2 | GPT-OSS 120B | OpenAI | 62.9 | with tools | Paper Tables 2-3 Reported by the authors, 17 Jan 2026 |
| 3 | Grok 4 Fast Reasoning | xAI | 61.2 | with tools | Paper Tables 2-3 Reported by the authors, 17 Jan 2026 |
| 4 | Claude Opus 4 | Anthropic | 57.3 | with tools | Paper Tables 2-3 Reported by the authors, 17 Jan 2026 |
| 5 | Claude Sonnet 4 | Anthropic | 56.7 | with tools | Paper Tables 2-3 Reported by the authors, 17 Jan 2026 |
| 6 | Claude Opus 4.1 | Anthropic | 56.3 | with tools | Paper Tables 2-3 Reported by the authors, 17 Jan 2026 |
| 7 | GPT-o3 | OpenAI | 53.4 | with tools | Paper Tables 2-3 Reported by the authors, 17 Jan 2026 |
| 8 | DeepSeek V3.1 | DeepSeek | 49.2 | with tools | Paper Tables 2-3 Reported by the authors, 17 Jan 2026 |
| 9 | Kimi K2 Instruct | Moonshot AI | 47.2 | with tools | Paper Tables 2-3 Reported by the authors, 17 Jan 2026 |
| 10 | GPT-4.1 | OpenAI | 46.6 | with tools | Paper Tables 2-3 Reported by the authors, 17 Jan 2026 |