# CAIA (Crypto AI Agent Benchmark) leaderboard

Time-anchored crypto analyst tasks (on-chain analysis, tokenomics and similar) in adversarial, misinformation-heavy conditions. Models are run with and without tools.

## CAIA (Crypto AI Agent Benchmark): accuracy % (majority vote, 5 runs) (higher is better, 10 models)

| Rank | Model | Maker | Score | Metric | Setting | Reported by | Source | URL | Date |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1 | gpt-5 | OpenAI | 67.4 | accuracy % (majority vote, 5 runs) | with tools | Reported by the authors | Paper Tables 2-3 | https://arxiv.org/html/2510.00332v2 | 17 Jan 2026 |
| 2 | GPT-OSS 120B | OpenAI | 62.9 | accuracy % (majority vote, 5 runs) | with tools | Reported by the authors | Paper Tables 2-3 | https://arxiv.org/html/2510.00332v2 | 17 Jan 2026 |
| 3 | Grok 4 Fast Reasoning | xAI | 61.2 | accuracy % (majority vote, 5 runs) | with tools | Reported by the authors | Paper Tables 2-3 | https://arxiv.org/html/2510.00332v2 | 17 Jan 2026 |
| 4 | Claude Opus 4 | Anthropic | 57.3 | accuracy % (majority vote, 5 runs) | with tools | Reported by the authors | Paper Tables 2-3 | https://arxiv.org/html/2510.00332v2 | 17 Jan 2026 |
| 5 | Claude Sonnet 4 | Anthropic | 56.7 | accuracy % (majority vote, 5 runs) | with tools | Reported by the authors | Paper Tables 2-3 | https://arxiv.org/html/2510.00332v2 | 17 Jan 2026 |
| 6 | Claude Opus 4.1 | Anthropic | 56.3 | accuracy % (majority vote, 5 runs) | with tools | Reported by the authors | Paper Tables 2-3 | https://arxiv.org/html/2510.00332v2 | 17 Jan 2026 |
| 7 | GPT-o3 | OpenAI | 53.4 | accuracy % (majority vote, 5 runs) | with tools | Reported by the authors | Paper Tables 2-3 | https://arxiv.org/html/2510.00332v2 | 17 Jan 2026 |
| 8 | DeepSeek V3.1 | DeepSeek | 49.2 | accuracy % (majority vote, 5 runs) | with tools | Reported by the authors | Paper Tables 2-3 | https://arxiv.org/html/2510.00332v2 | 17 Jan 2026 |
| 9 | Kimi K2 Instruct | Moonshot AI | 47.2 | accuracy % (majority vote, 5 runs) | with tools | Reported by the authors | Paper Tables 2-3 | https://arxiv.org/html/2510.00332v2 | 17 Jan 2026 |
| 10 | GPT-4.1 | OpenAI | 46.6 | accuracy % (majority vote, 5 runs) | with tools | Reported by the authors | Paper Tables 2-3 | https://arxiv.org/html/2510.00332v2 | 17 Jan 2026 |

Scores are never combined across benchmarks. Each row names its metric, source and date.

## About this benchmark

- Maker: Surf AI / Cybertino Lab and Princeton University
- Year: 2025
- Use case: Live financial market data
- Size: 178 tasks, 17 models
- Public split: All 178 questions (benchmark.csv on GitHub; dataset on Hugging Face). The release uses anonymized mock tools.
- Grading: Accuracy by majority vote over five runs, plus Pass@1 and Pass@5. Ground truth requires a reproducible tool-call sequence.
- Licence: CC-BY-4.0 (data); Apache-2.0 (code)
- Submission: Not stated
- Products allowed: Not stated
- Latest version: 0927 release; paper v2 (2026-01-17)
- What it does not cover: Crypto only. No equities or filings, no portfolio outcomes, and the public release uses mock rather than live tools.
- Last checked: 1 Nov 2026
- Links: paper https://arxiv.org/abs/2510.00332, data https://huggingface.co/datasets/cyberco/caia-0927, code https://github.com/caiba-ai/caia-benchmark-0927, leaderboard https://huggingface.co/spaces/cyberco/CAIA_0927_Leaderboard
- Directory page: https://benchmarks.finance/benchmarks/caia/
- Leaderboard: https://benchmarks.finance/leaderboards/caia/

Source: https://benchmarks.finance/leaderboards/caia/

Created and maintained by Moneda.