Leaderboard, made by Jiaxuan Lu, Kong Wang et al.
FinToolBench
Tests whether LLM agents can call real financial tools to answer questions while staying timely, on intent and in the right regulatory domain.
FinToolBench v2 (arXiv, 2026-08-01)
Soft Score, higher is better. 7 models.
Updated 1 Aug 2026
- Gemini-3.1-Pro (preview)0.822
- Claude Sonnet 4.50.712
- Grok-3-beta0.685
- GPT-5.40.625
- Doubao-Seed-1.60.463
- Qwen3-8B0.404
- GLM-4.7-Flash0.379
FinToolBench v2 (arXiv, 2026-08-01)
Soft Score, higher is better. 7 models.
Updated 1 Aug 2026
- 1Gemini-3.1-Pro (preview)0.822
- 2Claude Sonnet 4.50.712
- 3Grok-3-beta0.685
- 4GPT-5.40.625
- 5Doubao-Seed-1.60.463
- 6Qwen3-8B0.404
- 7GLM-4.7-Flash0.379
| 1 | Gemini-3.1-Pro (preview) | 0.822 | Full 295 questions, GPT-5.1 judge, mean of 3 repeats | Paper Table 2 Reported by the authors, reported 1 Aug 2026 · checked 6 Nov 2026 | |
|---|---|---|---|---|---|
| 2 | Claude Sonnet 4.5 | Anthropic | 0.712 | Full 295 questions, GPT-5.1 judge, mean of 3 repeats | Paper Table 2 Reported by the authors, reported 1 Aug 2026 · checked 6 Nov 2026 |
| 3 | Grok-3-beta | xAI | 0.685 | Full 295 questions, GPT-5.1 judge, mean of 3 repeats | Paper Table 2 Reported by the authors, reported 1 Aug 2026 · checked 6 Nov 2026 |
| 4 | GPT-5.4 | OpenAI | 0.625 | Full 295 questions, GPT-5.1 judge, mean of 3 repeats | Paper Table 2 Reported by the authors, reported 1 Aug 2026 · checked 6 Nov 2026 |
| 5 | Doubao-Seed-1.6 | ByteDance | 0.463 | Full 295 questions, GPT-5.1 judge, mean of 3 repeats | Paper Table 2 Reported by the authors, reported 1 Aug 2026 · checked 6 Nov 2026 |
| 6 | Qwen3-8B | Qwen | 0.404 | Full 295 questions, GPT-5.1 judge, mean of 3 repeats | Paper Table 2 Reported by the authors, reported 1 Aug 2026 · checked 6 Nov 2026 |
| 7 | GLM-4.7-Flash | Zhipu AI | 0.379 | Full 295 questions, GPT-5.1 judge, mean of 3 repeats | Paper Table 2 Reported by the authors, reported 1 Aug 2026 · checked 6 Nov 2026 |