Leaderboard
Advanced Financial Reasoning at Scale (CFA Level III evaluation)
CFA Level III mock exam: vignette multiple choice and essays. Prompting strategies compared include zero-shot, chain-of-thought with self-consistency, and Self-Discover.
Best reported
79.1
o4-mini
overall % (mean of MCQ and essay)
22 Sept 2025
Advanced Financial Reasoning at Scale (CFA Level III evaluation) arXiv v2 (2025-09-22); FinLLM @ IJCAI 2025
overall % (mean of MCQ and essay), higher is better. 8 models.
Updated 22 Sept 2025
- o4-mini79.1
- Gemini 2.5 Flash77.3
- Gemini 2.5 Pro75.9
- Claude Opus 474.9
- o3-mini74.4
- Grok-373.8
- Claude Sonnet 473.4
- Grok-3 Mini Low Effort72.4
Advanced Financial Reasoning at Scale (CFA Level III evaluation) arXiv v2 (2025-09-22); FinLLM @ IJCAI 2025
overall % (mean of MCQ and essay), higher is better. 8 models.
Updated 22 Sept 2025
- 1o4-mini79.1
- 2Gemini 2.5 Flash77.3
- 3Gemini 2.5 Pro75.9
- 4Claude Opus 474.9
- 5o3-mini74.4
- 6Grok-373.8
- 7Claude Sonnet 473.4
- 8Grok-3 Mini Low Effort72.4
| 1 | o4-mini | OpenAI | 79.1 | MCQ Self-Discover; essay CoT-SC N=3 | Paper Table 1 Reported by the authors, 22 Sept 2025 |
|---|---|---|---|---|---|
| 2 | Gemini 2.5 Flash | 77.3 | MCQ zero-shot; essay CoT-SC N=3 | Paper Table 1 Reported by the authors, 22 Sept 2025 | |
| 3 | Gemini 2.5 Pro | 75.9 | MCQ zero-shot; essay CoT-SC N=3 | Paper Table 1 Reported by the authors, 22 Sept 2025 | |
| 4 | Claude Opus 4 | Anthropic | 74.9 | MCQ CoT-SC N=3; essay CoT-SC N=3 | Paper Table 1 Reported by the authors, 22 Sept 2025 |
| 5 | o3-mini | OpenAI | 74.4 | MCQ CoT-SC N=5; essay Self-Discover or CoT-SC N=5 | Paper Table 1 Reported by the authors, 22 Sept 2025 |
| 6 | Grok-3 | xAI | 73.8 | MCQ CoT-SC N=3; essay CoT-SC N=5 | Paper Table 1 Reported by the authors, 22 Sept 2025 |
| 7 | Claude Sonnet 4 | Anthropic | 73.4 | MCQ CoT-SC N=3; essay CoT-SC N=3 | Paper Table 1 Reported by the authors, 22 Sept 2025 |
| 8 | Grok-3 Mini Low Effort | xAI | 72.4 | MCQ CoT-SC N=3; essay CoT-SC N=3 | Paper Table 1 Reported by the authors, 22 Sept 2025 |
Full details for Advanced Financial Reasoning at Scale (CFA Level III evaluation)