GoodFin, NYU and Rochester Institute of Technology authors (Shetty et al.), 2025
Advanced Financial Reasoning at Scale (CFA Level III evaluation)
CFA Level III mock exam: vignette multiple choice and essays. Prompting strategies compared include zero-shot, chain-of-thought with self-consistency, and Self-Discover.
Best reported
79.1
o4-miniOpenAI
overall % (mean of MCQ and essay)
Reported by the authors, 22 Sept 2025
About this benchmark
- Size
- 103 questions (60 MCQ in 10 vignettes, 43 essay questions worth 149 points), 23 models
- Public split
- none (paywalled AnalystPrep mock material)
- Grading
- MCQ accuracy. GPT-4.1 judges essays against the rubric and reference answer. A certified CFA Level III grader also scored the CoT-SC N=3 responses. Overall is the mean of MCQ and essay scores.
- Licence
- Not stated
- Submission
- none
- Products allowed
- No
- Latest version
- arXiv v2 (2025-09-22); FinLLM @ IJCAI 2025
- What it does not cover
- One exam level and one mock source. No tools, filings or live data.
Results
8 modelsEach score shows who reported it: the benchmark's authors (quoted from their paper), the benchmark's official board, or Moneda (run with the benchmark's own grader and rules). We do not copy other aggregators.
Advanced Financial Reasoning at Scale (CFA Level III evaluation) arXiv v2 (2025-09-22); FinLLM @ IJCAI 2025
overall % (mean of MCQ and essay), higher is better. 8 models.
Updated 22 Sept 2025
- o4-mini79.1
- Gemini 2.5 Flash77.3
- Gemini 2.5 Pro75.9
- Claude Opus 474.9
- o3-mini74.4
- Grok-373.8
- Claude Sonnet 473.4
- Grok-3 Mini Low Effort72.4
Advanced Financial Reasoning at Scale (CFA Level III evaluation) arXiv v2 (2025-09-22); FinLLM @ IJCAI 2025
overall % (mean of MCQ and essay), higher is better. 8 models.
Updated 22 Sept 2025
- 1o4-mini79.1
- 2Gemini 2.5 Flash77.3
- 3Gemini 2.5 Pro75.9
- 4Claude Opus 474.9
- 5o3-mini74.4
- 6Grok-373.8
- 7Claude Sonnet 473.4
- 8Grok-3 Mini Low Effort72.4
| 1 | o4-mini | OpenAI | 79.1 | MCQ Self-Discover; essay CoT-SC N=3 | Paper Table 1 Reported by the authors, 22 Sept 2025 |
|---|---|---|---|---|---|
| 2 | Gemini 2.5 Flash | 77.3 | MCQ zero-shot; essay CoT-SC N=3 | Paper Table 1 Reported by the authors, 22 Sept 2025 | |
| 3 | Gemini 2.5 Pro | 75.9 | MCQ zero-shot; essay CoT-SC N=3 | Paper Table 1 Reported by the authors, 22 Sept 2025 | |
| 4 | Claude Opus 4 | Anthropic | 74.9 | MCQ CoT-SC N=3; essay CoT-SC N=3 | Paper Table 1 Reported by the authors, 22 Sept 2025 |
| 5 | o3-mini | OpenAI | 74.4 | MCQ CoT-SC N=5; essay Self-Discover or CoT-SC N=5 | Paper Table 1 Reported by the authors, 22 Sept 2025 |
| 6 | Grok-3 | xAI | 73.8 | MCQ CoT-SC N=3; essay CoT-SC N=5 | Paper Table 1 Reported by the authors, 22 Sept 2025 |
| 7 | Claude Sonnet 4 | Anthropic | 73.4 | MCQ CoT-SC N=3; essay CoT-SC N=3 | Paper Table 1 Reported by the authors, 22 Sept 2025 |
| 8 | Grok-3 Mini Low Effort | xAI | 72.4 | MCQ CoT-SC N=3; essay CoT-SC N=3 | Paper Table 1 Reported by the authors, 22 Sept 2025 |