# Advanced Financial Reasoning at Scale (CFA Level III evaluation)

CFA Level III mock exam: vignette multiple choice and essays. Prompting strategies compared include zero-shot, chain-of-thought with self-consistency, and Self-Discover.

## Facts

- Maker: GoodFin, NYU and Rochester Institute of Technology authors (Shetty et al.)
- Year: 2025
- Use case: Finance knowledge
- Size: 103 questions (60 MCQ in 10 vignettes, 43 essay questions worth 149 points), 23 models
- Public split: none (paywalled AnalystPrep mock material)
- Grading: MCQ accuracy. GPT-4.1 judges essays against the rubric and reference answer. A certified CFA Level III grader also scored the CoT-SC N=3 responses. Overall is the mean of MCQ and essay scores.
- Licence: Not stated
- Submission: none
- Products allowed: No
- Latest version: arXiv v2 (2025-09-22); FinLLM @ IJCAI 2025
- What it does not cover: One exam level and one mock source. No tools, filings or live data.
- Last checked: 1 Nov 2026
- Links: paper https://arxiv.org/abs/2507.02954
- Directory page: https://benchmarks.finance/benchmarks/cfa-level-3-at-scale/
- Leaderboard: https://benchmarks.finance/leaderboards/cfa-level-3-at-scale/

## Results

### Advanced Financial Reasoning at Scale (CFA Level III evaluation): overall % (mean of MCQ and essay) (higher is better, 8 models)

| Rank | Model | Maker | Score | Metric | Setting | Reported by | Source | URL | Date |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1 | o4-mini | OpenAI | 79.1 | overall % (mean of MCQ and essay) | MCQ Self-Discover; essay CoT-SC N=3 | Reported by the authors | Paper Table 1 | https://arxiv.org/html/2507.02954v2 | 22 Sept 2025 |
| 2 | Gemini 2.5 Flash | Google | 77.3 | overall % (mean of MCQ and essay) | MCQ zero-shot; essay CoT-SC N=3 | Reported by the authors | Paper Table 1 | https://arxiv.org/html/2507.02954v2 | 22 Sept 2025 |
| 3 | Gemini 2.5 Pro | Google | 75.9 | overall % (mean of MCQ and essay) | MCQ zero-shot; essay CoT-SC N=3 | Reported by the authors | Paper Table 1 | https://arxiv.org/html/2507.02954v2 | 22 Sept 2025 |
| 4 | Claude Opus 4 | Anthropic | 74.9 | overall % (mean of MCQ and essay) | MCQ CoT-SC N=3; essay CoT-SC N=3 | Reported by the authors | Paper Table 1 | https://arxiv.org/html/2507.02954v2 | 22 Sept 2025 |
| 5 | o3-mini | OpenAI | 74.4 | overall % (mean of MCQ and essay) | MCQ CoT-SC N=5; essay Self-Discover or CoT-SC N=5 | Reported by the authors | Paper Table 1 | https://arxiv.org/html/2507.02954v2 | 22 Sept 2025 |
| 6 | Grok-3 | xAI | 73.8 | overall % (mean of MCQ and essay) | MCQ CoT-SC N=3; essay CoT-SC N=5 | Reported by the authors | Paper Table 1 | https://arxiv.org/html/2507.02954v2 | 22 Sept 2025 |
| 7 | Claude Sonnet 4 | Anthropic | 73.4 | overall % (mean of MCQ and essay) | MCQ CoT-SC N=3; essay CoT-SC N=3 | Reported by the authors | Paper Table 1 | https://arxiv.org/html/2507.02954v2 | 22 Sept 2025 |
| 8 | Grok-3 Mini Low Effort | xAI | 72.4 | overall % (mean of MCQ and essay) | MCQ CoT-SC N=3; essay CoT-SC N=3 | Reported by the authors | Paper Table 1 | https://arxiv.org/html/2507.02954v2 | 22 Sept 2025 |


Source: https://benchmarks.finance/benchmarks/cfa-level-3-at-scale/

Created and maintained by Moneda.