AfterQuery, 2025
FinanceQA
Analyst-style numerical tasks based on 10-K data: hand-spreading metrics, applying accounting and valuation conventions, and working from incomplete information that needs assumptions. Also includes conceptual questions.
About this benchmark
- Size
- 148 questions (HF test split; types: basic, assumption, conceptual)
- Public split
- 148-row test split on Hugging Face
- Grading
- Exact correct match, scored 1 or 0 with no partial credit
- Licence
- Apache-2.0 (data)
- Submission
- none
- Products allowed
- Not stated
- Latest version
- v1 (2025-01-30)
- What it does not cover
- No agentic retrieval, multi-document research or market data.
Results
5 modelsEach score shows who reported it: the benchmark's authors (quoted from their paper), the benchmark's official board, or Moneda (run with the benchmark's own grader and rules). We do not copy other aggregators.
FinanceQA v1 (2025-01-30)
accuracy (total, 0-1), higher is better. 5 models.
Updated 1 Nov 2026
- o10.487
- Claude-3-5-sonnet-202410220.399
- GPT-4o0.392
- Numa 10.355
- Llama-3.3-70B-Instruct0.311
FinanceQA v1 (2025-01-30)
accuracy (total, 0-1), higher is better. 5 models.
Updated 1 Nov 2026
- 1o10.487
- 2Claude-3-5-sonnet-202410220.399
- 3GPT-4o0.392
- 4Numa 10.355
- 5Llama-3.3-70B-Instruct0.311
| 1 | o1 | OpenAI | 0.487 | Default | Paper Table 1 Reported by the authors, 30 Jan 2025 |
|---|---|---|---|---|---|
| 2 | Claude-3-5-sonnet-20241022 | Anthropic | 0.399 | Default | Paper Table 1 Reported by the authors, 30 Jan 2025 |
| 3 | GPT-4o | OpenAI | 0.392 | Default | Paper Table 1 Reported by the authors, 30 Jan 2025 |
| 4 | Numa 1 | Moneda | 0.355 | Product with tools; 148-question test split; graded by a two-judge panel, not the authors' grader | Moneda run, 2026-11-01 Run by Moneda, 1 Nov 2026 |
| 5 | Llama-3.3-70B-Instruct | Meta | 0.311 | Default | Paper Table 1 Reported by the authors, 30 Jan 2025 |
Papers and links
Last checked 1 Nov 2026. Something out of date or wrong? Send a correction.