# FAITH leaderboard

Measures intrinsic hallucination on financial tables. A numeric span is masked in a sentence from a 10-K and the model must recover it from the report's tables and surrounding context.

## FAITH: overall accuracy % (higher is better, 10 models)

| Rank | Model | Provider | Score | Metric | Setting | Reported by | Source | URL | Date |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1 | Claude Sonnet 4 | Anthropic | 95.6 | overall accuracy % | Main split | Reported by the authors | Paper Table 3 | https://arxiv.org/html/2508.05201v2 | 24 Oct 2025 (checked 1 Nov 2026) |
| 2 | Gemini 2.5 Pro | Google | 91.9 | overall accuracy % | Main split | Reported by the authors | Paper Table 3 | https://arxiv.org/html/2508.05201v2 | 24 Oct 2025 (checked 1 Nov 2026) |
| 3 | GPT-4.1 | OpenAI | 89.2 | overall accuracy % | Main split | Reported by the authors | Paper Table 3 | https://arxiv.org/html/2508.05201v2 | 24 Oct 2025 (checked 1 Nov 2026) |
| 4 | Gemini 2.5 Flash | Google | 88.7 | overall accuracy % | Main split | Reported by the authors | Paper Table 3 | https://arxiv.org/html/2508.05201v2 | 24 Oct 2025 (checked 1 Nov 2026) |
| 5 | GPT-4.1 mini | OpenAI | 88.2 | overall accuracy % | Main split | Reported by the authors | Paper Table 3 | https://arxiv.org/html/2508.05201v2 | 24 Oct 2025 (checked 1 Nov 2026) |
| 6 | Claude-haiku-3 | Anthropic | 81.3 | overall accuracy % | Main split | Reported by the authors | Paper Table 3 | https://arxiv.org/html/2508.05201v2 | 24 Oct 2025 (checked 1 Nov 2026) |
| 7 | Qwen3-32B | Qwen | 73.9 | overall accuracy % | Main split | Reported by the authors | Paper Table 3 | https://arxiv.org/html/2508.05201v2 | 24 Oct 2025 (checked 1 Nov 2026) |
| 8 | GPT-4.1-nano | OpenAI | 70 | overall accuracy % | Main split | Reported by the authors | Paper Table 3 | https://arxiv.org/html/2508.05201v2 | 24 Oct 2025 (checked 1 Nov 2026) |
| 9 | Gemini-2.5-flash-lite | Google | 50.2 | overall accuracy % | Main split | Reported by the authors | Paper Table 3 | https://arxiv.org/html/2508.05201v2 | 24 Oct 2025 (checked 1 Nov 2026) |
| 10 | Llama-3.1-8B | Meta | 47.5 | overall accuracy % | Main split | Reported by the authors | Paper Table 3 | https://arxiv.org/html/2508.05201v2 | 24 Oct 2025 (checked 1 Nov 2026) |

Scores are never combined across benchmarks. Each row names its metric, source and date.

## About this benchmark

- Maker: Asian Institute of Digital Finance, National University of Singapore
- Year: 2025
- Use case: Financial Verifiability
- Size: Main split: 2,406 answerable spans from 453 S&P 500 companies (2024 10-Ks). Pilot split: 300 spans from 9 companies.
- Public split: Pilot and main splits plus eval code on GitHub
- Grading: Accuracy on the masked span, with a precision-relaxed numeric match plus unit-group matching. Also broken down by reasoning scenario A-D.
- Licence: Apache-2.0 (GitHub repo, which says it holds code and data). The paper footnote says CC BY-NC-SA.
- Submission: none
- Products allowed: No
- Latest version: arXiv v2 (2025-10-24), ACM ICAIF'25
- What it does not cover: Covers only single-span numeric recovery from filings. No open-ended generation, citations or tool use.
- Last checked: 1 Nov 2026
- Links: paper https://arxiv.org/abs/2508.05201, data https://github.com/ZHANG-MENGAO/FAITH, code https://github.com/ZHANG-MENGAO/FAITH
- Directory page: https://benchmarks.finance/benchmarks/faith/
- Leaderboard: https://benchmarks.finance/leaderboards/faith/

Source: https://benchmarks.finance/leaderboards/faith/

Created and maintained by Moneda.