Leaderboard
FAITH
Measures intrinsic hallucination on financial tables. A numeric span is masked in a sentence from a 10-K and the model must recover it from the report's tables and surrounding context.
FAITH arXiv v2 (2025-10-24), ACM ICAIF'25
overall accuracy %, higher is better. 10 models.
Updated 24 Oct 2025
- Claude Sonnet 495.6
- Gemini 2.5 Pro91.9
- GPT-4.189.2
- Gemini 2.5 Flash88.7
- GPT-4.1-mini88.2
- Claude-haiku-381.3
- Qwen-3-32B73.9
- GPT-4.1-nano70
- Gemini-2.5-flash-lite50.2
- Llama-3.1-8B47.5
FAITH arXiv v2 (2025-10-24), ACM ICAIF'25
overall accuracy %, higher is better. 10 models.
Updated 24 Oct 2025
- 1Claude Sonnet 495.6
- 2Gemini 2.5 Pro91.9
- 3GPT-4.189.2
- 4Gemini 2.5 Flash88.7
- 5GPT-4.1-mini88.2
- 6Claude-haiku-381.3
- 7Qwen-3-32B73.9
- 8GPT-4.1-nano70
- 9Gemini-2.5-flash-lite50.2
- 10Llama-3.1-8B47.5
| 1 | Claude Sonnet 4 | Anthropic | 95.6 | Main split | Paper Table 3 Reported by the authors, 24 Oct 2025 |
|---|---|---|---|---|---|
| 2 | Gemini 2.5 Pro | 91.9 | Main split | Paper Table 3 Reported by the authors, 24 Oct 2025 | |
| 3 | GPT-4.1 | OpenAI | 89.2 | Main split | Paper Table 3 Reported by the authors, 24 Oct 2025 |
| 4 | Gemini 2.5 Flash | 88.7 | Main split | Paper Table 3 Reported by the authors, 24 Oct 2025 | |
| 5 | GPT-4.1-mini | OpenAI | 88.2 | Main split | Paper Table 3 Reported by the authors, 24 Oct 2025 |
| 6 | Claude-haiku-3 | Anthropic | 81.3 | Main split | Paper Table 3 Reported by the authors, 24 Oct 2025 |
| 7 | Qwen-3-32B | Qwen | 73.9 | Main split | Paper Table 3 Reported by the authors, 24 Oct 2025 |
| 8 | GPT-4.1-nano | OpenAI | 70 | Main split | Paper Table 3 Reported by the authors, 24 Oct 2025 |
| 9 | Gemini-2.5-flash-lite | 50.2 | Main split | Paper Table 3 Reported by the authors, 24 Oct 2025 | |
| 10 | Llama-3.1-8B | Meta | 47.5 | Main split | Paper Table 3 Reported by the authors, 24 Oct 2025 |