Leaderboard
FinLFQA
Long-form answers to financial questions that compare two companies' 10-Qs. Answers must be attributed in three ways: supporting evidence from the filings, executable code for each calculation step, and the domain knowledge used.
Best reported
56.5
GPT-4o
evidence attribution F1
7 Oct 2025
FinLFQA arXiv v1 (2025-10-07), Findings of EMNLP 2025
evidence attribution F1, higher is better. 8 models.
Updated 7 Oct 2025
- GPT-4o56.5
- Mistral-Small-24B55.3
- Llama-3.3-70B52.9
- Qwen2.5-72B46.6
- phi-446.1
- Mistral-8x22B42.9
- Llama-3.2-3B11.8
- Llama-3.2-1B1.5
FinLFQA arXiv v1 (2025-10-07), Findings of EMNLP 2025
evidence attribution F1, higher is better. 8 models.
Updated 7 Oct 2025
- 1GPT-4o56.5
- 2Mistral-Small-24B55.3
- 3Llama-3.3-70B52.9
- 4Qwen2.5-72B46.6
- 5phi-446.1
- 6Mistral-8x22B42.9
- 7Llama-3.2-3B11.8
- 8Llama-3.2-1B1.5
| 1 | GPT-4o | OpenAI | 56.5 | post-hoc attribution, test set | Paper Table 6 Reported by the authors, 7 Oct 2025 |
|---|---|---|---|---|---|
| 2 | Mistral-Small-24B | Mistral AI | 55.3 | post-hoc attribution, test set | Paper Table 6 Reported by the authors, 7 Oct 2025 |
| 3 | Llama-3.3-70B | Meta | 52.9 | post-hoc attribution, test set | Paper Table 6 Reported by the authors, 7 Oct 2025 |
| 4 | Qwen2.5-72B | Qwen | 46.6 | post-hoc attribution, test set | Paper Table 6 Reported by the authors, 7 Oct 2025 |
| 5 | phi-4 | Microsoft | 46.1 | post-hoc attribution, test set | Paper Table 6 Reported by the authors, 7 Oct 2025 |
| 6 | Mistral-8x22B | Mistral AI | 42.9 | post-hoc attribution, test set | Paper Table 6 Reported by the authors, 7 Oct 2025 |
| 7 | Llama-3.2-3B | Meta | 11.8 | post-hoc attribution, test set | Paper Table 6 Reported by the authors, 7 Oct 2025 |
| 8 | Llama-3.2-1B | Meta | 1.5 | post-hoc attribution, test set | Paper Table 6 Reported by the authors, 7 Oct 2025 |
FinLFQA arXiv v1 (2025-10-07), Findings of EMNLP 2025
LLM-as-judge answer score (/15), higher is better. 3 models.
Updated 7 Oct 2025
- GPT-4o13.5
- Llama-3.3-70B12.9
- Qwen2.5-72B12.9
FinLFQA arXiv v1 (2025-10-07), Findings of EMNLP 2025
LLM-as-judge answer score (/15), higher is better. 3 models.
Updated 7 Oct 2025
- 1GPT-4o13.5
- 2Llama-3.3-70B12.9
- 2Qwen2.5-72B12.9
| 1 | GPT-4o | OpenAI | 13.5 | post-hoc attribution, test set | Paper Table 6 Reported by the authors, 7 Oct 2025 |
|---|---|---|---|---|---|
| 2 | Llama-3.3-70B | Meta | 12.9 | post-hoc attribution, test set | Paper Table 6 Reported by the authors, 7 Oct 2025 |
| 2 | Qwen2.5-72B | Qwen | 12.9 | post-hoc attribution, test set | Paper Table 6 Reported by the authors, 7 Oct 2025 |