# FinLFQA leaderboard

Long-form answers to financial questions that compare two companies' 10-Qs. Answers must be attributed in three ways: supporting evidence from the filings, executable code for each calculation step, and the domain knowledge used.

## FinLFQA: evidence attribution F1 (higher is better, 8 models)

| Rank | Model | Maker | Score | Metric | Setting | Reported by | Source | URL | Date |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1 | GPT-4o | OpenAI | 56.5 | evidence attribution F1 | post-hoc attribution, test set | Reported by the authors | Paper Table 6 | https://arxiv.org/html/2510.06426v1 | 7 Oct 2025 |
| 2 | Mistral-Small-24B | Mistral AI | 55.3 | evidence attribution F1 | post-hoc attribution, test set | Reported by the authors | Paper Table 6 | https://arxiv.org/html/2510.06426v1 | 7 Oct 2025 |
| 3 | Llama-3.3-70B | Meta | 52.9 | evidence attribution F1 | post-hoc attribution, test set | Reported by the authors | Paper Table 6 | https://arxiv.org/html/2510.06426v1 | 7 Oct 2025 |
| 4 | Qwen2.5-72B | Qwen | 46.6 | evidence attribution F1 | post-hoc attribution, test set | Reported by the authors | Paper Table 6 | https://arxiv.org/html/2510.06426v1 | 7 Oct 2025 |
| 5 | phi-4 | Microsoft | 46.1 | evidence attribution F1 | post-hoc attribution, test set | Reported by the authors | Paper Table 6 | https://arxiv.org/html/2510.06426v1 | 7 Oct 2025 |
| 6 | Mistral-8x22B | Mistral AI | 42.9 | evidence attribution F1 | post-hoc attribution, test set | Reported by the authors | Paper Table 6 | https://arxiv.org/html/2510.06426v1 | 7 Oct 2025 |
| 7 | Llama-3.2-3B | Meta | 11.8 | evidence attribution F1 | post-hoc attribution, test set | Reported by the authors | Paper Table 6 | https://arxiv.org/html/2510.06426v1 | 7 Oct 2025 |
| 8 | Llama-3.2-1B | Meta | 1.5 | evidence attribution F1 | post-hoc attribution, test set | Reported by the authors | Paper Table 6 | https://arxiv.org/html/2510.06426v1 | 7 Oct 2025 |

## FinLFQA: LLM-as-judge answer score (/15) (higher is better, 3 models)

| Rank | Model | Maker | Score | Metric | Setting | Reported by | Source | URL | Date |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1 | GPT-4o | OpenAI | 13.5 | LLM-as-judge answer score (/15) | post-hoc attribution, test set | Reported by the authors | Paper Table 6 | https://arxiv.org/html/2510.06426v1 | 7 Oct 2025 |
| 2 | Llama-3.3-70B | Meta | 12.9 | LLM-as-judge answer score (/15) | post-hoc attribution, test set | Reported by the authors | Paper Table 6 | https://arxiv.org/html/2510.06426v1 | 7 Oct 2025 |
| 2 | Qwen2.5-72B | Qwen | 12.9 | LLM-as-judge answer score (/15) | post-hoc attribution, test set | Reported by the authors | Paper Table 6 | https://arxiv.org/html/2510.06426v1 | 7 Oct 2025 |

Scores are never combined across benchmarks. Each row names its metric, source and date.

## About this benchmark

- Maker: New York University; Yale University; NYU Shanghai
- Year: 2025
- Use case: Financial Verifiability
- Size: 1,008 expert-annotated examples (302 dev, 706 test), 89 companies
- Public split: Dev and test splits on Hugging Face, plus code on GitHub. The paper says test answers are kept private, which was not checked against the HF files.
- Grading: Answer quality: ROUGE-L, BERTScore, a GPT-4o judge (out of 15) and numeric F1. Attribution: evidence precision/recall/F1, code-execution success and knowledge recall.
- Licence: MIT (Hugging Face dataset card). The GitHub repo declares no licence.
- Submission: Not stated
- Products allowed: No
- Latest version: arXiv v1 (2025-10-07), Findings of EMNLP 2025
- What it does not cover: Attribution is to supplied filings, not open-web citation. Uses US 10-Q pairs only.
- Last checked: 1 Nov 2026
- Links: paper https://arxiv.org/abs/2510.06426, data https://huggingface.co/datasets/Dragongon/FinLFQA, code https://github.com/yitaoLong/FinLFQA
- Directory page: https://benchmarks.finance/benchmarks/finlfqa/
- Leaderboard: https://benchmarks.finance/leaderboards/finlfqa/

Source: https://benchmarks.finance/leaderboards/finlfqa/

Created and maintained by Moneda.