# BlueFin

LLM agents on professional-finance spreadsheet work: building new workbooks (synthesis), editing existing ones (manipulation), and answering questions about them (interrogation/comprehension). It puts particular weight on 'dynamic correctness', meaning formulas that still work when inputs change.

## Facts

- Maker: Longitude Labs Inc., Carnegie Mellon University, Cornell University
- Year: 2026
- Use case: Financial spreadsheets and accounting
- Size: 131 tasks, 3,225 rubric criteria
- Public split: 11 public tasks; 120 held out for the main evaluation
- Grading: LM judge agent (GPT-5.4, high reasoning) scores weighted rubric criteria; score = points met / max points (0-100), non-completion = 0; judge validated against experts (Krippendorff alpha 0.826)
- Licence: CC-BY-NC-4.0 (public data, HF); paper CC BY-NC-SA 4.0
- Submission: none
- Products allowed: Not stated
- Latest version: arXiv v1 (2026-05-29)
- What it does not cover: No leaderboard; only 5 frontier models scored; no accounting close or live data
- Last checked: 1 Nov 2026
- Links: paper https://arxiv.org/abs/2605.30907, data https://huggingface.co/datasets/Longitude-Labs/bluefin-release, code https://github.com/Longitude-Labs/bluefin
- Directory page: https://benchmarks.finance/benchmarks/bluefin/
- Leaderboard: https://benchmarks.finance/leaderboards/bluefin/

## Results

### BlueFin: criteria-weighted score % (overall) (higher is better, 5 models)

| Rank | Model | Provider | Score | Metric | Setting | Reported by | Source | URL | Date |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1 | GPT-5.5 | OpenAI | 49.6 | criteria-weighted score % (overall) | held-out set, 120 tasks | Reported by the authors | Paper Table 2 | https://arxiv.org/html/2605.30907v1 | May 2026 (checked 1 Nov 2026) |
| 2 | Opus 4.7 | Anthropic | 49.2 | criteria-weighted score % (overall) | held-out set, 120 tasks | Reported by the authors | Paper Table 2 | https://arxiv.org/html/2605.30907v1 | May 2026 (checked 1 Nov 2026) |
| 3 | Gemini 3.1 Pro | Google | 45.9 | criteria-weighted score % (overall) | held-out set, 120 tasks | Reported by the authors | Paper Table 2 | https://arxiv.org/html/2605.30907v1 | May 2026 (checked 1 Nov 2026) |
| 4 | Sonnet 4.6 | Anthropic | 42.9 | criteria-weighted score % (overall) | held-out set, 120 tasks | Reported by the authors | Paper Table 2 | https://arxiv.org/html/2605.30907v1 | May 2026 (checked 1 Nov 2026) |
| 5 | Grok 4.20 | xAI | 31.3 | criteria-weighted score % (overall) | held-out set, 120 tasks | Reported by the authors | Paper Table 2 | https://arxiv.org/html/2605.30907v1 | May 2026 (checked 1 Nov 2026) |


Source: https://benchmarks.finance/benchmarks/bluefin/

Created and maintained by Moneda.