BlueFin

Size

131 tasks, 3,225 rubric criteria

Public split

11 public tasks; 120 held out for the main evaluation

Grading

LM judge agent (GPT-5.4, high reasoning) scores weighted rubric criteria; score = points met / max points (0-100), non-completion = 0; judge validated against experts (Krippendorff alpha 0.826)

Licence

CC-BY-NC-4.0 (public data, HF); paper CC BY-NC-SA 4.0

Submission

none

Products allowed

Not stated

Latest version

arXiv v1 (2026-05-29)

What it does not cover

No leaderboard; only 5 frontier models scored; no accounting close or live data

Each score shows who reported it: the benchmark's authors (quoted from their paper), the benchmark's official board, or Moneda (run with the benchmark's own grader and rules). We do not copy other aggregators.

1. 49.6 GPT-5.549.6
2. 49.2Opus 4.749.2
3. 45.9Gemini 3.1 Pro45.9
4. 42.9Sonnet 4.642.9
5. 31.3 Grok 4.2031.3

1. 1 GPT-5.549.649.6
2. 2Opus 4.749.249.2
3. 3Gemini 3.1 Pro45.945.9
4. 4Sonnet 4.642.942.9
5. 5 Grok 4.2031.331.3

BlueFin leaderboard (criteria-weighted score % (overall), higher is better)

Rank	Model	Provider	criteria-weighted score % (overall)	Setting	Source
1	GPT-5.5	OpenAI	49.6	held-out set, 120 tasks	Paper Table 2 Reported by the authors, reported May 2026 · checked 1 Nov 2026
2	Opus 4.7	Anthropic	49.2	held-out set, 120 tasks	Paper Table 2 Reported by the authors, reported May 2026 · checked 1 Nov 2026
3	Gemini 3.1 Pro	Google	45.9	held-out set, 120 tasks	Paper Table 2 Reported by the authors, reported May 2026 · checked 1 Nov 2026
4	Sonnet 4.6	Anthropic	42.9	held-out set, 120 tasks	Paper Table 2 Reported by the authors, reported May 2026 · checked 1 Nov 2026
5	Grok 4.20	xAI	31.3	held-out set, 120 tasks	Paper Table 2 Reported by the authors, reported May 2026 · checked 1 Nov 2026

• Paper
• Data
• Code

Last checked 1 Nov 2026. Something out of date or wrong? Send a correction.