Mercor, in partnership with Ramp, 2026
APEX-Accounting
Whether frontier models can do professional month-end close and bookkeeping work in simulated companies: reconciling accounts, accruing expenses, posting transactions and producing reports, inside an accounting system with spreadsheets, PDFs and other files.
About this benchmark
- Size
- Leaderboard page: 10 worlds, 100 tasks, 1,475 grading criteria. Paper: 160 private tasks across 10 worlds
- Public split
- One open world of 10 tasks (prompts, rubrics, files) on Hugging Face; the full task set stays private
- Grading
- Expert-written binary rubric criteria (avg 13.7 per task) judged by an LM judge (DeepSeek-v4-Flash in the paper). Board headline is Mean Score = average % of criteria passed; Pass@1, Pass@8 and Pass^8 are also reported
- Licence
- Not stated
- Submission
- via-maker
- Products allowed
- Not stated
- Latest version
- Paper v2 (2026-07-30); leaderboard live, no update date shown
- What it does not cover
- Simulated companies only; Ramp's own accounting product was explicitly not tested; no audit or tax work
Results
10 modelsEach score shows who reported it: the benchmark's authors (quoted from their paper), the benchmark's official board, or Moneda (run with the benchmark's own grader and rules). We do not copy other aggregators.
APEX-Accounting Paper v2 (2026-07-30); leaderboard live, no update date shown
Mean Score (% criteria passed), higher is better. 10 models.
Updated 1 Nov 2026
- Gemini 4 Argon71.5
- Gemini 4 Carbon70.7
- Muse Wonder 1.067.1
- Fable 5.166.7
- Opus 5.565.5
- Muse Spark 1.364.9
- GPT-6 Astra64.7
- Sonnet 5.564.5
- GPT-6.1 Sol64.3
- Opus 564.3
APEX-Accounting Paper v2 (2026-07-30); leaderboard live, no update date shown
Mean Score (% criteria passed), higher is better. 10 models.
Updated 1 Nov 2026
- 1Gemini 4 Argon71.5
- 2Gemini 4 Carbon70.7
- 3Muse Wonder 1.067.1
- 4Fable 5.166.7
- 5Opus 5.565.5
- 6Muse Spark 1.364.9
- 7GPT-6 Astra64.7
- 8Sonnet 5.564.5
- 9GPT-6.1 Sol64.3
- 9Opus 564.3
| 1 | Gemini 4 Argon | 71.5 | High effort, Loop harness | Mercor APEX-Accounting leaderboard Official board, 1 Nov 2026 | |
|---|---|---|---|---|---|
| 2 | Gemini 4 Carbon | 70.7 | High effort, Loop harness | Mercor APEX-Accounting leaderboard Official board, 1 Nov 2026 | |
| 3 | Muse Wonder 1.0 | Meta | 67.1 | Max effort, Loop harness | Mercor APEX-Accounting leaderboard Official board, 1 Nov 2026 |
| 4 | Fable 5.1 | Anthropic | 66.7 | Max effort, Loop harness | Mercor APEX-Accounting leaderboard Official board, 1 Nov 2026 |
| 5 | Opus 5.5 | Anthropic | 65.5 | Max effort, Loop harness | Mercor APEX-Accounting leaderboard Official board, 1 Nov 2026 |
| 6 | Muse Spark 1.3 | Meta | 64.9 | Max effort, Loop harness | Mercor APEX-Accounting leaderboard Official board, 1 Nov 2026 |
| 7 | GPT-6 Astra | OpenAI | 64.7 | Max effort, Loop harness | Mercor APEX-Accounting leaderboard Official board, 1 Nov 2026 |
| 8 | Sonnet 5.5 | Anthropic | 64.5 | Max effort, Loop harness | Mercor APEX-Accounting leaderboard Official board, 1 Nov 2026 |
| 9 | GPT-6.1 Sol | OpenAI | 64.3 | Max effort, Loop harness | Mercor APEX-Accounting leaderboard Official board, 1 Nov 2026 |
| 9 | Opus 5 | Anthropic | 64.3 | Max effort, Loop harness | Mercor APEX-Accounting leaderboard Official board, 1 Nov 2026 |
Papers and links
Last checked 1 Nov 2026. Something out of date or wrong? Send a correction.