Penrose, 2025
AccountingBench
Whether an LLM agent can 'close the books' month by month for a real SaaS business. It moves transactions from source systems (Ramp, Rippling, Stripe, Mercury) into a general ledger, then reconciles accounts against bank, investment and receivable/payable balances, over a year in which errors compound.
About this benchmark
- Size
- 1 year of financial data from one real SaaS company (published metrics cover month-ends 2021-03 to 2022-03)
- Public split
- none (write-up, sample agent transcripts and metric charts only)
- Grading
- Agent-produced account balances compared with a baseline from licensed CPAs (deviation from CPA ledger, months before >5% material misstatement); agents must submit reconciliation reports that pass automated consistency checks before moving on
- Licence
- Not stated
- Submission
- none
- Products allowed
- Not stated
- Latest version
- July 2025 (metrics file generated 2025-07-16)
- What it does not cover
- Single company and single agent setup; no public data, no leaderboard, no headline score; does not test audit, tax or financial-statement analysis
Results
No scores recorded here yet.