The Fin AI, 2026
Herculean
Tests AI agents on four end-to-end finance workflows (Trading, Hedging, Market Insights and Auditing) run as MCP-based tool environments.
About this benchmark
- Size
- 424 offline and 246 live instances across 4 workflows; 30 agent and model pairs, 3 runs each
- Public split
- Offline replay data (Dec 2024 to Mar 2026) on Hugging Face
- Grading
- Trades and hedges are replayed against realised prices (return, Sharpe, drawdown); reports are scored by an LLM judge against rubrics; audits by deterministic checks.
- Licence
- CC BY 4.0
- Submission
- No public leaderboard or submission process. You run the open-source code yourself.
- Products allowed
- Agent frameworks paired with model backbones
- Latest version
- v4 (arXiv, 2026-10-19)
- What it does not cover
- Retail banking, payments, tax and personal advice.
Results
No scores recorded here yet.
Papers and links
Last checked 6 Nov 2026. Something out of date or wrong? Send a correction.