# FinMCP-Bench

Measures how well LLM agents call real financial MCP tools in single-tool, multi-tool and multi-turn tasks.

## Facts

- Maker: Qwen DianJin Team (Alibaba Cloud)
- Year: 2026
- Use case: Analyst Financial research
- Size: 613 samples over 65 real financial MCP tools; 6 models
- Public split: none found
- Grading: Automatic. Predicted tool calls are compared with reference tool calls using Tool Precision, Tool Recall, Tool F1 and Exact Match Rate. Final answers are not graded.
- Licence: Not stated
- Submission: There is no leaderboard or submission process. Results appear only in the paper.
- Products allowed: Models only
- Latest version: v1 (arXiv, 2026-03-26)
- What it does not cover: Answer quality and live trades: only tool calls are scored.
- Last checked: 6 Nov 2026
- Links: paper https://arxiv.org/abs/2603.24943
- Directory page: https://benchmarks.finance/benchmarks/finmcp-bench/
- Leaderboard: https://benchmarks.finance/leaderboards/finmcp-bench/

## Results

### FinMCP-Bench: TF1 (%), All (higher is better, 6 models)

| Rank | Model | Provider | Score | Metric | Setting | Reported by | Source | URL | Date |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1 | Qwen3-235B-A22B-Thinking | Qwen | 64.3 | TF1 (%), All | All samples (single-tool, multi-tool, multi-turn), gold conversation history | Reported by the authors | Paper Table 2 | https://arxiv.org/html/2603.24943v1 | 26 Mar 2026 (checked 6 Nov 2026) |
| 2 | Qwen3-30B-A3B-Thinking | Qwen | 55.6 | TF1 (%), All | All samples (single-tool, multi-tool, multi-turn), gold conversation history | Reported by the authors | Paper Table 2 | https://arxiv.org/html/2603.24943v1 | 26 Mar 2026 (checked 6 Nov 2026) |
| 3 | Qwen3-4B-Thinking | Qwen | 50.1 | TF1 (%), All | All samples (single-tool, multi-tool, multi-turn), gold conversation history | Reported by the authors | Paper Table 2 | https://arxiv.org/html/2603.24943v1 | 26 Mar 2026 (checked 6 Nov 2026) |
| 4 | DeepSeek-R1 | DeepSeek | 49.9 | TF1 (%), All | All samples (single-tool, multi-tool, multi-turn), gold conversation history | Reported by the authors | Paper Table 2 | https://arxiv.org/html/2603.24943v1 | 26 Mar 2026 (checked 6 Nov 2026) |
| 5 | Seed-OSS-36B | Not stated | 39.3 | TF1 (%), All | All samples (single-tool, multi-tool, multi-turn), gold conversation history | Reported by the authors | Paper Table 2 | https://arxiv.org/html/2603.24943v1 | 26 Mar 2026 (checked 6 Nov 2026) |
| 6 | GPT-OSS-20B | OpenAI | 32.6 | TF1 (%), All | All samples (single-tool, multi-tool, multi-turn), gold conversation history | Reported by the authors | Paper Table 2 | https://arxiv.org/html/2603.24943v1 | 26 Mar 2026 (checked 6 Nov 2026) |


Source: https://benchmarks.finance/benchmarks/finmcp-bench/

Created and maintained by Moneda.