# FinSearchComp

Expert-written financial search-and-reasoning questions in three tiers: time-sensitive data fetching, simple historical lookup and complex historical investigation. There are Global and Greater China subsets.

## Facts

- Maker: ByteDance Seed (Columbia Business School listed as advisor)
- Year: 2025
- Use case: Live financial market data
- Size: 635 questions, 3 tasks, 2 market subsets
- Public split: The paper calls it fully open-source and the dataset is on Hugging Face. The paper does not state a held-out split.
- Grading: An LLM judge scores 0/1 against expert rubrics. Time-sensitive answers are checked against real-time API data within the day's high-low band. The paper reports about 95% judge-human agreement.
- Licence: CC-BY-4.0 (data, per Hugging Face card)
- Submission: Not stated
- Products allowed: Yes
- Latest version: arXiv v1 (2025-09-16); HF dataset last modified 2025-10-27
- What it does not cover: Does not test filing analysis depth, portfolio decisions or advice quality. Time-sensitive answers depend on the day they were run.
- Last checked: 1 Nov 2026
- Links: paper https://arxiv.org/abs/2509.13160, site https://randomtutu.github.io/FinSearchComp/, data https://huggingface.co/datasets/ByteSeedXpert/FinSearchComp, leaderboard https://xpert.bytedance.com/leaderboard?id=FinSearchComp
- Directory page: https://benchmarks.finance/benchmarks/finsearchcomp/
- Leaderboard: https://benchmarks.finance/leaderboards/finsearchcomp/

## Results

### FinSearchComp: Global subset average accuracy % (higher is better, 11 models)

| Rank | Model | Provider | Score | Metric | Setting | Reported by | Source | URL | Date |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1 | Grok 4 | xAI | 68.9 | Global subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 2 | GPT-5-Thinking | OpenAI | 63.9 | Global subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 3 | Numa 1 | Moneda | 59.4 | Global subset average accuracy % | Product with tools; historical-lookup tiers only (time-sensitive tier not run); two-judge panel | Run by Moneda | Moneda run, 2026-11-01 | https://benchmarks.finance/method/ | 1 Nov 2026 |
| 4 | Gemini 2.5 Pro | Google | 42.6 | Global subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 5 | DouBao | ByteDance | 39.1 | Global subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 6 | Gemini2.5-pro (Google Search) | Google | 38.2 | Global subset average accuracy % | API with search | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 7 | Qwen3-235B-A22B-2507 | Qwen | 37.4 | Global subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 8 | YuanBao-DeepSeek-V3 | Not stated | 30.5 | Global subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 9 | DouBao-Thinking | ByteDance | 29.8 | Global subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 9 | HunYuan-T1-Thinking | Tencent | 29.8 | Global subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 9 | YuanBao-DeepSeek-R1 | Not stated | 29.8 | Global subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |

### FinSearchComp: Greater China subset average accuracy % (higher is better, 11 models)

| Rank | Model | Provider | Score | Metric | Setting | Reported by | Source | URL | Date |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1 | Numa 1 | Moneda | 57.1 | Greater China subset average accuracy % | Product with tools; historical-lookup tiers only (time-sensitive tier not run); two-judge panel | Run by Moneda | Moneda run, 2026-11-01 | https://benchmarks.finance/method/ | 1 Nov 2026 |
| 2 | DouBao | ByteDance | 54.2 | Greater China subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 3 | YuanBao-DeepSeek-R1 | Not stated | 52.5 | Greater China subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 4 | Grok 4 | xAI | 51.9 | Greater China subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 5 | HunYuan-T1-Thinking | Tencent | 50.5 | Greater China subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 6 | DouBao-Thinking | ByteDance | 49 | Greater China subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 7 | YuanBao-DeepSeek-V3 | Not stated | 48.8 | Greater China subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 8 | GPT-5-Thinking | OpenAI | 46.4 | Greater China subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 9 | ERNIE-X1 | Baidu | 40.8 | Greater China subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 10 | DeepSeek-R1 | DeepSeek | 40.5 | Greater China subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |
| 11 | Kimi K2 | Moonshot AI | 38.3 | Greater China subset average accuracy % | web product | Reported by the authors | Paper Table 5 | https://arxiv.org/html/2509.13160v1 | 16 Sept 2025 (checked 1 Nov 2026) |


Source: https://benchmarks.finance/benchmarks/finsearchcomp/

Created and maintained by Moneda.