Leaderboard
FinTrust
Measures how trustworthy LLMs are in finance across seven dimensions: truthfulness, safety (jailbreaks over 10 financial-crime categories), fairness, robustness, privacy, transparency or fiduciary disclosure, and knowledge discovery.
Best reported
1
o4-mini
safety harmfulness score (1-10, GPT-4.1 mini judge)
17 Oct 2025
FinTrust arXiv v1 (2025-10-17), EMNLP 2025 Main
safety harmfulness score (1-10, GPT-4.1 mini judge), lower is better. 11 models.
Updated 17 Oct 2025
- o4-mini1
- XuanYuan-Fin-X1-Preview5.8
- Gemini 2.5 Flash6.2
- DianJin-R17.3
- GPT-4.17.4
- GPT-4.1-mini7.4
- Fin-R17.6
- Llama-4-Maverick-17B-128E-Instruct-FP87.6
- Qwen3-235B-A22B-FP88.5
- DeepSeek-R18.6
- DeepSeek-V39.0
FinTrust arXiv v1 (2025-10-17), EMNLP 2025 Main
safety harmfulness score (1-10, GPT-4.1 mini judge), lower is better. Top 10 of 11 models.
Updated 17 Oct 2025
- 1o4-mini1
- 2XuanYuan-Fin-X1-Preview5.8
- 3Gemini 2.5 Flash6.2
- 4DianJin-R17.3
- 5GPT-4.17.4
- 6GPT-4.1-mini7.4
- 7Fin-R17.6
- 8Llama-4-Maverick-17B-128E-Instruct-FP87.6
- 9Qwen3-235B-A22B-FP88.5
- 10DeepSeek-R18.6
| 1 | o4-mini | OpenAI | 1 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
|---|---|---|---|---|---|
| 2 | XuanYuan-Fin-X1-Preview | Not stated | 5.8 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 3 | Gemini 2.5 Flash | 6.2 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 | |
| 4 | DianJin-R1 | Not stated | 7.3 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 5 | GPT-4.1 | OpenAI | 7.4 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 6 | GPT-4.1-mini | OpenAI | 7.4 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 7 | Fin-R1 | Not stated | 7.6 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 8 | Llama-4-Maverick-17B-128E-Instruct-FP8 | Meta | 7.6 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 9 | Qwen3-235B-A22B-FP8 | Qwen | 8.5 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 10 | DeepSeek-R1 | DeepSeek | 8.6 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 11 | DeepSeek-V3 | DeepSeek | 9.0 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |