# FinTrust leaderboard

Measures how trustworthy LLMs are in finance across seven dimensions: truthfulness, safety (jailbreaks over 10 financial-crime categories), fairness, robustness, privacy, transparency or fiduciary disclosure, and knowledge discovery.

## FinTrust: safety harmfulness score (1-10, GPT-4.1 mini judge) (lower is better, 11 models)

| Rank | Model | Maker | Score | Metric | Setting | Reported by | Source | URL | Date |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1 | o4-mini | OpenAI | 1 | safety harmfulness score (1-10, GPT-4.1 mini judge) | genetic-algorithm jailbreak | Reported by the authors | Paper Table 7 (App. A.2) | https://arxiv.org/html/2510.15232v1 | 17 Oct 2025 |
| 2 | XuanYuan-Fin-X1-Preview | Not stated | 5.8 | safety harmfulness score (1-10, GPT-4.1 mini judge) | genetic-algorithm jailbreak | Reported by the authors | Paper Table 7 (App. A.2) | https://arxiv.org/html/2510.15232v1 | 17 Oct 2025 |
| 3 | Gemini 2.5 Flash | Google | 6.2 | safety harmfulness score (1-10, GPT-4.1 mini judge) | genetic-algorithm jailbreak | Reported by the authors | Paper Table 7 (App. A.2) | https://arxiv.org/html/2510.15232v1 | 17 Oct 2025 |
| 4 | DianJin-R1 | Not stated | 7.3 | safety harmfulness score (1-10, GPT-4.1 mini judge) | genetic-algorithm jailbreak | Reported by the authors | Paper Table 7 (App. A.2) | https://arxiv.org/html/2510.15232v1 | 17 Oct 2025 |
| 5 | GPT-4.1 | OpenAI | 7.4 | safety harmfulness score (1-10, GPT-4.1 mini judge) | genetic-algorithm jailbreak | Reported by the authors | Paper Table 7 (App. A.2) | https://arxiv.org/html/2510.15232v1 | 17 Oct 2025 |
| 6 | GPT-4.1-mini | OpenAI | 7.4 | safety harmfulness score (1-10, GPT-4.1 mini judge) | genetic-algorithm jailbreak | Reported by the authors | Paper Table 7 (App. A.2) | https://arxiv.org/html/2510.15232v1 | 17 Oct 2025 |
| 7 | Fin-R1 | Not stated | 7.6 | safety harmfulness score (1-10, GPT-4.1 mini judge) | genetic-algorithm jailbreak | Reported by the authors | Paper Table 7 (App. A.2) | https://arxiv.org/html/2510.15232v1 | 17 Oct 2025 |
| 8 | Llama-4-Maverick-17B-128E-Instruct-FP8 | Meta | 7.6 | safety harmfulness score (1-10, GPT-4.1 mini judge) | genetic-algorithm jailbreak | Reported by the authors | Paper Table 7 (App. A.2) | https://arxiv.org/html/2510.15232v1 | 17 Oct 2025 |
| 9 | Qwen3-235B-A22B-FP8 | Qwen | 8.5 | safety harmfulness score (1-10, GPT-4.1 mini judge) | genetic-algorithm jailbreak | Reported by the authors | Paper Table 7 (App. A.2) | https://arxiv.org/html/2510.15232v1 | 17 Oct 2025 |
| 10 | DeepSeek-R1 | DeepSeek | 8.6 | safety harmfulness score (1-10, GPT-4.1 mini judge) | genetic-algorithm jailbreak | Reported by the authors | Paper Table 7 (App. A.2) | https://arxiv.org/html/2510.15232v1 | 17 Oct 2025 |
| 11 | DeepSeek-V3 | DeepSeek | 9.0 | safety harmfulness score (1-10, GPT-4.1 mini judge) | genetic-algorithm jailbreak | Reported by the authors | Paper Table 7 (App. A.2) | https://arxiv.org/html/2510.15232v1 | 17 Oct 2025 |

Scores are never combined across benchmarks. Each row names its metric, source and date.

## About this benchmark

- Maker: NYU Shanghai; National University of Singapore; Yale University; NYU Center for Data Science
- Year: 2025
- Use case: Financial advice and safety
- Size: 15,680 instances across 7 dimensions
- Public split: Full dataset on Hugging Face. The GitHub repo has per-dimension scripts and 100-case samples.
- Grading: Per dimension. GPT-4.1 mini acts as judge (safety 1-10, informativeness 1-5 and others), with rule-based scoring for hallucination and certainty and Cohen's d for fairness. There is no aggregate score.
- Licence: MIT (Hugging Face dataset card). The GitHub repo declares no licence.
- Submission: none
- Products allowed: No
- Latest version: arXiv v1 (2025-10-17), EMNLP 2025 Main
- What it does not cover: No single headline score and no agentic or tool-using evaluation. Evaluates bare models only.
- Last checked: 1 Nov 2026
- Links: paper https://arxiv.org/abs/2510.15232, data https://huggingface.co/datasets/HughieHu/FinTrust, code https://github.com/HughieHu/FinTrust/
- Directory page: https://benchmarks.finance/benchmarks/fintrust/
- Leaderboard: https://benchmarks.finance/leaderboards/fintrust/

Source: https://benchmarks.finance/leaderboards/fintrust/

Created and maintained by Moneda.