Best AI for Advice and safety (November 2026)
Data refreshed 1 Nov 2026
o4-mini leads FinTrust with 1 (safety harmfulness score (1-10, GPT-4.1 mini judge)). LLaMA-3 leads FinSafetyBench with 32.3 (attack success rate % (financial crimes, English)).
- Benchmarks in this use case
- 4
- With dated scores
- 2
- Models with scores
- 17
- Latest source date
- 1 May 2026
Advice and safety picks
The leader on each benchmark, never combinedBest on FinTrust
o4-miniOpenAI
1 safety harmfulness score (1-10, GPT-4.1 mini judge)
Reported by the authors, 17 Oct 2025
Best on FinSafetyBench
LLaMA-3Meta
32.3 attack success rate % (financial crimes, English)
Reported by the authors, 1 May 2026
Advice and safety leaderboard
FinTrust arXiv v1 (2025-10-17), EMNLP 2025 Main
safety harmfulness score (1-10, GPT-4.1 mini judge), lower is better. Top 5 of 11 models.
Updated 17 Oct 2025
- 1o4-mini1
- 2XuanYuan-Fin-X1-Preview5.8
- 3Gemini 2.5 Flash6.2
- 4DianJin-R17.3
- 5GPT-4.17.4
| 1 | o4-mini | OpenAI | 1 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
|---|---|---|---|---|---|
| 2 | XuanYuan-Fin-X1-Preview | Not stated | 5.8 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 3 | Gemini 2.5 Flash | 6.2 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 | |
| 4 | DianJin-R1 | Not stated | 7.3 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 5 | GPT-4.1 | OpenAI | 7.4 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 6 | GPT-4.1-mini | OpenAI | 7.4 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 7 | Fin-R1 | Not stated | 7.6 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 8 | Llama-4-Maverick-17B-128E-Instruct-FP8 | Meta | 7.6 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 9 | Qwen3-235B-A22B-FP8 | Qwen | 8.5 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 10 | DeepSeek-R1 | DeepSeek | 8.6 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
| 11 | DeepSeek-V3 | DeepSeek | 9.0 | genetic-algorithm jailbreak | Paper Table 7 (App. A.2) Reported by the authors, 17 Oct 2025 |
FinSafetyBench arXiv v1 (2026-05-01), Findings of ACL 2026
attack success rate % (financial crimes, English), lower is better. Top 5 of 6 models.
Updated 1 May 2026
- 1LLaMA-332.3
- 2XuanYuan57.5
- 3InternLM372.2
- 4GLM-472.3
- 5Qwen2.588.2
| 1 | LLaMA-3 | Meta | 32.3 | Meta-Llama-3-8B-Instruct; average of PAIR, ReNeLLM, FlipAttack | Paper Table 2 Reported by the authors, 1 May 2026 |
|---|---|---|---|---|---|
| 2 | XuanYuan | Not stated | 57.5 | XuanYuan-13B-Chat; average of PAIR, ReNeLLM, FlipAttack | Paper Table 2 Reported by the authors, 1 May 2026 |
| 3 | InternLM3 | Not stated | 72.2 | InternLM3-8B-Instruct; average of PAIR, ReNeLLM, FlipAttack | Paper Table 2 Reported by the authors, 1 May 2026 |
| 4 | GLM-4 | Zhipu AI | 72.3 | GLM-4-9B-0414; average of PAIR, ReNeLLM, FlipAttack | Paper Table 2 Reported by the authors, 1 May 2026 |
| 5 | Qwen2.5 | Qwen | 88.2 | Qwen2.5-32B-Instruct; average of PAIR, ReNeLLM, FlipAttack | Paper Table 2 Reported by the authors, 1 May 2026 |
| 6 | Mistral | Mistral AI | 93.5 | Mistral-Small-24B-Instruct-2501; average of PAIR, ReNeLLM, FlipAttack | Paper Table 2 Reported by the authors, 1 May 2026 |