# ClaimBench

Checks a claim against a 10-K excerpt and labels it supported, contradicted or insufficient_evidence. Excerpts range from short to about 16k tokens.

## Facts

- Maker: Vals AI (United States)
- Year: Not stated
- Use case: Financial Verifiability
- Size: 1,960 claims in 5 categories (392 supported, 785 contradicted, 783 insufficient_evidence)
- Public split: 10 public examples
- Grading: Accuracy against gold labels that come from the construction rules, not from a model. The page does not say whether scoring is done by code or by an LLM judge.
- Licence: Not stated
- Submission: Not stated
- Products allowed: Not stated
- Latest version: v1 (updated 2026-10-23)
- What it does not cover: Only classifies claims. No open-ended research, calculation or generation of answers.
- Last checked: 1 Nov 2026
- Links: site https://www.vals.ai/benchmarks/claimbench, leaderboard https://www.vals.ai/benchmarks/claimbench
- Directory page: https://benchmarks.finance/benchmarks/vals-claimbench/
- Leaderboard: https://benchmarks.finance/leaderboards/vals-claimbench/

## Results

### ClaimBench: accuracy % (higher is better, 10 models)

| Rank | Model | Provider | Score | Metric | Setting | Reported by | Source | URL | Date |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| 1 | d1-ultra | Not stated | 100 | accuracy % | decision models board | Official board | Vals.ai leaderboard | https://www.vals.ai/benchmarks/claimbench | 23 Oct 2026 (checked 1 Nov 2026) |
| 2 | Decider V1.2 | Not stated | 99.7 | accuracy % | decision models board | Official board | Vals.ai leaderboard | https://www.vals.ai/benchmarks/claimbench | 23 Oct 2026 (checked 1 Nov 2026) |
| 3 | Drex Max | Not stated | 99.2 | accuracy % | decision models board | Official board | Vals.ai leaderboard | https://www.vals.ai/benchmarks/claimbench | 23 Oct 2026 (checked 1 Nov 2026) |
| 4 | Mercury Decide | Not stated | 99.1 | accuracy % | decision models board | Official board | Vals.ai leaderboard | https://www.vals.ai/benchmarks/claimbench | 23 Oct 2026 (checked 1 Nov 2026) |
| 5 | Decider V1.1 27B | Not stated | 98.8 | accuracy % | decision models board | Official board | Vals.ai leaderboard | https://www.vals.ai/benchmarks/claimbench | 23 Oct 2026 (checked 1 Nov 2026) |
| 6 | d1 | Not stated | 98.7 | accuracy % | decision models board | Official board | Vals.ai leaderboard | https://www.vals.ai/benchmarks/claimbench | 23 Oct 2026 (checked 1 Nov 2026) |
| 7 | Microsoft-Decision-1 | Microsoft | 98.2 | accuracy % | decision models board | Official board | Vals.ai leaderboard | https://www.vals.ai/benchmarks/claimbench | 23 Oct 2026 (checked 1 Nov 2026) |
| 8 | Jev 1.14 | Not stated | 98.1 | accuracy % | decision models board | Official board | Vals.ai leaderboard | https://www.vals.ai/benchmarks/claimbench | 23 Oct 2026 (checked 1 Nov 2026) |
| 9 | Drex v1.5 | Not stated | 98.0 | accuracy % | decision models board | Official board | Vals.ai leaderboard | https://www.vals.ai/benchmarks/claimbench | 23 Oct 2026 (checked 1 Nov 2026) |
| 10 | Jev 1.13.0 | Not stated | 97.7 | accuracy % | decision models board | Official board | Vals.ai leaderboard | https://www.vals.ai/benchmarks/claimbench | 23 Oct 2026 (checked 1 Nov 2026) |


Source: https://benchmarks.finance/benchmarks/vals-claimbench/

Created and maintained by Moneda.