Visa, 2026
Visa Agentic Commerce Benchmark
Whether an AI shopping assistant finds out what a customer needs, then picks a matching product or declines when nothing fits.
About this benchmark
- Size
- 15 private episodes, 3 runs each; 22 models in the first snapshot
- Public split
- none: episodes are private; one can be played as a demo
- Grading
- A simulated customer reveals requirements only when asked. A run passes if the assistant finds every requirement and the right buy or no-buy decision is made, checked against a hidden answer key.
- Licence
- Not stated
- Submission
- None published; Visa invites model developers' input
- Products allowed
- Models only, in Visa's harness
- Latest version
- October 2026 snapshot (released 2026-11-05)
- What it does not cover
- Checkout, payment and order placement (planned).
Results
22 modelsEach score shows who reported it: the benchmark's authors (quoted from their paper), the benchmark's official board, or Moneda (run with the benchmark's own grader and rules). We do not copy other aggregators.
Visa Agentic Commerce Benchmark October 2026 snapshot (released 2026-11-05)
pass rate %, higher is better. 22 models.
Updated 5 Nov 2026
- Gemini 4 Argon82.2
- Muse Wonder82.2
- Claude Fable 5.175.6
- GLM 5.366.7
- Claude Sonnet 5.564.4
- GPT-6 Astra64.4
- Muse Spark 1.364.4
- Claude Opus 5.562.2
- GPT-6.1 Sol62.2
- Grok 4.760
- Qwen 3.8 Max60
- DeepSeek V4.1 Flash57.8
- Beam 1.055.6
- Gemini 3.8 Flash53.3
- GPT-6 Luna53.3
- Kimi K3.1 Pro53.3
- GPT-6.1 Terra51.1
- MiniMax M348.9
- DeepSeek V4 Pro42.2
- Gemini 3.1 Pro35.6
- MiniMax M3.1 Flash33.3
- Inkling28.9
Visa Agentic Commerce Benchmark October 2026 snapshot (released 2026-11-05)
pass rate %, higher is better. Top 10 of 22 models.
Updated 5 Nov 2026
- 1Gemini 4 Argon82.2
- 1Muse Wonder82.2
- 1Claude Fable 5.175.6
- 1GLM 5.366.7
- 1Claude Sonnet 5.564.4
- 1GPT-6 Astra64.4
- 1Muse Spark 1.364.4
- 1Claude Opus 5.562.2
- 1GPT-6.1 Sol62.2
- 1Grok 4.760
| 1 | Gemini 4 Argon | 82.2 (69.0–91.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 | |
|---|---|---|---|---|---|
| 1 | Muse Wonder | Meta | 82.2 (69.0–91.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 1 | Claude Fable 5.1 | Anthropic | 75.6 (61.0–86.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 1 | GLM 5.3 | Zhipu AI | 66.7 (52.0–79.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 1 | Claude Sonnet 5.5 | Anthropic | 64.4 (50.0–77.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 1 | GPT-6 Astra | OpenAI | 64.4 (50.0–77.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 1 | Muse Spark 1.3 | Meta | 64.4 (50.0–77.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 1 | Claude Opus 5.5 | Anthropic | 62.2 (48.0–75.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 1 | GPT-6.1 Sol | OpenAI | 62.2 (48.0–75.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 1 | Grok 4.7 | xAI | 60 (45.0–73.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 1 | Qwen 3.8 Max | Qwen | 60 (45.0–73.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 1 | DeepSeek V4.1 Flash | DeepSeek | 57.8 (43.0–71.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 1 | Beam 1.0 | Reflection AI | 55.6 (41.0–69.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 14 | Gemini 3.8 Flash | 53.3 (39.0–67.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 | |
| 14 | GPT-6 Luna | OpenAI | 53.3 (39.0–67.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 14 | Kimi K3.1 Pro | Moonshot AI | 53.3 (39.0–67.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 14 | GPT-6.1 Terra | OpenAI | 51.1 (37.0–65.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 14 | MiniMax M3 | MiniMax | 48.9 (35.0–63.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 14 | DeepSeek V4 Pro | DeepSeek | 42.2 (29.0–57.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 14 | Gemini 3.1 Pro | 35.6 (23.0–50.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 | |
| 14 | MiniMax M3.1 Flash | MiniMax | 33.3 (21.0–48.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
| 14 | Inkling | Thinking Machines | 28.9 (18.0–43.0), 95% interval, as published by the source | 15 private episodes, 3 runs each | Visa Research, Oct 2026 snapshot Official board, reported 5 Nov 2026 · checked 6 Nov 2026 |
Papers and links
Last checked 6 Nov 2026. Something out of date or wrong? Send a correction.