Sierra, 2026
τ³-Banking
Text agents resolve banking customer-service tasks by retrieving from a knowledge base of about 700 documents. Published as τ-Knowledge; the banking_knowledge domain of τ³-bench.
About this benchmark
- Size
- Not stated
- Public split
- Code and tasks in the open tau2-bench repository
- Grading
- pass^k: the share of tasks solved in all of k trials, with an LLM user simulator. Headline is pass^1.
- Licence
- MIT (code)
- Submission
- Submissions to the τ-bench leaderboard
- Products allowed
- Not stated
- Also known as
- Tau3-Banking, tau-knowledge
- What it does not cover
- Customer service only: no investment analysis, trading or filings.
Results
5 modelsEach score shows who reported it: the benchmark's authors (quoted from their paper), the benchmark's official board, or Moneda (run with the benchmark's own grader and rules). We do not copy other aggregators.
τ³-Banking
pass^1 %, higher is better. 5 models.
Updated 4 Aug 2026
- Qwen 3.8 Max55.2
- Claude Opus 548.7
- Grok 4.547.9
- GPT-5.6 Sol46.9
- GPT-5.544.6
τ³-Banking
pass^1 %, higher is better. 5 models.
Updated 4 Aug 2026
- 1Qwen 3.8 Max55.2
- 2Claude Opus 548.7
- 3Grok 4.547.9
- 4GPT-5.6 Sol46.9
- 5GPT-5.544.6
| 1 | Qwen 3.8 Max | Qwen | 55.2 | all tools retrieval, gpt-5.2 user simulator | τ-bench leaderboard (Sierra) Official board, reported 4 Aug 2026 · checked 2 Nov 2026 |
|---|---|---|---|---|---|
| 2 | Claude Opus 5 | Anthropic | 48.7 | all tools retrieval, gpt-5.2 user simulator | τ-bench leaderboard (Sierra) Official board, reported 4 Aug 2026 · checked 2 Nov 2026 |
| 3 | Grok 4.5 | xAI | 47.9 | all tools retrieval, gpt-5.2 user simulator | τ-bench leaderboard (Sierra) Official board, reported 4 Aug 2026 · checked 2 Nov 2026 |
| 4 | GPT-5.6 Sol | OpenAI | 46.9 | all tools retrieval, gpt-5.2 user simulator | τ-bench leaderboard (Sierra) Official board, reported 4 Aug 2026 · checked 2 Nov 2026 |
| 5 | GPT-5.5 | OpenAI | 44.6 | all tools retrieval, gpt-5.2 user simulator | τ-bench leaderboard (Sierra) Official board, reported 5 May 2026 · checked 2 Nov 2026 |
Papers and links
Last checked 2 Nov 2026. Something out of date or wrong? Send a correction.