AI Benchmark Scores in a Bank Simulator: Claude Opus 5.5 Scores 609, Grok 4.7 Scores 241
Veris AI releases FinAgent Arena, which tests AI agents on AML cases and TreasuryQA in a simulated bank. Out of 1000, Claude Opus 5.5 scores 609 and Grok 4.7 scores 241; costs also deduct points.
Veris AI releases FinAgent Arena, an index that measures AI agents performing financial tasks in a high-fidelity simulated banking environment. The simulated target is the back end of an entire bank: core banking, CRM, productivity tools, financial crime systems, project management, and engineering systems, with internal collaborators and external customers also simulated. In testing, leading models are paired with their native harnesses to run real tasks. The first two benchmarks are TreasuryQA and AML Bench, with more added each week.
In the leaderboard, ten agents each handle an anti-money-laundering case and then answer a question from a document library, all in the same simulated bank and the same frozen world. AML Bench is 153 investigations × 3; TreasuryQA is 70 questions × 3. Arena Index is out of 1000, rewards correct and stable performance, weighted by what US banks pay people for these two types of work; costs above the current leader incur deductions. The rule is stated plainly: every doubling of cost above $1.95 deducts 20 points.
The video breaks the simulated bank into three layers: the server room, the branch, and the back office. The dominant color is green; people queue at the counters, while back-office staff process cases on dual monitors. The work the agents have to do falls within these systems.
| Rank | Agent | Harness | Arena Index | AML Bench | TreasuryQA | Basket Cost | Cost Penalty |
|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Claude Code | 609 | 652.6 | 568.5 | ~$2.00 | −0.7 |
| 2 | GPT-6.1 Sol | Codex | 567 | 691.1 | 461.7 | ~$0.88 | 0 |
| 3 | Gemini 4 Argon | Gemini CLI | 566 | 615.4 | 541.8 | ≈$2.98 | −12.2 |
| 4 | Claude Sonnet 5.5 | Claude Code | 492 | 672.0 | 355.8 | ~$1.40 | 0 |
| 5 | Gemini 3.8 Flash | Gemini CLI | 478 | 511.1 | 452.6 | ≈$2.22 | −3.7 |
| 6 | DeepSeek V4.1 Flash | Pi | 415 | 611.2 | 277.5 | ~$0.31 | 0 |
| 7 | Kimi K3 | Pi | 384 | 642.4 | 238.0 | ~$2.86 | −11.0 |
| 8 | GLM 5.3 Flash | Pi | 362 | 652.1 | 195.9 | ≈$0.26 | 0 |
| 9 | GPT-5.6 Terra | Codex | 278 | 575.0 | 130.2 | ~$1.46 | 0 |
| 10 | Grok 4.7 | Grok CLI | 241 | 625.6 | 105.6 | ~$4.04 | −21.0 |
The TreasuryQA chart puts cost and accuracy together: the x-axis is average agent cost per question, on a log scale from $0.25 to $4.00; the y-axis is pass@1, from 10% to 70%. The green line is the Pareto frontier. The chart shows that cheap models are not necessarily worse, and expensive models are not necessarily more stable.
Looking at pass@1 alone makes things look much better, but banks will not pay just because an answer is right once. Arena Index puts stability, task weighting, and cost into a single score, making the ranking closer to an operational perspective. For example, GPT-6.1 Sol has a higher AML Bench score than Claude Opus 5.5, but a lower TreasuryQA score and lower basket cost; its Index is still 42 points behind.
The full report is at veris.ai/leaderboard/arena. TreasuryQA is in collaboration with @hu_yifei and the @reductoai team. The original post is here.
发布时间: 2026-11-03 02:40