After Topping the Voice Agent Blind Test, the Comments Care More About That 8.3%

On Artificial Analysis's Speech Agent Arena, Cartesia Managed Agents has taken the top spot.

Elo 1115, 98 points higher than Cartesia's previous-generation voice agent product Line. Task success rate rose from 77.5% to 91.7%.

This Arena is a blind-testing platform where people talk to different voice agents in real time and then rate them. Ratings aren't just about pronunciation; they're more about whether the agent can get things done in conversation. So in addition to Elo, it also measures task success rate—that is, in conversations that require tool calls, what proportion can correctly complete the entire task.

Specific data:

• Elo 1115 (95% confidence interval 1086 to 1144, 685 samples)
• Elo 1112 for agent tasks requiring tool calls
• 91.7% of qualified tool-call conversations ended with the correct final tool call

Cartesia is a company that makes voice models. Managed Agents is its fully managed platform for developing and deploying voice agents. The architecture is cascaded: Cartesia's own Ink speech-to-text and Sonic text-to-speech, with a developer-selected LLM running in the middle. This evaluation used GPT-5.5, with voices Skylar and Daniel.

Pricing: $0.06 per minute, or $3.60 per hour. LLM costs are billed separately by model.

The full leaderboard and methodology are available on the Artificial Analysis website.

The numbers look impressive. The comments offer another perspective.

One user said task success rate is the number voice agents should really be judged on. Callers don't rate the voice; they just hang up when the agent messes up a booking.

Another user added that the tasks in the Arena are clean. Real calls have crosstalk, poor mobile signals, interruptions mid-conversation, and corrections halfway through a sentence. It would be best to break down success rate by interruption handling and noise level.

Someone else put it more bluntly: the evaluation runs on clean audio; on real phone lines, the final boss for voice agents is one bar of signal and a phone on speaker.

Other users pressed on that 8.3% of failures: mid-call hang-ups, timeouts, tool failures—these are also outcomes users will really encounter in deployment. A 91.7% success rate sounds good, but roughly 1 in every 12 calls will fail. For a customer service hotline, it matters whether a human took over or the bot just hung up.

One user asked: is this a gain from changing the underlying model, or from orchestration?

Another user reposted: the numbers are impressive, but task completion rate matters more than anyone thinks. Overall, it still looks behind Grok and Gemini.

These doubts are not unreasonable. The evaluation runs on clean audio. In real scenarios, one bar of signal, the other party on speakerphone, or a sudden interruption can all break the chain of tool calls. 91.7% is a good starting point, but for voice agents to enter real customer service systems, they first have to get past that 8.3% barrier.