We tested 27 models on marketing videos: the strongest and the most cost-effective are not the same

Artificial Analysis and HeyGen have collaborated to release a new benchmark, Code2Video-AA. It is an independent implementation of HeyGen's Code2Video Bench, designed to measure whether a model can turn a product/brand marketing brief into a video.

How it was tested

27 models, 114 design briefs. Each model reads the brief and writes a HyperFrames HTML composition file, without being able to call tools. HyperFrames renders the file into a video, with no network access during rendering. Videos that fail to render automatically lose all comparisons.

The judge is Gemini 4 Argon (high). For two videos for the same brief, it performs pairwise comparisons at 8 fps and scores them across five dimensions: Engagement, Prompt Intent, Composition, Temporal Consistency, and Craft. A separate Bradley-Terry fit is done for each dimension to calculate the Elo for that dimension. Elo is a relative score calculated from pairwise win/loss outcomes, commonly used in chess and esports. Overall Elo is the average of the five dimension Elos, anchored to GPT-5.5 (xhigh) at 1000.

The whole process involved more than 35,000 judgments, ranking 27 models, with 95% confidence intervals.

Results

Claude Opus 5.5 (max with fallback) ranks first overall with an Elo of 1536, 93 points ahead of second place. But it costs $3.08 per task. Claude Sonnet 5.5 (max with fallback, 1443) and GPT-6 Astra (max, 1427) are statistically tied for second. GPT-6.1 Sol (max) is fourth with 1369 points and costs only $0.31 per task, about one-tenth of Opus 5.5.

Claude Opus 5.5 is first in all five dimensions. Craft has the largest lead, 128 points above Sonnet 5.5; Prompt Intent has the smallest lead, 48 points. Temporal Consistency has the smallest gap between models among all dimensions, with an Elo spread of 1057 points; the other four dimensions have spreads between 1222 and 1349 points. No model moves more than 6 places between any single-dimension ranking and the overall ranking.

Cost

Per-task cost ranges from $0.01 to $3.89, a spread of more than 300x. A high score does not necessarily mean spending more: GPT-6.1 Sol ranks fourth and costs only one-tenth of the top model. Eight models fall on the Pareto frontier, from MiMo-V2.6-Flash (1052 points, $0.01) to Claude Opus 5.5 (1536 points, $3.08).

Open-weight models currently trail by at least 483 points. The strongest open-weight models are DeepSeek V4.1 Flash (1053 points), MiMo-V2.6-Flash (1052), and MiMo-V2.6-Pro (1052), with per-task costs between $0.01 and $0.08.

Examples

Same brief: a chat bubble that cycles through colors. The outputs from four models are below. Claude Opus 5.5 cost $3.08 and about 152k tokens; DeepSeek V4.1 Flash $0.08, 65k tokens; MiniMax-M3 $0.08, 63k tokens; Nemotron 3 Ultra $0.03, 11k tokens.

In the comments, someone did the math: at $3.08 per task to make one marketing video, it might be cheaper to hire a person. One commenter said, finally a benchmark that behaves like a real client: if it can't render, it loses immediately. Another added a client request: the real benchmark is request #115—keep the video the same, just make the logo a little bigger, and it's due Friday.

Full leaderboard: https://artificialanalysis.ai/evaluations/code2video-aa
Methodology: https://artificialanalysis.ai/methodology/intelligence-benchmarking#code2video-aa
Announcement article: https://artificialanalysis.ai/articles/code2video-aa

Thanks to HeyGen's Code2Video Bench and HyperFrames.