The first open-ended CUA dataset? On Fig's launch day, a 600-hour dataset from 7 months earlier was dug up

On November 5, Fig released RealCUA v0.1, and the announcement included a very strong word: first. According to Fig, this is the first computer-use dataset to record open-ended work in continuously evolving environments. On launch day, p(doom) author Franz Srambical said directly in a reply that this claim does not hold up.

Let's first lay out the facts from both sides.

What RealCUA includes

RealCUA v0.1 contains 606.5 hours of footage, 512 recordings, and 354,769 steps, including 274,984 actions and 79,785 idle states. There are 27,700 app switches, covering more than 35 apps, and 154 GB after compression. The longest continuous trajectory is 741 minutes, about 12.4 hours; the median recording is 27 minutes, with a mean of 71 minutes; 87% of the hours come from recordings longer than 1 hour. Multi-monitor recordings account for 14.5%.

These numbers were not produced by scripted tasks. Fig says the recorders are senior practitioners in the United States from engineering, operations, product, and research roles, doing real work on their own personal machines, with no assigned tasks and no scripted instructions; recording, use, and distribution were all done with consent.

What the data includes: H.264 video at 30 fps, multi-monitor; raw input events with sub-millisecond timestamps; step-level action JSON consolidated from input events; current app and window metadata; accessibility tree snapshots; cropped HTML and DOM snapshots; workflow segments and atomic tasks.

Fig's featured video:

The pain point it aims to solve

A computer-use agent needs to learn how to complete tasks on a desktop: open apps, switch windows, read pages, edit documents, reply to messages, and look things up. Many past datasets relied on scripted tasks, synthetic data, app clones, or very cleanly delineated task boundaries. RealCUA does the opposite: it starts recording from real work and preserves overlapping goals, notifications, loading states, interruptions, and resumptions. Fig counts an average of 45.7 app switches per hour.

App coverage is also very specific. Notion accounts for 25.28% of focused hours, Brave 15.04%, Cursor 11.51%, Claude 8.86%, Chrome 7.17%, Slack 6.74%, and Dia 5.24%. Further down are WindowManager, ChatGPT, Firefox, Terminal, Ghostty, Granola, Figma, Zoom, Discord, Finder, Spotify, and others.

For people training CUAs, the value of this kind of data lies in context: which window a click occurs in, which DOM element, which accessibility node, and what the preceding and following states are. For people doing evaluation, the value lies in long trajectories and interwoven multiple goals. Fig provides an 11-hour continuous-trajectory sample; the full dataset requires an application, and academic and open-source research groups can get in touch. The address is https://data.fig.inc/realcua .

The controversy: the two 600-hour datasets are not the same thing

Franz Srambical's reply said Fig's claim is not true. Seven months earlier, they had released more than 600 hours of data, with 15,000 hours internally, drawn from 3 to 12 months of work by top PhD students, and the cross-platform capture tool was open-sourced. That dataset is called AGI-CAST-0.6k, and at release it emphasized long-horizon AGI research, screencast-only, and a CC0 license. The accompanying crowd-cast is a privacy-preserving desktop app used to crowdsource behavior-cloning data with action annotations.

The p(doom) release video at the time:

Both datasets are called 600 hours, but they mean different things. AGI-CAST focuses on screen recordings of long-horizon research work, was released earlier, and has a more open license. RealCUA emphasizes continuously evolving environments, interwoven multiple independent goals, complete workspace state, step-level actions, and window context. Fig's 'first' has a string of qualifiers in front of it; the more qualifiers, the narrower the boundary. By the line '600 hours of real computer work made public,' RealCUA is not the first. By the line 'open-ended multiple goals, environments that are not reset, and step-level actions plus DOM/accessibility context,' Fig believes it is filling a different gap.

Words like 'first' are easily caught up by old records. The data itself will not be. People training CUAs can look at the 11-hour sample and the application channel; people doing evaluation should pay attention to access restrictions and license differences; people who want to reproduce should first compare the three layers: input events, state snapshots, and task segmentation. In the replies, an online commenter asked how private content such as passwords and messages is handled, and whether the 11-hour sample can be used directly for training. No public answer was seen in the source material.

Fig did not respond to Franz's challenge in the source material. RealCUA's data card, sample, and application entry point have already been released, and the code, data, and licenses for AGI-CAST and crowd-cast are also in public links. To judge who is closer to 'first,' one must first define what exactly is first.