> ## Content Index > Fetch the complete content index at: https://www.fig.inc/llms.txt > Use this file to discover other available public pages before exploring further. # Models are Outpacing Environment Creation - URL: https://www.fig.inc/blog/models-are-outpacing-environment-creation/ - Published: 2026-11-11T16:45:57.000Z - Updated: 2026-11-11T16:45:57.000Z - Author: Fig Team - Tags: Perspectives Last week, we released [RealCUA](https://data.fig.inc/realcua?utm%5Fsource=blog&utm%5Fmedium=blogpost&utm%5Fcampaign=models-outpacing-environment-creation&utm%5Fcontent=intro%5Flink), a dataset with over 600 hours of unscripted, multi-objective computer work from real people on their own desktops. It captures the signals agents need to continuously work as objectives and conditions change. Given the interest it received from frontier labs and research teams, we wanted to share more on how we’re thinking about turning real, open-ended work into consumable learning signals for training. Each frontier model release increases the horizon of work that can be completed autonomously. In 2025, METR estimated that this horizon doubles about every seven months\[1\]. Their 2026 update puts the doubling time since 2023 at about four months, with frontier agents now succeeding half the time on tasks that take people 12 to 17 hours\[2\]. Building agents that take on open-ended work? Get an early preview of our datasets and environments. Request a preview → The environments used to train the next generation of agents must also grow alongside that horizon, and as tasks get longer and harder, so do the time and cost of creating them. OSWorld 2.1, the long-horizon computer-use benchmark Anthropic reports for Fable 5.2, has 108 tasks estimated to take a skilled person a median of about 1.6 hours each\[3\]. Building them took trained annotators, practitioner interviews, and audits of every task. The 175 workplace tasks in TheAgentCompany took about 3,000 person-hours to build\[4\]. *That leaves the open question: where do environments calibrated to workflows that span tens of hours, weeks, or months come from?* Throughout our research into models capable of open-ended work, and our work on SoTA evals with [MultiNet](https://arxiv.org/abs/2506.09172?ref=fig.inc), we’ve come to believe that models will soon outpace our ability to build verifiable environments by hand. We’re working to solve that bottleneck by turning the real work people already do on their computers every day into consumable learning signals for training and evaluating agents. Part of our recent focus has been capturing the breadth of work people do in their own jobs across roles. Our next dataset goes further, mapping computer activity through linked actions, tasks, and objectives, over months of real work. Ahead of its release, we’re previewing our observations from a small slice of the hundreds of hours we’ve recorded to show what that structure captures. The slice is about seven hours from four sessions across four roles: a software engineer, research lead, product marketer, and designer. ## The hierarchy of real work The boundaries of scripted tasks are set during creation\[5\]. They start with a defined initial state snapshot that can be reset to, and end with a verifier that checks the result when the agent declares it has finished or reaches the defined step limit\[6\]. While that structure makes tasks gradable, each attempt starts fresh, and the work sits isolated from the context that would normally surround it in the real world. In open-ended work, boundaries are defined by the work itself. Each task is recorded with the actions surrounding it: the earlier work left behind, other tasks still open, and the actions that follow it. To capture that context, we use a model to segment each session into a hierarchy of three levels of abstraction. The smallest is an action. Actions are the atomic units of activity represented as clicks, keystrokes, scrolling or dragging. We group those actions into tasks, which break out into pieces of work with results someone could check, like renaming a page, sending a message, or drafting a comment. Tasks are then grouped and organized into objectives, the larger outcomes they build toward: completing a code review, writing a blog post, redesigning a webpage. Objectives are linked to the tasks that serve the same result, even when other work happens between them. ### Objective completion time by role Software engineerResearch leadProduct marketerDesigner 1234567Median 16 minMedian 2 tasks010203040506070Minutes openTasks1234567Median 16 minMedian 2 tasks0204060Minutes openTasks Figure 1: How long each of the 31 objectives stays open, from its first task to its last, by role. The median objective stays open for about 16 minutes and holds two tasks. Across those seven hours, the work breaks down into 103 tasks and 31 objectives, or about 16 and 5 an hour respectively. Tasks range from 2 to 95 actions, with the median landing at 17\. They also run back to back, with a median of 13 seconds between one task ending and the next starting. Objectives span a median of two tasks and 65 actions and stay open for a median of about 16 minutes from their first task to their last, with some running close to an hour as other work is interleaved within their boundaries. This preview shows that structure within single sessions, while the full dataset maps it over months of real work. ## Context lives between tasks Because sessions are recorded continuously, the context surrounding every task and objective stays intact, tying each piece of work to the broader context around it. That continuity gives us insight into how individual tasks are sequenced, how desktops evolve, and how work travels across multiple applications. ### Work builds on itself Much of what we do on a computer follows from the result of a task before it: test results decide which file gets fixed, and an earlier draft becomes the subject of the next edit. The transitions between tasks happen quickly: 91% of tasks start or end within 30 seconds of another task, and 4% overlap another task. ### Time between sequenced tasks by role Software engineerResearch leadProduct marketerDesigner 05101520Median 13 s≤112481632641282565121024Gap between tasks (seconds)Tasks12313151312101253105101520Median 13 s≤1416642561024Gap between tasks (seconds)Tasks123131513121012531 Figure 2: Gaps between consecutive tasks across the 99 task transitions in four sessions, by role. The median gap is 13 seconds. The continuity in recorded sessions of open-ended work keeps those transitions in place. In one session, a research lead reviewing a pull request with an AI assistant stopped partway through the files to spend 27 minutes drafting comments on two of them. Those comments carried into the review’s final five tasks, as the research lead reviewed the remaining files, ran the tests and submitted the review with the comments attached. Handwritten tasks are built one at a time, so connections to the work around them exist only if someone writes them in. Agents handed work midstream have to carry forward what earlier tasks produced, and the transitions between tasks capture that learning signal. ### Starting where the last task left off Our desktops are never staged for the work ahead of us. Windows remain open, tabs pile up, and the page with the information needed might sit behind several others. The time between tasks is spent tidying up, looking around, and sorting through notifications before deciding what to do next. In the sessions recorded, 18% of all inputs (clicks, keystrokes, scrolls, and drags) fall outside any task, from bringing windows forward and closing tabs to clicking through pages between tasks. Because those actions are recorded alongside the tasks, the starting state for each task is captured as it was, along with how it was created. Agents working alongside us have to interpret real desktops and identify task-relevant context among everything else that’s open. The actions between tasks are the learning signal for how desktops change. ### Tasks cross app boundaries Across these sessions, more than half the tasks span two or more apps, with the share varying by role. The software engineer and research lead prompted an AI agent and then switched apps to check their results. The designer and product marketer typically finished a task in one app before moving to the next. ### Multi-app tasks by role | Role | Tasks | Multi-app tasks | Share | Median app switches per objective | | ----------------- | ----- | --------------- | ----- | --------------------------------- | | Software engineer | 14 | 13 of 14 | 93% | 4 | | Research lead | 33 | 28 of 33 | 85% | 11 | | Product marketer | 16 | 6 of 16 | 38% | 0 | | Designer | 40 | 10 of 40 | 25% | 0 | | All roles | 103 | 57 of 103 | 55% | 3 | Table 1: Tasks that span two or more apps, by role. Of the 103 tasks, 57 are multi-app, from 93% (13 of 14) for the software engineer to 25% (10 of 40) for the designer. New tasks rarely coincide with switches to a different app: 84 of the 99 transitions begin in the app where the previous task ended. Benchmark tasks open the apps a task needs and state the goal up front. In real work, apps from earlier work are left open, and retrieving the information is part of the goal. Agents working alongside us have to move between apps without losing sight of it. Every app transition we capture inside a task holds signal for that capability. ## Mapping failures to missing signals The capabilities we’ve described map to the failure modes observed in benchmarks like OSWorld 2.1\. In Anthropic’s own run on updated task files, Fable 5.2 earns 89.3% partial credit and completes 54.6% of tasks outright\[7\]. The benchmark’s analysis of earlier frontier models attributes failures to agents losing track of constraints, skipping the step to verify their work, struggling with tasks that depend on hidden states they had to recover, and missing information that arrived mid-task\[3\]. Three of those failures trace back to signal in open-ended work that we’ve outlined: constraints set by earlier results, the hidden state of an inherited desktop, and information that sits in another app. In ChainWorld, two to four short tasks from the original OSWorld run back to back in one session without resetting the desktop\[8\]. The chains are chosen by searching for tasks that fit together, so dependencies between them are mapped after creation, not from real work. When tasks are revealed one at a time, fragmented progress and disengaging from later tasks make up 37% of coded failures, against 8% when the whole chain is given at once. Real objectives already take that shape: in our preview, 20 of the 31 objectives hold two to four tasks, and 19 of the 25 with more than one task have other tasks between them. Whether it’s a single task or a chain of sequenced tasks, agents fail where work is dependent on context that wasn’t provided. The learning signals on how to capture that context, from how tasks chain into objectives to how work moves across the desktop and between apps, already exist in real, open-ended work. ## What comes next Maintaining, or even accelerating, the progress of model capabilities requires the difficulty and realism of the tasks they train on to scale alongside them. Soon, that progress will outpace our ability to create those tasks by hand, as the length and difficulty required, coupled with a shrinking pool of people who have the expertise to build them, drive up cost. Our research focuses on new approaches to environment creation where length and difficulty come from real, open-ended work, and scale with models to stay just past the frontier. Next week, we’re sharing a frontier dataset of open-ended work, hierarchically decomposed into linked actions, tasks, and objectives. Building agents that take on open-ended work? Get an early preview of our datasets and environments. Request a preview → ## Citation ``` @article{fig_models_are_outpacing_environment_creation_2026, title = {Models are Outpacing Environment Creation}, author = {Joseph Hakim and {Fig AI Team}}, year = {2026}, institution = {Fig}, url = {https://www.fig.inc/blomodels-are-outpacing-environment-creation/} } ``` ## References 1. T. Kwa et al., ["Measuring AI Ability to Complete Long Software Tasks,"](https://arxiv.org/abs/2503.14499?ref=fig.inc) Mar. 18, 2025, arXiv. doi: 10.48550/arXiv.2503.14499. 2. METR, ["Time Horizon 1.1,"](https://metr.org/blog/2026-1-29-time-horizon-1-1/?ref=fig.inc) Jan. 29, 2026. 3. M. Yuan et al., ["OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks,"](https://arxiv.org/abs/2606.29537?ref=fig.inc) Jun. 28, 2026, arXiv. doi: 10.48550/arXiv.2606.29537. 4. F. F. Xu et al., ["TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks,"](https://arxiv.org/abs/2412.14161?ref=fig.inc) Dec. 18, 2024, arXiv. doi: 10.48550/arXiv.2412.14161. 5. T. Xie et al., ["OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments,"](https://arxiv.org/abs/2404.07972?ref=fig.inc) Apr. 11, 2024, arXiv. doi: 10.48550/arXiv.2404.07972. 6. A. Sharma et al., ["Autonomous Reinforcement Learning: Formalism and Benchmarking,"](https://arxiv.org/abs/2112.09605?ref=fig.inc) Dec. 17, 2021, arXiv. doi: 10.48550/arXiv.2112.09605. 7. Anthropic, "[System Card: Claude Fable 5.2 & Claude Mythos 5.2,](https://assets.claude.com/27d60918d218125e7504bc7354c4b4626ec56b11.pdf?ref=fig.inc)" Nov. 9, 2026. 8. V. Siu et al., ["ChainWorld: Composing Long-Horizon Desktop Workloads from Atomic OSWorld Tasks,"](https://arxiv.org/abs/2606.21654?ref=fig.inc) Jun. 19, 2026, arXiv. doi: 10.48550/arXiv.2606.21654.