Alibaba's OpenGCUA opens a computer-use agent stack that lifts eight models on OSWorld-v2

What Alibaba's team released

The MAI-UI team at Alibaba's Tongyi lab has released the first version of OpenGCUA, short for Open Generalist CUA, a project to build an open stack for AI agents that use computers. The technical report went up on arXiv on 15 October 2026, the same day the GitHub repository was created under an Apache 2.0 licence.

The release has four parts: an integration with the OSWorld-v2 benchmark containing reference agents, evaluation trajectories from eight frontier models, a preliminary real-device harness that runs an agent on your own Mac, and a preliminary Cross-Device SDK meant to give one programming interface for computers, browsers and phones. The model the team actually wants to build, a dedicated GCUA model, is listed as the next step and is not part of this release.

The headline claim is about the harness, not a model. On OSWorld-v2, the team says its design raised the mean partial score of all eight models it tried from 45.3 to 64.9, an average gain of 19.6 points, while cutting the steps each task took by 53 per cent on average.

What a generalist computer-use agent is

Most computer-use agents work the way a cautious person with a mouse would: take a screenshot, decide on one click or keystroke, take another screenshot. The OpenGCUA authors argue that frontier systems have moved past that, and they name OpenAI's GPT-6 Astra computer-use system as the typical example of what they call a generalist computer-use agent.

In their definition, a GCUA is a system in which the model itself chooses how to observe, how to act and how to organise its work. It can read an app's accessibility tree, call the app's API, run shell commands, and fall back to clicking the screen only when nothing better is available. It can also share the computer with a person by working in background windows.

The team's point is that those systems are mostly closed, so nobody outside the labs can study which design choices matter. That is the same gap that motivated work like Google Research's Harness2, which treats the agent harness as something to be researched and improved in the open rather than a fixed wrapper.

How OpenGCUA works

OpenGCUA rests on two ideas. The first is code execution: instead of emitting one GUI action at a time, the model writes a Python program that can mix GUI actions with accessibility-tree reads, API calls and shell commands, with loops, branches and checks. Intermediate results can then decide the next operation without another round trip to the model.

The second is what the authors call harness-to-model decision transfer. In a typical GUI agent the harness decides when to take a screenshot and how to track progress. OpenGCUA hands those choices to the model: it calls screenshot() only when it needs to look, keeps its own todo list, and states its intent before each program. The README describes GUI agents, command-line agents and the team's earlier Qwen-UI-Agent hybrid as special cases that fix some of these decisions in advance.

The README shows a real step from GPT-6 Astra on a CRM task. In one program, the model records its plan, writes a script that stops if the app changed since it last looked, applies 29 lead-status changes through the app's own API, verifies each one, and only then marks the task as verified. That is the kind of work that would take dozens of clicks and screenshots in a GUI-only agent.

The OSWorld-v2 results

The main test is the offline set of OSWorld-v2, 82 tasks according to the project's results chart, compared against a GUI-only baseline that already batches actions. The eight models were GPT-6 Astra, Claude Opus 5, Gemini 3.8 Flash, GLM-5.3-Flash, Qwen3.8-Max, Qwen3.8-Flash, Kimi K3 and DeepSeek-V4.1-Flash.

Every model improved. GPT-6 Astra went from 67.1 to 75.7 and Claude Opus 5 from 60.7 to 70.1, gains of about nine points each. The weaker models gained most: GLM-5.3-Flash rose 35.2 points, from 29.6 to 64.9, and DeepSeek-V4.1-Flash 29.8 points, from 28.0 to 57.7. As a result, the gap between the strongest and weakest model shrank from 39.1 points to 18.0.

Steps per task fell for all eight. GPT-6 Astra dropped from 102 to 34 steps, a 67 per cent cut, and Qwen3.8-Flash from 232 to 86. Gemini 3.8 Flash saw the smallest reduction, from 167 to 101.

The team also isolated the effect of code execution. With the same GUI and shell access, writing operations as code added 6.1 points and cut steps by 14.8 per cent compared with a GUI-plus-command-line agent, averaged over three models. That matters because it suggests the gain is not only from giving the agent a terminal.

Real Macs, the SDK and how to try it

Benchmarks leave out much of what goes wrong on a real machine, so the team also built a macOS harness. It runs as a menu-bar app, uses the open-source CuaDriver from trycua to control the Mac, and needs Accessibility and Screen Recording permissions. It requires macOS 14 or later, and the README warns that pressing Ctrl-C stops the current turn but does not roll back operations already sent to apps or the shell. A native SwiftUI app with voice and a floating task panel exists as a local development build.

On four real macOS tasks with the harness held fixed, the team reports that Qwen3.8-Max used 19.4 times as many reasoning tokens as GPT-6 Astra and completed three of the four tasks, while GPT-6 Astra completed all four. The authors trace that gap to model behaviour, which is what they say their future GCUA model will target.

The demo video shows four setups filling in the same visa form on a Mac: OpenGCUA, OpenGCUA set to prefer the GUI, a batched GUI agent and OpenAI's Codex. In the frame used as the thumbnail, the default OpenGCUA run shows done at 2 minutes 28 seconds and the GUI-preferring run at 12 minutes 4 seconds, while the other two are still running at 15 minutes 24 seconds. It is one task, chosen by the team.

The Cross-Device SDK, published as qwen-intelligence-sdk, is a Rust core with Python bindings and adapters for macOS, Android, browsers and Linux. It binds each operation to an explicit device, app, window or tab, which is how one program can coordinate several devices in a task. Developers point the harness at any OpenAI-compatible or Anthropic endpoint by editing a local configuration file.

Why an open harness matters

Computer use is becoming a standard feature of AI assistants. OpenAI's ChatGPT Chrome extension can now act inside open tabs, and smaller labs such as mncai have trained models built specifically for computer use. What has been missing is an open, inspectable version of the full system around the model.

OpenGCUA's numbers suggest that the harness matters as much as the model for this kind of work. If a mid-sized open model can more than double its OSWorld-v2 score by acting through code and choosing when to look, then builders who cannot afford the top closed models get a cheaper path, and researchers get released trajectories from eight models to study where agents fail.

It also changes what a fair comparison looks like. A model tested inside a bare screenshot-and-click loop may look much weaker than the same model inside a harness like this one, which is a useful caution for anyone reading computer-use agent rankings.

Limits and open questions

All results come from the OpenGCUA team and have not been independently reproduced. The OSWorld-v2 scores are partial-credit averages on an offline set, the real-device test covers four tasks, and the demo is a single form. The baseline is the team's own GUI-only setup, so the size of the gain depends on how strong that baseline is.

Much of the stack is labelled preliminary. The macOS harness and SDK are early versions, the native app is a development build, and the GCUA model, efficiency work and a real-device benchmark are future items with no dates. The team says it does not commit to timelines.

An agent that can run shell commands and drive apps on your own computer also carries obvious risk. The project asks for broad macOS permissions, and actions are not rolled back when you stop a run, so it is best tried on a test machine or account. Things to watch are independent reruns of the OSWorld-v2 numbers, the first GCUA model release, and whether other labs adopt the code-execution design.