Double Your AI Agents success rate. Without changing the LLM
Five sentences to take with you
StructAgent is a harness design from UC San Diego and Aether AI that replaces an agent’s raw interaction history with a compact, typed record of task progress.
Every progress update must pass a verifier that checks hard evidence such as files, URLs, and shell output, so the planner and actor can propose but never approve.
On OSWorld-Verified, the identical Qwen3.5-27B model jumps from 31.6% to 62.2% task success inside this harness, with zero fine-tuning.
For builders, it is the strongest recent evidence that context architecture, not model choice, decides whether long tasks survive their own history.
In the audited failures, verifier mistakes are nearly as common as actor mistakes, which makes evidence design the next bottleneck to watch.
Run a desktop-automation benchmark twice. Same model, Qwen3.5-27B. Same 100-step budget. Same scoring pipeline. The first run completes 31.6 percent of the tasks. The second completes 62.2 percent.
Nothing about the model changed between those runs. What changed is what the agent was allowed to remember, and who was allowed to say “that part is done.”
The second run used StructAgent, a framework published on July 13 by researchers at UC San Diego and Aether AI. Its core claim sounds almost bureaucratic: agents should not work from their raw interaction history. They should work from a small, structured state of task progress, and nothing gets written into that state without evidence a verifier has checked. That one design decision, applied consistently, is worth more on this benchmark than several model generations.
Where long tasks go to die
A computer-use agent runs a loop that hasn’t changed much since ReAct: observe the screen (a screenshot or an accessibility tree), think, click or type, observe again. Every observation, action, and tool output gets appended to the context window, and that growing transcript is, in most systems, the agent’s entire memory of the task.
For a 5-step task, a transcript is fine. For the tasks in OSWorld, a benchmark of real desktop work spanning spreadsheets, browsers, email clients, GIMP, and VS Code, trajectories run dozens of steps across multiple applications. The transcript fills with things that are no longer true: a file path from an attempt that failed, an edit that a later action overwrote, a subtask the agent believes it finished because the screen looked right at the time.
Two rots set in. Useful facts get buried under noise, and progress becomes ambiguous. The agent scrolls back through its own history and can’t reliably tell which of its past claims still hold. The signature symptom is the false “Done”: the model announces completion, the final screenshot looks plausible, and the exported file simply isn’t on disk.
The StructAgent authors put the diagnosis bluntly: existing agents operate over raw history, which makes progress “difficult to interpret, verify, and recover.” Their fix is not a better model or a longer context window. It’s a different data structure.
A ledger with three columns and a bouncer
Simply put, StructAgent swaps the transcript for a ledger. Instead of every module rereading the whole history and inferring where things stand, the harness maintains a compact state with exactly three kinds of entries:
Requirements: what the current task still demands, each carrying a status of Pending, Verified, or Invalidated. This is what must be achieved and what the verifier should check.
Values: concrete facts execution has produced and later steps will need. File paths, URLs, selected entities, extracted fields.
Evidences: the verified proof supporting current progress, kept so later turns can audit and reuse it.
The interesting part isn’t the schema. It’s the write permissions. In StructAgent, every state transition must be triggered by a verifier-backed decision. The planner can propose a subgoal. The actor can claim it executed one. Neither can flip a requirement to Verified. Only the verifier, after probing the actual environment (reading files, running shell checks, inspecting application state, following URLs) can commit progress, and only conflicting evidence can revoke it by marking a requirement Invalidated.
Think of it like a codebase with branch protection. Anyone can open a pull request. Nothing merges without the checks passing. Most current agents, by contrast, let the intern who wrote the code also click approve.
Proposals don’t count, commits do
The runtime loop is a fixed cycle: the planner reads the state and picks the next subgoal, the actor attempts it in the environment, the verifier examines the evidence, and only then does the state update. A memory module suggests reusable tactics, and recurring procedures get promoted into typed tools, but their outcomes pass through the same gate as everything else.
The paper’s case study shows why the gate matters. In a multi-application task, the actor exports an image from a document attachment and, by all visual appearances, succeeds. The verifier looks for the file and finds nothing. Instead of accepting the completion claim, it keeps the subgoal Pending, which blocks the false Done and triggers a targeted probe. Once the extracted image actually shows up on disk, the requirement commits, and the planner safely moves on to using that image as a wallpaper.
That same discipline produces two capabilities that transcripts can’t offer. Because the state is a minimal, self-contained snapshot of progress, it doubles as a checkpoint: an agent can resume a long workflow without replaying its history. And because failures leave verified evidence behind, recovery gets targeted. The system can distinguish “insufficient evidence,” “conflicts with a verified requirement,” “repeated failure under this strategy,” and “blocked by the environment,” then pick replanning, retrying, extra checks, or human escalation accordingly, rather than looping through generic retries.
What doubling actually looked like
The headline numbers come from OSWorld-Verified, the cleaned split of OSWorld, under a controlled protocol: identical runner, scorer, and 100-step budget for every system. Wrapped in StructAgent, Qwen3.5-9B goes from 27.0% to 46.9% task success. Qwen3.5-27B goes from 31.6% to 62.2%. With MiniMax-M3 as the backbone, the framework reaches 78.9%, which the authors report as a new state of the art among open-source setups on this benchmark.
The comparison to other harnesses is the fairer test, and it holds up. Under the same matched settings on the 27B backbone, reproduced versions of Agent S3 reach 60.2%, OS-Symphony 52.8%, and VLAA-GUI 48.2%. StructAgent’s 62.2% edges out all three, and the domain breakdown says something about why. Office tasks, where the 27B single-model baseline scores 26.1%, jump to 71.7%. Spreadsheets and documents expose durable intermediate state (cells, files, application-level checks), which is exactly what an evidence-driven verifier can grip.
The authors also isolated the verifier itself on 354 scored trajectories, using OSWorld’s environment checker as ground truth. Against a screenshot-only verifier at the final completion gate, structured verification raises recall from 0.67 to 0.81 and F1 from 0.70 to 0.76 at similar precision. In a hand review of 100 cases where the two verifiers disagreed, the structured one was right 63% of the time versus 31% for the visual one. Its advantage came from latent evidence: files, command outputs, URLs, and application state that no screenshot reliably shows.
One more result deserves attention because it tests the idea rather than the implementation. The team rebuilt the same architecture in Minecraft, swapping desktop probes for inventory checks: a subgoal like craft one iron pickaxe only commits when the item is verifiably in the inventory. With a GPT-4o planner, that instantiation averages 76% success across task tiers against 59% for Optimus-1, a published Minecraft agent, with the biggest gains on long crafting chains. Same state discipline, completely different evidence source.
This is not a fancier to-do list
Plenty of harnesses already keep a plan or a task list in the prompt, so it’s tempting to file StructAgent under prompt engineering. That misses the mechanism. A to-do list the model edits freely is still self-graded homework; the failure mode isn’t that agents lack a list, it’s that the same model that does the work gets to mark its own work complete, with no requirement that the claim survive contact with the environment.
The load-bearing feature here is the separation of powers plus the evidence requirement. Progress claims become checkable objects with provenance, not sentences in a transcript. Requirements can be un-verified when new evidence contradicts them, which a checked-off list item never does. That’s also what makes this more than a benchmark trick: audit trails, resumable checkpoints, and human takeover points are what production automation teams keep asking agents for.
The verifier is the new weakest link
Now the honest part, and the reason to hold the applause. The authors audited 83 zero-score trajectories from the 27B run and assigned each a primary failing module. Actor failures account for 33%. Planner failures, 30%. Verifier failures, 30%, with environment blockers taking the rest. The single largest failure type across the audit is state verification itself: the system accepting weak evidence, missing a condition that was actually satisfied, or failing to notice that a later action invalidated earlier progress.
Read that again. In an architecture whose entire premise is “trust only the verifier,” the verifier is wrong roughly as often as the component doing the clicking. The gate works, the numbers show it works, and the gate is still the part that fails most in what remains. The bouncer occasionally accepts a fake ID.
Where the approach thins out is also mapped. On Mind2Web’s web tasks, gains are modest (52.3% overall versus 51.7% for the best reproduced baseline on the 27B backbone), because dynamic sites, pop-ups, and site-specific flows create grounding problems that state tracking alone doesn’t solve. Tasks whose success lives in visual layout, like GIMP edits or document formatting, resist evidence-based checking because there’s little durable state to probe. And cross-application tasks stay hardest for small backbones, where planning over dependencies, not progress tracking, is the bottleneck.
When to reach for this pattern
The selection rule falls out of the failure map. This pattern pays off when your agent’s tasks leave externally checkable traces: files written, database rows changed, URLs reached, shell commands whose output can be parsed, application state an API can read. If you can write an assertion about the world after each subgoal, you can gate state transitions on it.
It pays off least when success is a judgment call about pixels. If your task’s ground truth is “the slide looks right,” a structured verifier degenerates into the screenshot judge you already had.
If you adopt the pattern, evaluate it the way this paper does rather than by end-to-end success alone. Track the false-commit rate at the completion gate. Sample the cases where your verifier and a visual check disagree and review them by hand; the 63/31 split above is exactly the measurement that tells you whether your evidence sources are earning their keep. And treat test-time scaling as a later add-on: the paper’s best-of-N experiments improve results, but most of the gain arrives by three rollouts, and none of it substitutes for the single-rollout structure.
The paper opens with a Dijkstra line about abstraction, that its purpose is “to create a new semantic level in which one can be absolutely precise.” That’s the real shift on offer. For two years, the field’s default answer to unreliable agents has been a bigger model. These results argue the first question is different: what does your agent know for sure right now, and how does it know it? Answer that with a data structure instead of a transcript, and the model you already have may be better than you think.
References and Further Reading
StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structure. Wu, Zhu, Zhou, Salvi, Song, Huang. arXiv, July 13, 2026. The primary paper behind this article.
StructAgent project page. Demos, figures, and result summaries.
StructAgent code. The open-source implementation.
OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. Xie et al., 2024. The benchmark family used for the main results.
OSWorld-Verified. The verified split and public leaderboard referenced throughout.
ReAct: Synergizing Reasoning and Acting in Language Models. Yao et al., 2022. The loop most agent harnesses still descend from.
Mind2Web: Towards a Generalist Agent for the Web. Deng et al., 2023. The web benchmark used in the generalization study.
Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks. Li et al., 2024. The Minecraft baseline StructAgent is compared against.