Step-level Optimization for Efficient Computer-use Agents
Jinbiao Wei , Kangqi Ni , Yilun Zhao , Guo Gan , Arman Cohan
- 🏛 Institutions
- Unknown
- 📅 Date
- April 29, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- General GUI
- 🔑 Keywords
TLDR
This paper targets the cost and latency of computer-use agents that call large multimodal models at nearly every GUI step. It argues that long-horizon trajectories have heterogeneous step difficulty and studies step-level optimization so routine actions can be handled by cheaper policies while high-risk moments receive stronger compute.
Related papers (24)
- GUIFlow: Temporal Redundancy-Aware Incremental Inference for GUI AgentsOctober 26, 2026 · arXiv
- Lost in Recovery: Demystifying Recovery Behavior of Computer-Use Agents via Goal-Oriented Action ReplayOctober 20, 2026 · arXiv
- When to Compile a Computer-Use Agent? Measuring Payback and Making Compilation Decisions for Token EfficiencyOctober 2, 2026 · arXiv
- cua-speedrun: Standardized Benchmarking of the Speed of Computer-Use AgentsSeptember 30, 2026 · arXiv
- TRACE: Trajectory-robust Admission with Evidence Ordering for Efficient GUI AgentsSeptember 9, 2026 · arXiv
- Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime OptimizationSeptember 2, 2026 · arXiv
- STaR-KV: Spatio-Temporal Adaptive Re-weighting for KV Cache Compression in GUI Vision-Language ModelsJune 1, 2026 · arXiv
- What Does "Now" Mean to Your Computer-Use Agent? Stale Screenshots, Page Loads, and Which Check to Run Before a ClickOctober 29, 2026 · arXiv
- CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use AgentsOctober 20, 2026 · arXiv
- Tandem: Serving Hybrid Linear-Attention GUI Agents on Mobile NPUsOctober 19, 2026 · arXiv
- STP-GUI: Spatially-Aware Token Pruning for Efficient Web-GUI AgentsOctober 14, 2026 · arXiv
- Interaction-as-a-JudgeOctober 13, 2026 · arXiv
- SoftwareGym: Scaling Computer-Use Agent Training with Diverse Synthesized Software EnvironmentsOctober 8, 2026 · arXiv
- Secure-CUA: Controlling Untrusted Influence in Computer-Use AgentsOctober 7, 2026 · arXiv
- PlaySuite: A Large-Scale Benchmark for Interactive Visual IntelligenceOctober 5, 2026 · arXiv
- Control OSWorld: An AI Control Environment for GUI Computer Use AgentsOctober 2, 2026 · arXiv
- CUAWright: A Minimal Unified Interface for Digital AgentsOctober 2, 2026 · arXiv
- Neuro-Symbolic Computer Use: Learning Reusable Policies for Reliable and Efficient ExecutionSeptember 29, 2026 · arXiv
- PhoneCLI: From App Interfaces to Callable Commands for Mobile AgentsSeptember 28, 2026 · arXiv
- PastForward: Faster On-Device GUI Agents via Computational Experience ReuseSeptember 26, 2026 · arXiv
- CUA-Sandbox: Efficient Environments for Computer-Use Agent Reinforcement LearningSeptember 26, 2026 · arXiv
- Jev-Mobile: Jev as an Executor for Mobile GUI AgentsSeptember 24, 2026 · arXiv
- A11y-Compressor: A Framework for Enhancing the Efficiency of GUI Agent Observations through Visual Context Reconstruction and Redundancy ReductionMay 1, 2026 · arXiv
- STEP: Success-Rate-Aware Trajectory-Efficient Policy OptimizationNovember 17, 2025 · Findings of ACL 2026