From Interaction Traces to Persistent Skills: Online Evolution for Computer-Use Agents
Longtao Hu , Xiao Liang , Linchao Zhu
- 🏛 Institutions
- University of Electronic Science and Technology of China , Zhejiang University
- 📅 Date
- September 4, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- Desktop General GUI
- 🔑 Keywords
TLDR
This work converts computer-use-agent trajectories and evaluator feedback into a persistent, versioned library of reusable procedures through online skill evolution, running each iteration against a frozen library snapshot to isolate the library's incremental value.
Related papers (24)
- CONTRAMEM: Learning Self-Evolving Procedural Memory from Contrasting Multi-Model TrajectoriesAugust 23, 2026 · arXiv
- CUA-Skill: Develop Skills for Computer Using AgentJanuary 28, 2026 · arXiv
- FinCUABuild: Can Agents Build Reliable Benchmarks for Dynamic Financial Computer Use?September 7, 2026 · arXiv
- AgentHijack: Visual Patch Attacks on Multimodal Computer-Use AgentsSeptember 6, 2026 · arXiv
- SIR: Self-improving Red-teaming for Compute Use AgentsAugust 31, 2026 · arXiv
- MAGNET: Towards Adaptive GUI Agents with Memory-Driven Knowledge EvolutionJanuary 27, 2026 · arXiv
- VRL-Bench: Benchmarking agents on computer control tasks under finite trial budgetsSeptember 11, 2026 · arXiv
- TRACE: Trajectory-robust Admission with Evidence Ordering for Efficient GUI AgentsSeptember 9, 2026 · arXiv
- JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task CompositionSeptember 9, 2026 · arXiv
- Selective Knowledge Control for Continual GUI Agent Learning over Application StreamsSeptember 6, 2026 · arXiv
- CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI AgentsSeptember 4, 2026 · arXiv
- Do GUI Agents Know When Not to Act? Enabling Conflict-Aware Termination for Multimodal GUI AgentsSeptember 3, 2026 · arXiv
- OmegaUse-SOP: SOP Engineering for Professional Computer Use from Human DemonstrationsSeptember 2, 2026 · arXiv
- Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime OptimizationSeptember 2, 2026 · arXiv
- Are We There Yet? Assessing Computer-Use Agents for Blind Users' Accessible Interaction with Desktop ApplicationsSeptember 1, 2026 · arXiv
- Iron: Intent-Aligned and Retrospective Dual Learning Framework for Enhancing Generalist Virtual AgentsAugust 28, 2026 · arXiv
- CURA: Certified Runtime Alarms for Computer-Use AgentsAugust 28, 2026 · arXiv
- UI-Venus-2 Technical ReportAugust 27, 2026 · arXiv
- ASIL: Replacing Screenshot-and-Click with Structured State and Semantic ActionsAugust 27, 2026 · arXiv
- LocalLSTC: A Long Short-Term Control Architecture for Locally Deployed GUI AgentsAugust 26, 2026 · arXiv
- Task-Adaptive Rubrics for GUI Reward ModelingAugust 25, 2026 · arXiv
- Reflection with Action-Induced Visual Differences for Desktop GUI AgentsAugust 25, 2026 · arXiv
- ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across DevicesAugust 25, 2026 · arXiv
- CausalCache: Conditional High-Fidelity Restoration for Long-Horizon GUI AgentsAugust 23, 2026 · arXiv