Reflect, Revise, Reuse: Training-Free Skill Evolution for GUI Agents
Bofan Chen , Boxuan Zhang , Fei Tang , Zhengxi Lu , Yong Du , Tongbo Chen , Weiming Lu , Jun Xiao , Yueting Zhuang , Yongliang Shen
- 🏛 Institutions
- ZJU , UESTC
- 📅 Date
- September 15, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- Mobile Desktop
- 🔑 Keywords
TLDR
EvoSkill-GUI makes GUI agent skills revisable at deployment time instead of fixed beforehand, representing each skill as a multi-file package holding retrieval metadata, executable plans, backup localization, failure-recovery rules, and failure cases. A reflect-revise-reuse loop lets the executor patch skills mid-rollout while an isolated critic diagnoses failed trajectories, improving multiple base models without any training by up to 16.2% on MobileWorld, 6.0% on AndroidWorld, and 10.5% on OSWorld.
Related papers (24)
- Surfer 2: The Next Generation of Cross-Platform Computer Use AgentsOctober 22, 2025 · arXiv
- AndroidFlux: Evaluating Failure Recovery and Recovery Action Reward Modeling in Mobile-Use Multimodal AgentsOctober 25, 2026 · arXiv
- VeriCUA: Learning Proactive Verification for Reliable GUI Agent RewardsOctober 19, 2026 · arXiv
- Adaptation of Computer Use Agents to New Software with User ManualsOctober 16, 2026 · arXiv
- OpenGCUA: Toward a Fully Open Stack for Generalist Computer UseOctober 15, 2026 · arXiv
- OS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using AgentsOctober 15, 2026 · arXiv
- Not All Graphs Are Maps: A Task-Agnostic Exploration Framework for Mobile GUI NavigationOctober 15, 2026 · arXiv
- LEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use AgentsOctober 15, 2026 · arXiv
- Task-State Governance: A Virtual Machine for Verifiable and Recoverable GUI Agent ExecutionOctober 14, 2026 · arXiv
- A Decade in a Day: Synthesizing Agent-Native Interfaces from Human-Oriented Software EvolutionOctober 11, 2026 · arXiv
- EchoPath: Execution-Level Replayable Memory for GUI AgentsSeptember 15, 2026 · arXiv
- RSIAgent: Autonomous Exploration for Recursive Self-improvement in New EnvironmentsSeptember 14, 2026 · arXiv
- GraphDroid: Asynchronous LLM-Based Mobile App GUI Testing via History-Aware Exploration and Hybrid Intent FulfillmentSeptember 9, 2026 · arXiv
- Improving Proficiency and Efficiency of Android GUI Agents via Self-Generating Tool ActionsSeptember 6, 2026 · arXiv
- CUA-Universe: A Scalable and Dynamic Environment for Hybrid GUI+CLI AgentsSeptember 4, 2026 · arXiv
- OmegaUse-SOP: SOP Engineering for Professional Computer Use from Human DemonstrationsSeptember 2, 2026 · arXiv
- ASIL: Replacing Screenshot-and-Click with Structured State and Semantic ActionsAugust 27, 2026 · arXiv
- StructAgent: Harness Long-horizon Digital Agents with Unified Causal StructureJuly 13, 2026 · arXiv
- AOHP: An Open-Source OS-Level Agent Harness for Personalized, Efficient and Secure InteractionJune 22, 2026 · arXiv
- MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent ResearchMay 25, 2026 · arXiv
- ScaleWoB: Guiding GUI Agents with Coding Agents via Large-Scale Environmental SynthesisMay 24, 2026 · arXiv
- VLAA-GUI: Knowing When to Stop, Recover, and Search, A Modular Framework for GUI AutomationApril 23, 2026 · arXiv
- ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI AgentsApril 13, 2026 · arXiv
- EE-MCP: Self-Evolving MCP-GUI Agents via Automated Environment Generation and Experience LearningApril 10, 2026 · arXiv