From Interaction to Capability: Task Advancement as State and Supervision for GUI Agents
Ziyang Chen , Qian Li , Chengwei Qin
- 🏛 Institutions
- HKUST , Shandong University
- 📅 Date
- October 27, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- General GUI
- 🔑 Keywords
TLDR
Introduces Task Advancement Memory (TAM), defining task progress states as dense supervision to help GUI agents retain valuable operation milestones and avoid cascading errors in cross-application workflows.
Related papers (24)
- ScopeMem: Latent Memory That Learns Where to Revise Frozen GUI AgentsOctober 19, 2026 · arXiv
- PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience RetrievalSeptember 29, 2026 · arXiv
- CausalCache: Conditional High-Fidelity Restoration for Long-Horizon GUI AgentsAugust 23, 2026 · arXiv
- Probe to Act: Elevating Browser-Use Agent via Active Visual ProbingSeptember 27, 2026 · Findings of EMNLP 2026
- Where to Learn and How Far to Roll Out: Efficient Adaptive Training for GUI AgentsOctober 29, 2026 · arXiv
- Focused Omni-Modal GUI AgentsOctober 19, 2026 · arXiv
- Not All Experience Belongs in the Weights: Component Routing for Self-Improving GUI AgentsOctober 1, 2026 · arXiv
- LongPuzzleBench: Evaluating GUI Agents on Long-Horizon Visual PuzzlesSeptember 28, 2026 · arXiv
- Relevance Does Not Imply Applicability: Experience Activation for Personal GUI AgentsSeptember 27, 2026 · arXiv
- Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime OptimizationSeptember 2, 2026 · arXiv
- MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI AgentsMay 18, 2026 · arXiv
- Executable Agentic Memory for GUI AgentMay 12, 2026 · arXiv
- CocoaBench: Evaluating Unified Digital Agents in the WildApril 13, 2026 · arXiv
- LongHorizonUI: A Unified Framework for Robust long-horizon Task Automation of GUI AgentJanuary 26, 2026 · ICLR 2026 (Poster)
- Training High-Level Schedulers with Execution-Feedback Reinforcement Learning for Long-Horizon GUI AutomationNovember 27, 2025 · CVPR 2026
- Subgoal-aware Memory Graph for GUI AgentOctober 22, 2026 · arXiv
- CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use AgentsOctober 20, 2026 · arXiv
- Learning How to Remember: Memory Modality Selection for GUI AgentsOctober 13, 2026 · arXiv
- Adaptive-Group Process Rewards for GUI AgentsOctober 13, 2026 · arXiv
- Work2Skill: Capturing Human GUI Experience for Reusable Agent Skill InductionOctober 9, 2026 · arXiv
- Latent World Action Model for GUI AgentsOctober 9, 2026 · arXiv
- DeskForge: Dense Supervision from Desktop Environments for Computer-Use AgentsOctober 1, 2026 · arXiv
- Auditing Web Agent Evaluation on WebArena-Lite: Human Review of Outcomes and TrajectoriesOctober 1, 2026 · NeurIPS 2026 Workshop on Who Verifies the Agents? Toward Reliable Agent Development
- Action Conditioned Bisimulation For GUI Agent MemorySeptember 30, 2026 · arXiv