Lost in Recovery: Demystifying Recovery Behavior of Computer-Use Agents via Goal-Oriented Action Replay
Byeonghun Bang , Sangwoo Park , Jinheon Baek , Kangsan Kim , Youngwan Lee , Sung Ju Hwang
- 🏛 Institutions
- KAIST , ETRI , DeepAuto.ai
- 📅 Date
- October 20, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- General GUI
- 🔑 Keywords
TLDR
ReTrace evaluates computer-use agent recovery through goal-oriented action replay under controlled error injection, operationalizing transition-consistent recovery to analyze autonomous error mitigation.
Related papers (24)
- CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use AgentsOctober 20, 2026 · arXiv
- Step-level Optimization for Efficient Computer-use AgentsApril 29, 2026 · arXiv
- BEAP-Agent: Backtrackable Execution and Adaptive Planning for GUI AgentsJanuary 29, 2026 · arXiv
- InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative ReasonersApril 19, 2025 · arXiv
- What Does "Now" Mean to Your Computer-Use Agent? Stale Screenshots, Page Loads, and Which Check to Run Before a ClickOctober 29, 2026 · arXiv
- Interaction-as-a-JudgeOctober 13, 2026 · arXiv
- SoftwareGym: Scaling Computer-Use Agent Training with Diverse Synthesized Software EnvironmentsOctober 8, 2026 · arXiv
- Secure-CUA: Controlling Untrusted Influence in Computer-Use AgentsOctober 7, 2026 · arXiv
- PlaySuite: A Large-Scale Benchmark for Interactive Visual IntelligenceOctober 5, 2026 · arXiv
- Control OSWorld: An AI Control Environment for GUI Computer Use AgentsOctober 2, 2026 · arXiv
- Xiaomi-GUI-0 Technical ReportJune 30, 2026 · arXiv
- Agent-SAMA: State-Aware Mobile AssistantMay 29, 2025 · AAAI 2026
- Where to Learn and How Far to Roll Out: Efficient Adaptive Training for GUI AgentsOctober 29, 2026 · arXiv
- Right Click, Wrong Box: Grading GUI Grounding by What a Click DoesOctober 27, 2026 · arXiv
- Learning to Look Beyond Screens: Transferable Visual Observation from GUI GroundingOctober 27, 2026 · arXiv
- From Interaction to Capability: Task Advancement as State and Supervision for GUI AgentsOctober 27, 2026 · arXiv
- AdaGround: Token-Normalized Adaptive Zooming with Verifier-in-the-Loop Refinement for GUI GroundingOctober 27, 2026 · arXiv
- GUIFlow: Temporal Redundancy-Aware Incremental Inference for GUI AgentsOctober 26, 2026 · arXiv
- PreView: Enhancing GUI Grounding with Preprocessed Views and CuesOctober 24, 2026 · arXiv
- Build or Adopt? Custom-Built vs. General-Purpose LLM Agent Harnesses for Automotive GUI TestingOctober 24, 2026 · arXiv
- IGG: A Benchmark for Interactive GUI Grounding under Visibility ConstraintsOctober 23, 2026 · arXiv
- Correct Answers, Closed Gates: SolveBind for GUI Agents at the Verification BarrierOctober 23, 2026 · arXiv
- Trace2World: Reconstructing App Worlds from Real Interaction Traces for Scalable GUI Agent TrainingOctober 22, 2026 · arXiv
- PointerBench: A Diagnostic Benchmark for Fine-Grained GUI GroundingOctober 22, 2026 · arXiv