Absorbed in Inertia: Activation Analysis for Computer-Use Agents
Giulio Segalini , Zhi Wen Soi , Jérémie Decouchant , Lydia Chen
- 🏛 Institutions
- University of Neuchâtel , Delft University of Technology
- 📅 Date
- September 29, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- Desktop
- 🔑 Keywords
TLDR
Shows that computer-use agents can exhibit inertia, repeating fruitless actions while recognizing that they are ineffective, and that inertia corresponds to an absorbing region of the underlying model's activation space. Proposes R3 (Reset, Reroute, Restore), which temporarily resets the context trajectory and then restores it, lowering measured inertia by 17-55% across models.
Related papers (24)
- GPA: Learning GUI Process Automation from DemonstrationsApril 2, 2026 · arXiv
- GUIDE: Resolving Domain Bias in GUI Agents through Real-Time Web Video Retrieval and Plug-and-Play AnnotationMarch 27, 2026 · arXiv
- Improved GUI Grounding via Iterative NarrowingNovember 18, 2024 · arXiv
- How GUI Agents Fail: GraceBench for Execution Stability DiagnosisOctober 14, 2026 · arXiv
- MHVF: Training-Free GUI Grounding via Verification-Guided Multi-Hypothesis SearchOctober 13, 2026 · arXiv
- WebFovea: When the Model Is Right but the Click Is Wrong -- Reliable Round Trips for Vision-Based Web Agents on Live WebsitesOctober 2, 2026 · arXiv
- Auditing Web Agent Evaluation on WebArena-Lite: Human Review of Outcomes and TrajectoriesOctober 1, 2026 · NeurIPS 2026 Workshop on Who Verifies the Agents? Toward Reliable Agent Development
- In-Context Teacher–Student Guidance for Open-Weight Browser AgentsSeptember 2026 · KDMiLe 2026
- Action Conditioned Bisimulation For GUI Agent MemorySeptember 30, 2026 · arXiv
- Same Tasks, Different Apps: Why Mobile GUI Agents Fail to Generalize?September 28, 2026 · Findings of EMNLP 2026
- PhoneCLI: From App Interfaces to Callable Commands for Mobile AgentsSeptember 28, 2026 · arXiv
- GUITAR: Structured Failure Diagnosis of GUI Agents via State TransitionsSeptember 28, 2026 · arXiv
- Relevance Does Not Imply Applicability: Experience Activation for Personal GUI AgentsSeptember 27, 2026 · arXiv
- Constructing Challenging Browser-Use Tasks by Controlled Environment InterventionsSeptember 20, 2026 · arXiv
- STaR-KV: Spatio-Temporal Adaptive Re-weighting for KV Cache Compression in GUI Vision-Language ModelsJune 1, 2026 · arXiv
- UI-Zoomer: Uncertainty-Driven Adaptive Zoom-In for GUI GroundingApril 15, 2026 · arXiv
- Zoom to Essence: Trainless GUI Grounding by Inferring upon Interface ElementsMarch 15, 2026 · arXiv
- Why Do LLM-based Web Agents Fail? A Hierarchical Planning PerspectiveMarch 15, 2026 · arXiv
- M^2: Dual-Memory Augmentation for Long-Horizon Web Agents via Trajectory Summarization and Insight RetrievalFebruary 28, 2026 · arXiv
- Trifuse: Enhancing Attention-Based GUI Grounding via Multimodal FusionFebruary 6, 2026 · arXiv
- Darwinian Memory: A Training-Free Self-Regulating Memory System for GUI Agent EvolutionJanuary 30, 2026 · arXiv
- MVP: Multiple View Prediction Improves GUI GroundingDecember 9, 2025 · arXiv
- Zoom in, Click out: Unlocking and Evaluating the Potential of Zooming for GUI GroundingDecember 5, 2025 · arXiv
- WebATLAS: An LLM Agent with Experience-Driven Memory and Action SimulationOctober 26, 2025 · NeurIPS 2025 Workshop on Language Agents and World Models