ToolTok: Tool Tokenization for Efficient and Generalizable GUI Agents

ToolTok treats GUI operations as paths over learnable tool tokens with semantic anchoring and curriculum learning. This makes GUI agents more efficient and generalizable, reaching competitive performance with far less training data than other post-training methods.

• CocoaBench: Evaluating Unified Digital Agents in the Wild
  April 13, 2026 · arXiv
• Are GUI Agents Focused Enough? Automated Distraction via Semantic-level UI Element Injection
  April 9, 2026 · arXiv
• Autonomous Continual Learning of Computer-Use Agents for Environment Adaptation
  February 10, 2026 · arXiv
• iSHIFT: Lightweight Slow-Fast GUI Agent with Adaptive Perception
  December 26, 2025 · arXiv
• Visual Grounding for User Interfaces
  June 16, 2024 · NAACL 2024 Industry Track
• Seg2Click: Learning Pixel-Level Action Masks for GUI Visual Grounding
  October 12, 2026 · arXiv
• When Charts Deceive Web Agents: A Paired Benchmark for Misleading Visualization-Grounded Tasks
  October 9, 2026 · arXiv
• Adversarial Images Hijack Web Agents from Visual Grounding to Browser Execution
  October 7, 2026 · arXiv
• DashAct: A Progressive Diagnostic Benchmark for GUI Agents in Interactive Dashboard Analysis
  September 26, 2026 · arXiv
• Screenshots or Tools? Eliciting Tool Use and Managing Multimodal Context in Hybrid GUI-MCP Computer-Use Agents
  August 4, 2026 · arXiv
• The Amazing Agent Race: Strong Tool Users, Weak Navigators
  April 11, 2026 · arXiv
• The Tool Illusion: Rethinking Tool Use in Web Agents
  April 3, 2026 · arXiv
• Mobile-Agent-v3.5: Multi-platform Fundamental GUI Agents
  February 15, 2026 · arXiv
• Learning with Challenges: Adaptive Difficulty-Aware Data Generation for Mobile GUI Agent Training
  January 30, 2026 · arXiv
• How do Visual Attributes Influence Web Agents? A Comprehensive Evaluation of User Interface Design Factors
  January 29, 2026 · arXiv
• GUIPilot: A Consistency-Based Mobile GUI Testing Approach for Detecting Application-Specific Bugs
  June 9, 2025 · ISSTA 2025
• ReGUIDE: Data Efficient GUI Grounding via Spatial Reasoning and Search
  May 21, 2025 · arXiv
• GUI-R1: A Generalist R1-Style Vision-Language Action Model for GUI Agents
  April 14, 2025 · arXiv
• Iris: Breaking GUI Complexity with Adaptive Focus and Self-Refining
  December 13, 2024 · arXiv
• Dual-View Visual Contextualization for Web Navigation
  February 6, 2024 · CVPR 2024 (Poster)
• Where to Learn and How Far to Roll Out: Efficient Adaptive Training for GUI Agents
  October 29, 2026 · arXiv
• Right Click, Wrong Box: Grading GUI Grounding by What a Click Does
  October 27, 2026 · arXiv
• Learning to Look Beyond Screens: Transferable Visual Observation from GUI Grounding
  October 27, 2026 · arXiv
• From Interaction to Capability: Task Advancement as State and Supervision for GUI Agents
  October 27, 2026 · arXiv