Chart the Invisible: Interactive Perception for Hidden-Target Exploration in GUI Agents
Yongxiang Hu , Ruiyang Liu , Qining He , Xuan Wang , Yangfan Zhou
- 🏛 Institutions
- Fudan
- 📅 Date
- October 26, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- Mobile
- 🔑 Keywords
TLDR
Presents Hagasu, a multimodal model for scrollable-region perception that helps mobile GUI agents detect hidden elements and execute targeted swipe exploration, boosting task completion from 64.8% to 82.7%.
Related papers (24)
- Online Dense Reinforcement Learning with Planning-Grounding Agent for UI Navigation on Live IOS AppsOctober 29, 2026 · arXiv
- GUI-JEPA: Candidate-Conditioned JEPA Latents as a Fast Action Policy for Mobile GUI AgentsOctober 27, 2026 · arXiv
- iPrefBench: A User-Centric Benchmark for Personalized Mobile Use AgentsOctober 26, 2026 · arXiv
- AndroidFlux: Evaluating Failure Recovery and Recovery Action Reward Modeling in Mobile-Use Multimodal AgentsOctober 25, 2026 · arXiv
- Tandem: Serving Hybrid Linear-Attention GUI Agents on Mobile NPUsOctober 19, 2026 · arXiv
- MobileJailBench: Red Queen Jailbreak Benchmarking for Mobile AgentsOctober 16, 2026 · arXiv
- MobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency InjectionOctober 16, 2026 · arXiv
- Not All Graphs Are Maps: A Task-Agnostic Exploration Framework for Mobile GUI NavigationOctober 15, 2026 · arXiv
- How GUI Agents Fail: GraceBench for Execution Stability DiagnosisOctober 14, 2026 · arXiv
- OpenMobile-2: Building Versatile Mobile Agents with Scalable Environments and App-Native ToolsOctober 9, 2026 · arXiv
- Subgoal-aware Memory Graph for GUI AgentOctober 22, 2026 · arXiv
- Reuse, Rewrite, or Reject: Evidence-Constrained Trace Reuse for Repeated Mobile GUI ControlOctober 22, 2026 · arXiv
- Diagnosing Personalization Failures in Mobile GUI AgentsOctober 22, 2026 · arXiv
- ScopeMem: Latent Memory That Learns Where to Revise Frozen GUI AgentsOctober 19, 2026 · arXiv
- Beyond Next-Screen Prediction: The Decision Value of GUI World ModelsOctober 17, 2026 · arXiv
- MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer EnvironmentsOctober 14, 2026 · arXiv
- Learning How to Remember: Memory Modality Selection for GUI AgentsOctober 13, 2026 · arXiv
- Adaptive-Group Process Rewards for GUI AgentsOctober 13, 2026 · arXiv
- Seg2Click: Learning Pixel-Level Action Masks for GUI Visual GroundingOctober 12, 2026 · arXiv
- PrivacyGuard: Source-Linked Sanitization and Locally Resolvable PII References for Cloud GUI AgentsOctober 12, 2026 · arXiv
- Latent World Action Model for GUI AgentsOctober 9, 2026 · arXiv
- ConClick: Element-Wise Contrastive Learning Improves GUI GroundingOctober 9, 2026 · arXiv
- Right Screen, Wrong Transition: World Models as Verifiers for GUI AgentsOctober 8, 2026 · arXiv
- DeltaReplay: Task-Relative Memory Reuse for Mobile GUI AgentsOctober 8, 2026 · arXiv