ClickAgent: Enhancing UI Location Capabilities of Autonomous Agents

Jakub Hoscilowicz, Bartosz Maj, Bartosz Kozakiewicz, Oleksii Tymoshchuk, Artur Janicki

🏛 Institutions: Samsung R&D Poland, Warsaw University of Technology
📅 Date: October 9, 2024
📑 Publisher: SIGDIAL 2025
💻 Env: Mobile
🔑 Keywords: framework GUI grounding AITW ClickAgent mobile control

TLDR

Proposes ClickAgent, a mobile agent framework that separates high-level reasoning from precise UI element localization. By pairing an MLLM planner with a dedicated grounding component, it improves task success on AITW and on real-device Android evaluations.

Open paper Edit on GitHub Report issue

Related papers

You Only Look at Screens: Multimodal Chain-of-Action Agents

September 20, 2023 · Findings of ACL 2024
OpeFlo: Automated UX Evaluation via Simulated Human Web Interaction with GUI Grounding

February 25, 2026 · arXiv
ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agents

April 13, 2026 · arXiv
GraphPilot: GUI Task Automation with One-Step LLM Reasoning Powered by Knowledge Graph

January 24, 2026 · Journal of Intelligent Computing and Networking
GUITester: Enabling GUI Agents for Exploratory Defect Discovery

January 8, 2026 · arXiv
VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding Tasks

December 18, 2025 · arXiv