GPA: Learning GUI Process Automation from Demonstrations
Zirui Zhao , Jun Hao Liew , Yan Yang , Wenzhuo Yang , Ziyang Luo , Doyen Sahoo , Silvio Savarese , Junnan Li
- 🏛 Institutions
- Salesforce AI Research
- 📅 Date
- April 2, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- Desktop
- 🔑 Keywords
TLDR
GPA is a vision-based GUI process automation system that enables fast and stable process replay from a single demonstration. Using Sequential Monte Carlo-based localization and readiness calibration, it achieves higher success rates with 10x faster execution than Gemini 3 Pro on long-horizon GUI tasks, running entirely locally without cloud LLMs.
Related papers (24)
- M^2: Dual-Memory Augmentation for Long-Horizon Web Agents via Trajectory Summarization and Insight RetrievalFebruary 28, 2026 · arXiv
- CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use AgentsOctober 20, 2026 · arXiv
- DeskForge: Dense Supervision from Desktop Environments for Computer-Use AgentsOctober 1, 2026 · arXiv
- Absorbed in Inertia: Activation Analysis for Computer-Use AgentsSeptember 29, 2026 · arXiv
- StructAgent: Harness Long-horizon Digital Agents with Unified Causal StructureJuly 13, 2026 · arXiv
- Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional FieldsJune 9, 2026 · arXiv
- HealthAdminBench: Evaluating Computer-Use Agents on Healthcare Administration TasksApril 10, 2026 · arXiv
- Gym-Anything: Turn any Software into an Agent EnvironmentApril 7, 2026 · arXiv
- GUIDE: Resolving Domain Bias in GUI Agents through Real-Time Web Video Retrieval and Plug-and-Play AnnotationMarch 27, 2026 · arXiv
- OS-Marathon: Benchmarking Computer-Use Agents on Long-Horizon Repetitive TasksJanuary 28, 2026 · arXiv
- Are LLM Agents the New RPA? A Comparative Study with RPA Across Enterprise WorkflowsSeptember 4, 2025 · arXiv
- Improved GUI Grounding via Iterative NarrowingNovember 18, 2024 · arXiv
- From Interaction to Capability: Task Advancement as State and Supervision for GUI AgentsOctober 27, 2026 · arXiv
- MHVF: Training-Free GUI Grounding via Verification-Guided Multi-Hypothesis SearchOctober 13, 2026 · arXiv
- In-Context Teacher–Student Guidance for Open-Weight Browser AgentsSeptember 2026 · KDMiLe 2026
- Action Conditioned Bisimulation For GUI Agent MemorySeptember 30, 2026 · arXiv
- PrecogUI: Proactive GUI Agents via Pre-cognitive Simulation and Experience RetrievalSeptember 29, 2026 · arXiv
- PhoneCLI: From App Interfaces to Callable Commands for Mobile AgentsSeptember 28, 2026 · arXiv
- LongPuzzleBench: Evaluating GUI Agents on Long-Horizon Visual PuzzlesSeptember 28, 2026 · arXiv
- Relevance Does Not Imply Applicability: Experience Activation for Personal GUI AgentsSeptember 27, 2026 · arXiv
- Probe to Act: Elevating Browser-Use Agent via Active Visual ProbingSeptember 27, 2026 · Findings of EMNLP 2026
- The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying KnowledgeSeptember 24, 2026 · Findings of EMNLP 2026
- CausalCache: Conditional High-Fidelity Restoration for Long-Horizon GUI AgentsAugust 23, 2026 · arXiv
- STaR-KV: Spatio-Temporal Adaptive Re-weighting for KV Cache Compression in GUI Vision-Language ModelsJune 1, 2026 · arXiv