Generalist Virtual Agents: A Survey on Autonomous Agents Across Digital Platforms
Minghe Gao , Wendong Bu , Bingchen Miao , Yang Wu , Yunfei Li , Juncheng Li , Siliang Tang , Qi Wu , Yueting Zhuang , Meng Wang
- 🏛 Institutions
- ZJU , Ant Group , University of Adelaide , Hefei University of Technology
- 📅 Date
- November 17, 2024
- 📑 Publisher
- arXiv
- 💻 Env
- Desktop Mobile Web
- 🔑 Keywords
TLDR
Surveys Generalist Virtual Agents as autonomous agents that operate across multiple digital platforms rather than a single interface. The paper traces the evolution of these agents, organizes prior work by environments, tasks, and capabilities, and highlights realistic evaluation and long-horizon decision-making as key open problems.
Related papers (24)
- GUI Agents with Foundation Models: A Comprehensive SurveyNovember 7, 2024 · arXiv
- Learning How to Remember: Memory Modality Selection for GUI AgentsOctober 13, 2026 · arXiv
- Adaptive-Group Process Rewards for GUI AgentsOctober 13, 2026 · arXiv
- PrivacyGuard: Source-Linked Sanitization and Locally Resolvable PII References for Cloud GUI AgentsOctober 12, 2026 · arXiv
- Latent World Action Model for GUI AgentsOctober 9, 2026 · arXiv
- Right Screen, Wrong Transition: World Models as Verifiers for GUI AgentsOctober 8, 2026 · arXiv
- Secure-CUA: Controlling Untrusted Influence in Computer-Use AgentsOctober 7, 2026 · arXiv
- Imagine to Act: High-Fidelity Data Synthesis via Image Editing World Model for Scalable GUI Agent TrainingOctober 5, 2026 · arXiv
- RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use AgentsSeptember 18, 2026 · arXiv
- MintAct: A Unified Visual Agent for Digital EnvironmentsSeptember 18, 2026 · arXiv
- LLaDA-UI: Bringing Block-wise Diffusion to Vision-Language GUI AgentsSeptember 9, 2026 · arXiv
- MAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action DistillationJuly 31, 2026 · arXiv
- Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI AgentsJuly 30, 2026 · arXiv
- OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward ModelsJuly 30, 2026 · arXiv
- Mobile-Agent-v3.5: Multi-platform Fundamental GUI AgentsFebruary 15, 2026 · arXiv
- Mapping the Design Space of User Experience for Computer Use AgentsFebruary 7, 2026 · IUI 2026
- VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding TasksDecember 18, 2025 · arXiv
- OS-Oracle: A Comprehensive Framework for Cross-Platform GUI Critic ModelsDecember 18, 2025 · arXiv
- Surfer 2: The Next Generation of Cross-Platform Computer Use AgentsOctober 22, 2025 · arXiv
- ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform DataSeptember 18, 2025 · ICLR 2026 (Oral)
- UI-Venus Technical Report: Building High-performance UI Agents with RFTAugust 14, 2025 · arXiv
- Test‑Time Reinforcement Learning for GUI Grounding via Region ConsistencyAugust 7, 2025 · AAAI 2026
- GuirlVG: Incentivize GUI Visual Grounding via Empirical Exploration on Reinforcement LearningAugust 6, 2025 · ICLR 2026 (Poster)
- NaviMaster: Learning a Unified Policy for GUI and Embodied Navigation TasksAugust 4, 2025 · arXiv