JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
Zixiang Chen , Yuheng Lu , Zihao Cheng , Zeming Liu , Jizeng Bai , Ziye Huang , Zhiyin Lin , Zihan Li , Yuhang Guo , Yunhong Wang , Haifeng Wang
- 🏛 Institutions
- Beihang University , Beijing Institute of Technology , Baidu Inc.
- 📅 Date
- September 9, 2026
- 📑 Publisher
- arXiv
- 💻 Env
- Mobile Desktop
- 🔑 Keywords
TLDR
JarvisGUI is a dynamic benchmark for GUI agents on cross-device workflows spanning Android, Windows, and Ubuntu, formulating tasks as typed input-output transformations so multi-step cross-device workflows can be composed and evaluated automatically.
Related papers (24)
- MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer EnvironmentsOctober 14, 2026 · arXiv
- nanoMuse: An Open-Source Personal Agent for Every Device You OwnOctober 6, 2026 · arXiv
- RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use AgentsSeptember 18, 2026 · arXiv
- ADeptS-Bench: Measuring the Trustworthiness of Computer Use Agents Across DevicesAugust 25, 2026 · arXiv
- OSReward: Instituting Standardized Evaluation for Cross-Platform Computer-Use Reward ModelsJuly 30, 2026 · arXiv
- PIRA-Bench: A Transition from Reactive GUI Agents to GUI-based Proactive Intent Recommendation AgentsMarch 9, 2026 · arXiv
- VenusBench-GD: A Comprehensive Multi-Platform GUI Benchmark for Diverse Grounding TasksDecember 18, 2025 · arXiv
- OS-Oracle: A Comprehensive Framework for Cross-Platform GUI Critic ModelsDecember 18, 2025 · arXiv
- NaturalGAIA: Pushing the Frontiers of GUI Agents with a Challenging Benchmark and High-Quality Trajectory DatasetAugust 2, 2025 · arXiv
- On the Robustness of GUI Grounding Models Against Image AttacksApril 7, 2025 · arXiv
- CRAB: Cross-environment Agent Benchmark for Multimodal Language Model AgentsJuly 1, 2024 · Findings of ACL 2025
- GUI-World: A Video Benchmark and Dataset for Multimodal GUI-oriented UnderstandingJune 16, 2024 · ICLR 2025 (Poster)
- SeeClick: Harnessing GUI Grounding for Advanced Visual GUI AgentsJanuary 17, 2024 · ACL 2024
- iPrefBench: A User-Centric Benchmark for Personalized Mobile Use AgentsOctober 26, 2026 · arXiv
- AndroidFlux: Evaluating Failure Recovery and Recovery Action Reward Modeling in Mobile-Use Multimodal AgentsOctober 25, 2026 · arXiv
- CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use AgentsOctober 20, 2026 · arXiv
- CUA-Lite: An Open Platform for Computer Use AgentsOctober 19, 2026 · arXiv
- PersonaCUABench: Benchmarking Computer-Use Agents Serving Diverse PersonasOctober 16, 2026 · arXiv
- MobileJailBench: Red Queen Jailbreak Benchmarking for Mobile AgentsOctober 16, 2026 · arXiv
- MobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency InjectionOctober 16, 2026 · arXiv
- Not All Graphs Are Maps: A Task-Agnostic Exploration Framework for Mobile GUI NavigationOctober 15, 2026 · arXiv
- How GUI Agents Fail: GraceBench for Execution Stability DiagnosisOctober 14, 2026 · arXiv
- A Decade in a Day: Synthesizing Agent-Native Interfaces from Human-Oriented Software EvolutionOctober 11, 2026 · arXiv
- OpenMobile-2: Building Versatile Mobile Agents with Scalable Environments and App-Native ToolsOctober 9, 2026 · arXiv