GUI Agents Papers
Star · 921

The Verifiable Action Card: Trustworthy Human-in-the-Loop Control for Secure Autonomous Agents

Hasnain Irshad , Anam Mughees , Neelam Mughees , Abdullah Mughees , Imtiaz Ali Soomro

🏛 Institutions
Sir Syed CASE Institute of Technology , University of Engineering and Technology Lahore , National Textile University , KFUPM
📅 Date
September 16, 2026
📑 Publisher
arXiv
💻 Env
Web
🔑 Keywords
TLDR

The Verifiable Action Card reconstructs approval information from the ground-truth pending browser action and trusted intent provenance, renders it out-of-band in trusted browser chrome, and binds the approval to the exact action re-verified at dispatch. On a 24-scenario benchmark covering confused-deputy attacks, dialog forging, indirect prompt injection, and provenance evasion, attack success falls from 68-100% without the mechanism to 0% on every evaluated model, with 78% legitimate-task completion and no false blocks.

Open paper Report issue
Related papers (24)