Reliable Weak-to-Strong Monitoring of LLM Agents
Neil Kale , Chen Bo Calvin Zhang , Kevin Zhu , Ankit Aich , Paula Rodriguez , Scale Red Team , Christina Q. Knight , Zifan Wang
- 🏛 Institutions
- Scale AI , CMU , MIT
- 📅 Date
- August 26, 2025
- 📑 Publisher
- ICLR 2025
- 💻 Env
- 🔑 Keywords
TLDR
Stress-tests LLM agent monitoring systems for detecting covert misbehavior using a monitor red-teaming (MRT) workflow varying agent/monitor awareness and adversarial evasion strategies, evaluated on SHADE-Arena for tool-calling agents and CUA-SHADE-Arena for computer-use agents.
Related papers (10)
- Control OSWorld: An AI Control Environment for GUI Computer Use AgentsOctober 2, 2026 · arXiv
- SIR: Self-improving Red-teaming for Compute Use AgentsAugust 31, 2026 · arXiv
- Genesis: Evolving Attack Strategies for LLM Web Agent Red-TeamingOctober 21, 2025 · ICME 2026
- MobileJailBench: Red Queen Jailbreak Benchmarking for Mobile AgentsOctober 16, 2026 · arXiv
- MMA-SafetyBench: A Benchmark for Multimodal Agent Safety EvaluationOctober 14, 2026 · arXiv
- Adversarial Images Hijack Web Agents from Visual Grounding to Browser ExecutionOctober 7, 2026 · arXiv
- SCOUT: Synergizing Reasoning and Tool-Use for Computer-Use SafetySeptember 28, 2026 · arXiv
- Before Acting, Change the State: Prospective State Intervention for Web Agents under Deceptive InterfacesSeptember 28, 2026 · arXiv
- A Dual-Process Perspective on Nudge Susceptibility in LLM-Based GUI AgentsSeptember 17, 2026 · arXiv
- AgentHijack: Visual Patch Attacks on Multimodal Computer-Use AgentsSeptember 6, 2026 · arXiv