Security Risks in Computer-Using Agents

The field of computer-using agents is rapidly advancing, with agents becoming increasingly sophisticated and capable of autonomously performing complex tasks. However, this growth also introduces novel safety and security risks, including vulnerabilities in large language models and the integration of multiple software components. Researchers are working to address these risks through the development of comprehensive taxonomies of defensive strategies and the evaluation of agent safety and performance. A key area of focus is the threat of backdoor attacks, which can be used to manipulate agent behavior and compromise security.

Noteworthy papers include:

  • A Survey on the Safety and Security Threats of Computer-Using Agents, which provides a systematization of knowledge on the safety and security threats of computer-using agents.
  • EVA: Red-Teaming GUI Agents via Evolving Indirect Prompt Injection, which proposes a red teaming framework for indirect prompt injection attacks.
  • Hidden Ghost Hand: Unveiling Backdoor Vulnerabilities in MLLM-Powered Mobile GUI Agents, which introduces a framework for red-teaming backdoor attacks on mobile GUI agents.
  • BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization, which exposes the backdoor vulnerabilities of vision-language-action models.

Sources

A Survey on the Safety and Security Threats of Computer-Using Agents: JARVIS or Ultron?

EVA: Red-Teaming GUI Agents via Evolving Indirect Prompt Injection

Hidden Ghost Hand: Unveiling Backdoor Vulnerabilities in MLLM-Powered Mobile GUI Agents

BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization

Built with on top of