Source URL: https://www.schneier.com/blog/archives/2025/09/indirect-prompt-injection-attacks-against-llm-assistants.html
Source: Schneier on Security
Title: Indirect Prompt Injection Attacks Against LLM Assistants
Feedly Summary: Really good research on practical attacks against LLM agents.
“Invitation Is All You Need! Promptware Attacks Against LLM-Powered Assistants in Production Are Practical and Dangerous”
Abstract: The growing integration of LLMs into applications has introduced new security risks, notably known as Promptware—maliciously engineered prompts designed to manipulate LLMs to compromise the CIA triad of these applications. While prior research warned about a potential shift in the threat landscape for LLM-powered applications, the risk posed by Promptware is frequently perceived as low. In this paper, we investigate the risk Promptware poses to users of Gemini-powered assistants (web application, mobile application, and Google Assistant). We propose a novel Threat Analysis and Risk Assessment (TARA) framework to assess Promptware risks for end users. Our analysis focuses on a new variant of Promptware called Targeted Promptware Attacks, which leverage indirect prompt injection via common user interactions such as emails, calendar invitations, and shared documents. We demonstrate 14 attack scenarios applied against Gemini-powered assistants across five identified threat classes: Short-term Context Poisoning, Permanent Memory Poisoning, Tool Misuse, Automatic Agent Invocation, and Automatic App Invocation. These attacks highlight both digital and physical consequences, including spamming, phishing, disinformation campaigns, data exfiltration, unapproved user video streaming, and control of home automation devices. We reveal Promptware’s potential for on-device lateral movement, escaping the boundaries of the LLM-powered application, to trigger malicious actions using a device’s applications. Our TARA reveals that 73% of the analyzed threats pose High-Critical risk to end users. We discuss mitigations and reassess the risk (in response to deployed mitigations) and show that the risk could be reduced significantly to Very Low-Medium. We disclosed our findings to Google, which deployed dedicated mitigations…
AI Summary and Description: Yes
Summary: The text discusses critical research on security vulnerabilities associated with LLM (Large Language Model)-powered applications, particularly focusing on a threat type called Promptware. This represents a significant concern for security professionals as it exposes various risks and attack scenarios that could compromise user data and application integrity.
Detailed Description: The research outlined in the text emphasizes the emerging security threats linked to LLMs, specifically the concept of Promptware, which comprises maliciously crafted prompts that can exploit these models. The significance of this research lies in its practical implications for both developers and users of LLM-powered systems.
– **Key Findings**:
– LLM integration into applications has elevated security risks, primarily through Promptware.
– Promptware can manipulate LLMs, thereby jeopardizing the CIA (Confidentiality, Integrity, Availability) triad within applications.
– The research presents a Threat Analysis and Risk Assessment (TARA) framework, assisting in the evaluation of Promptware risks.
– A novel category, Targeted Promptware Attacks, uses indirect prompt injection through common user interactions like emails and shared documents.
– **Attack Scenarios Under Investigation**:
– **Short-term Context Poisoning**: Compromising an LLM’s short-term memory leading to unintended actions.
– **Permanent Memory Poisoning**: A longer-term compromise affecting an LLM’s knowledge.
– **Tool Misuse**: Exploiting the tools that LLMs interface with for malicious gains.
– **Automatic Agent Invocation**: Triggering actions without user consent or knowledge.
– **Automatic App Invocation**: Compromising the functioning of other applications via LLM manipulation.
– **Consequences of Attacks**: The study outlines various potential outcomes, including:
– **Spamming and Phishing**: Unsolicited messages misleading users.
– **Disinformation Campaigns**: Dissemination of harmful or false information.
– **Data Exfiltration**: Unauthorized data retrieval.
– **Home Automation Control**: Risking physical safety by commandeering connected devices.
– **Risk Assessment**: The findings reveal that about 73% of the examined threats pose a High-Critical risk to end users. Mitigation strategies were discussed, with the potential to reduce risks significantly.
– **Call for Fundamental Changes in LLM Security**: The text highlights that prompt injection isn’t a mere security issue but an intrinsic flaw in current LLM technologies. The research emphasizes a need for new scientific approaches to address these challenges effectively.
This investigation presents a compelling argument for the increased attention that security and compliance professionals must pay to the evolving landscape of LLM threats, advocating for enhanced security measures and industry awareness.