Source URL: https://www.schneier.com/blog/archives/2025/09/time-of-check-time-of-use-attacks-against-llms.html
Source: Schneier on Security
Title: Time-of-Check Time-of-Use Attacks Against LLMs
Feedly Summary: This is a nice piece of research: “Mind the Gap: Time-of-Check to Time-of-Use Vulnerabilities in LLM-Enabled Agents“.:
Abstract: Large Language Model (LLM)-enabled agents are rapidly emerging across a wide range of applications, but their deployment introduces vulnerabilities with security implications. While prior work has examined prompt-based attacks (e.g., prompt injection) and data-oriented threats (e.g., data exfiltration), time-of-check to time-of-use (TOCTOU) remain largely unexplored in this context. TOCTOU arises when an agent validates external state (e.g., a file or API response) that is later modified before use, enabling practical attacks such as malicious configuration swaps or payload injection. In this work, we present the first study of TOCTOU vulnerabilities in LLM-enabled agents. We introduce TOCTOU-Bench, a benchmark with 66 realistic user tasks designed to evaluate this class of vulnerabilities. As countermeasures, we adapt detection and mitigation techniques from systems security to this setting and propose prompt rewriting, state integrity monitoring, and tool-fusing. Our study highlights challenges unique to agentic workflows, where we achieve up to 25% detection accuracy using automated detection methods, a 3% decrease in vulnerable plan generation, and a 95% reduction in the attack window. When combining all three approaches, we reduce the TOCTOU vulnerabilities from an executed trajectory from 12% to 8%. Our findings open a new research direction at the intersection of AI safety and systems security…
AI Summary and Description: Yes
Summary: This research addresses significant vulnerabilities related to time-of-check to time-of-use (TOCTOU) in Large Language Model (LLM)-enabled agents. By introducing a benchmark called TOCTOU-Bench and proposing effective countermeasures, it aims to enhance the security frameworks within AI applications. This work has profound implications for AI security professionals, emphasizing the need to address new classes of vulnerabilities specific to LLMs.
Detailed Description:
The study titled “Mind the Gap: Time-of-Check to Time-of-Use Vulnerabilities in LLM-Enabled Agents” presents crucial insights into vulnerabilities inherent in LLM-enabled agents, particularly focusing on TOCTOU vulnerabilities. This research is of great relevance in the AI Security domain and carries significant implications for professionals working with AI and infrastructure security. Here are the major points discussed in the research:
– **Context**:
– LLM-enabled agents are increasingly used for various applications but have security concerns that have not been fully addressed, particularly TOCTOU.
– **Vulnerability Explanation**:
– TOCTOU vulnerabilities occur when an agent checks an external state (like a file or an API response) that may be altered before it can be used, opening pathways for attacks.
– Practical attacks that can exploit TOCTOU include malicious configuration swaps and payload injections.
– **Research Contributions**:
– This is the first study that investigates TOCTOU vulnerabilities within the context of LLM-enabled agents.
– Introduction of **TOCTOU-Bench**: A benchmark incorporating 66 realistic user tasks specifically designed to test this category of vulnerabilities.
– **Countermeasures Proposed**:
– The research adapts techniques from systems security, suggesting methods such as:
– **Prompt Rewriting**: Modifying user inputs to LLMs to prevent exploitation.
– **State Integrity Monitoring**: Ensuring that the data or external states accessed by the agent have not been tampered with.
– **Tool-Fusing**: Combining multiple tools to enhance detection and mitigation efforts.
– **Results Achieved**:
– The study reports a detection accuracy of up to 25% using automated methods, a 3% decrease in the generation of vulnerable plans, and a substantial 95% reduction in the attack window.
– By integrating all three countermeasure approaches, the research demonstrates a significant reduction of TOCTOU vulnerabilities from 12% to 8% in executed trajectories.
– **Implications for AI Safety and Systems Security**:
– The findings challenge existing security paradigms and suggest a new intersection of AI safety and systems security, indicating a need for further research in this area.
This research not only sheds light on a previously under-explored aspect of AI security but also equips professionals with practical countermeasures to protect LLM-enabled systems from emerging threats.