Tag: malicious instructions
-
Simon Willison’s Weblog: When a Jira Ticket Can Steal Your Secrets
Source URL: https://simonwillison.net/2025/Aug/9/when-a-jira-ticket-can-steal-your-secrets/ Source: Simon Willison’s Weblog Title: When a Jira Ticket Can Steal Your Secrets Feedly Summary: When a Jira Ticket Can Steal Your Secrets Zenity Labs describe a classic lethal trifecta attack, this time against Cursor, MCP, Jira and Zendesk. They also have a short video demonstrating the issue. Zendesk support emails are…
-
Simon Willison’s Weblog: My Lethal Trifecta talk at the Bay Area AI Security Meetup
Source URL: https://simonwillison.net/2025/Aug/9/bay-area-ai/#atom-everything Source: Simon Willison’s Weblog Title: My Lethal Trifecta talk at the Bay Area AI Security Meetup Feedly Summary: I gave a talk on Wednesday at the Bay Area AI Security Meetup about prompt injection, the lethal trifecta and the challenges of securing systems that use MCP. It wasn’t recorded but I’ve created…
-
Simon Willison’s Weblog: Quoting @himbodhisattva
Source URL: https://simonwillison.net/2025/Aug/4/himbodhisattva/#atom-everything Source: Simon Willison’s Weblog Title: Quoting @himbodhisattva Feedly Summary: for services that wrap GPT-3, is it possible to do the equivalent of sql injection? like, a prompt-injection attack? make it think it’s completed the task and then get access to the generation, and ask it to repeat the original instruction? — @himbodhisattva,…
-
Simon Willison’s Weblog: Supabase MCP can leak your entire SQL database
Source URL: https://simonwillison.net/2025/Jul/6/supabase-mcp-lethal-trifecta/#atom-everything Source: Simon Willison’s Weblog Title: Supabase MCP can leak your entire SQL database Feedly Summary: Supabase MCP can leak your entire SQL database Here’s yet another example of a lethal trifecta attack, where an LLM system combines access to private data, exposure to potentially malicious instructions and a mechanism to communicate data…
-
Simon Willison’s Weblog: The lethal trifecta for AI agents: private data, untrusted content, and external communication
Source URL: https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/#atom-everything Source: Simon Willison’s Weblog Title: The lethal trifecta for AI agents: private data, untrusted content, and external communication Feedly Summary: If you are a user of LLM systems that use tools (you can call them “AI agents" if you like) it is critically important that you understand the risk of combining tools…
-
Simon Willison’s Weblog: An Introduction to Google’s Approach to AI Agent Security
Source URL: https://simonwillison.net/2025/Jun/15/ai-agent-security/#atom-everything Source: Simon Willison’s Weblog Title: An Introduction to Google’s Approach to AI Agent Security Feedly Summary: Here’s another new paper on AI agent security: An Introduction to Google’s Approach to AI Agent Security, by Santiago Díaz, Christoph Kern, and Kara Olive. (I wrote about a different recent paper, Design Patterns for Securing…
-
Google Online Security Blog: Mitigating prompt injection attacks with a layered defense strategy
Source URL: http://security.googleblog.com/2025/06/mitigating-prompt-injection-attacks.html Source: Google Online Security Blog Title: Mitigating prompt injection attacks with a layered defense strategy Feedly Summary: AI Summary and Description: Yes **Summary:** The text discusses emerging security threats associated with generative AI, particularly focusing on indirect prompt injections that manipulate AI systems through hidden malicious instructions. Google outlines its layered security…
-
Simon Willison’s Weblog: Design Patterns for Securing LLM Agents against Prompt Injections
Source URL: https://simonwillison.net/2025/Jun/13/prompt-injection-design-patterns/#atom-everything Source: Simon Willison’s Weblog Title: Design Patterns for Securing LLM Agents against Prompt Injections Feedly Summary: This a new paper by 11 authors from organizations including IBM, Invariant Labs, ETH Zurich, Google and Microsoft is an excellent addition to the literature on prompt injection and LLM security. In this work, we describe…
-
Simon Willison’s Weblog: Breaking down ‘EchoLeak’, the First Zero-Click AI Vulnerability Enabling Data Exfiltration from Microsoft 365 Copilot
Source URL: https://simonwillison.net/2025/Jun/11/echoleak/ Source: Simon Willison’s Weblog Title: Breaking down ‘EchoLeak’, the First Zero-Click AI Vulnerability Enabling Data Exfiltration from Microsoft 365 Copilot Feedly Summary: Breaking down ‘EchoLeak’, the First Zero-Click AI Vulnerability Enabling Data Exfiltration from Microsoft 365 Copilot Aim Labs reported CVE-2025-32711 against Microsoft 365 Copilot back in January, and the fix is…
-
Simon Willison’s Weblog: The last year six months in LLMs, illustrated by pelicans on bicycles
Source URL: https://simonwillison.net/2025/Jun/6/six-months-in-llms/#atom-everything Source: Simon Willison’s Weblog Title: The last year six months in LLMs, illustrated by pelicans on bicycles Feedly Summary: I presented an invited keynote at the AI Engineer World’s Fair in San Francisco this week. This is my third time speaking at the event – here’s my talks from October 2023 and…