Experimental News Clipping Site

Tag: prompt-injection

  • Slashdot: New Hack Uses Prompt Injection To Corrupt Gemini’s Long-Term Memory

    —

    by

    Kurt Seifried
    in

    Source URL: https://it.slashdot.org/story/25/02/12/0011205/new-hack-uses-prompt-injection-to-corrupt-geminis-long-term-memory?utm_source=rss1.0mainlinkanon&utm_medium=feed Source: Slashdot Title: New Hack Uses Prompt Injection To Corrupt Gemini’s Long-Term Memory Feedly Summary: AI Summary and Description: Yes Summary: The text discusses a newly demonstrated attack by researcher Johann Rehberger that compromises Google’s Gemini chatbot by manipulating its long-term memory functionality through untrusted document summarization. The attack bypasses existing prompt…

  • Embrace The Red: Hacking Gemini’s Memory with Prompt Injection and Delayed Tool Invocation

    —

    by

    Kurt Seifried
    in

    Source URL: https://embracethered.com/blog/posts/2025/gemini-memory-persistence-prompt-injection/ Source: Embrace The Red Title: Hacking Gemini’s Memory with Prompt Injection and Delayed Tool Invocation Feedly Summary: Imagine your AI rewriting your personal history… A while ago Google added memories to Gemini. Memories allow Gemini to store user-related data across sessions, storing information in long-term memory. The feature is only available to…

  • Wired: DeepSeek’s Safety Guardrails Failed Every Test Researchers Threw at Its AI Chatbot

    —

    by

    Kurt Seifried
    in

    Source URL: https://www.wired.com/story/deepseeks-ai-jailbreak-prompt-injection-attacks/ Source: Wired Title: DeepSeek’s Safety Guardrails Failed Every Test Researchers Threw at Its AI Chatbot Feedly Summary: Security researchers tested 50 well-known jailbreaks against DeepSeek’s popular new AI chatbot. It didn’t stop a single one. AI Summary and Description: Yes Summary: The text highlights the ongoing battle between hackers and security researchers…

  • Simon Willison’s Weblog: How we estimate the risk from prompt injection attacks on AI systems

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Jan/29/prompt-injection-attacks-on-ai-systems/ Source: Simon Willison’s Weblog Title: How we estimate the risk from prompt injection attacks on AI systems Feedly Summary: How we estimate the risk from prompt injection attacks on AI systems The “Agentic AI Security Team" at Google DeepMind share some details on how they are researching indirect prompt injection attacks. They…

  • Simon Willison’s Weblog: ChatGPT Operator system prompt

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Jan/26/chatgpt-operator-system-prompt/#atom-everything Source: Simon Willison’s Weblog Title: ChatGPT Operator system prompt Feedly Summary: ChatGPT Operator system prompt Johann Rehberger snagged a copy of the ChatGPT Operator system prompt. As usual, the system prompt doubles as better written documentation than any of the official sources. It asks users for confirmation a lot: ## Confirmations Ask…

  • Simon Willison’s Weblog: Introducing Operator

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Jan/23/introducing-operator/ Source: Simon Willison’s Weblog Title: Introducing Operator Feedly Summary: Introducing Operator OpenAI released their “research preview" today of Operator, a cloud-based browser automation platform rolling out today to $200/month ChatGPT Pro subscribers. They’re calling this their first "agent". In the Operator announcement video Sam Altman defined that notoriously vague term like this:…

  • Simon Willison’s Weblog: Trading Inference-Time Compute for Adversarial Robustness

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Jan/22/trading-inference-time-compute/ Source: Simon Willison’s Weblog Title: Trading Inference-Time Compute for Adversarial Robustness Feedly Summary: Trading Inference-Time Compute for Adversarial Robustness Brand new research paper from OpenAI, exploring how inference-scaling “reasoning" models such as o1 might impact the search for improved security with respect to things like prompt injection. We conduct experiments on the…

  • Simon Willison’s Weblog: Lessons From Red Teaming 100 Generative AI Products

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Jan/18/lessons-from-red-teaming/ Source: Simon Willison’s Weblog Title: Lessons From Red Teaming 100 Generative AI Products Feedly Summary: Lessons From Red Teaming 100 Generative AI Products New paper from Microsoft describing their top eight lessons learned red teaming (deliberately seeking security vulnerabilities in) 100 different generative AI models and products over the past few years.…

  • Embrace The Red: AI Domination: Remote Controlling ChatGPT ZombAI Instances

    —

    by

    Kurt Seifried
    in

    Source URL: https://embracethered.com/blog/posts/2025/spaiware-and-chatgpt-command-and-control-via-prompt-injection-zombai/ Source: Embrace The Red Title: AI Domination: Remote Controlling ChatGPT ZombAI Instances Feedly Summary: At Black Hat Europe I did a fun presentation titled SpAIware and More: Advanced Prompt Injection Exploits. Without diving into the details of the entire talk, the key point I was making is that prompt injection can impact…

  • Simon Willison’s Weblog: AI’s next leap requires intimate access to your digital life

    —

    by

    system automation
    in

    Source URL: https://simonwillison.net/2025/Jan/6/ais-next-leap/#atom-everything Source: Simon Willison’s Weblog Title: AI’s next leap requires intimate access to your digital life Feedly Summary: AI’s next leap requires intimate access to your digital life I’m quoted in this Washington Post story by Gerrit De Vynck about “agents" – which in this case are defined as AI systems that operate…