Experimental News Clipping Site

Tag: prompt injection attacks

  • Simon Willison’s Weblog: ChatGPT Operator system prompt

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Jan/26/chatgpt-operator-system-prompt/#atom-everything Source: Simon Willison’s Weblog Title: ChatGPT Operator system prompt Feedly Summary: ChatGPT Operator system prompt Johann Rehberger snagged a copy of the ChatGPT Operator system prompt. As usual, the system prompt doubles as better written documentation than any of the official sources. It asks users for confirmation a lot: ## Confirmations Ask…

  • Simon Willison’s Weblog: Lessons From Red Teaming 100 Generative AI Products

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Jan/18/lessons-from-red-teaming/ Source: Simon Willison’s Weblog Title: Lessons From Red Teaming 100 Generative AI Products Feedly Summary: Lessons From Red Teaming 100 Generative AI Products New paper from Microsoft describing their top eight lessons learned red teaming (deliberately seeking security vulnerabilities in) 100 different generative AI models and products over the past few years.…

  • CSA: AI and Compliance for the Mid-Market

    —

    by

    Kurt Seifried
    in

    Source URL: https://www.scrut.io/post/ai-and-compliance-for-the-mid-market Source: CSA Title: AI and Compliance for the Mid-Market Feedly Summary: AI Summary and Description: Yes **Summary:** The text emphasizes the urgent need for small and medium-sized businesses (SMBs) to adopt AI responsibly, given the potential cybersecurity vulnerabilities and evolving regulatory landscape associated with AI technologies. It outlines practical guidance and standards…

  • CSA: LLM Dragons: Why DSPM is the Key to AI Security

    —

    by

    Kurt Seifried
    in

    Source URL: https://cloudsecurityalliance.org/articles/training-your-llm-dragons-why-dspm-is-the-key-to-ai-security Source: CSA Title: LLM Dragons: Why DSPM is the Key to AI Security Feedly Summary: AI Summary and Description: Yes Summary: The text emphasizes the security risks associated with AI implementations, particularly custom large language models (LLMs) and Microsoft Copilot. It outlines key threats such as data leakage and compliance failures and…

  • The Register: Just as your LLM once again goes off the rails, Cisco, Nvidia are at the door smiling

    —

    by

    Kurt Seifried
    in

    Source URL: https://www.theregister.com/2025/01/17/nvidia_cisco_ai_guardrails_security/ Source: The Register Title: Just as your LLM once again goes off the rails, Cisco, Nvidia are at the door smiling Feedly Summary: Some of you have apparently already botched chatbots or allowed ‘shadow AI’ to creep in Cisco and Nvidia have both recognized that as useful as today’s AI may be,…

  • Embrace The Red: AI Domination: Remote Controlling ChatGPT ZombAI Instances

    —

    by

    Kurt Seifried
    in

    Source URL: https://embracethered.com/blog/posts/2025/spaiware-and-chatgpt-command-and-control-via-prompt-injection-zombai/ Source: Embrace The Red Title: AI Domination: Remote Controlling ChatGPT ZombAI Instances Feedly Summary: At Black Hat Europe I did a fun presentation titled SpAIware and More: Advanced Prompt Injection Exploits. Without diving into the details of the entire talk, the key point I was making is that prompt injection can impact…

  • Embrace The Red: Trust No AI: Prompt Injection Along the CIA Security Triad Paper

    —

    by

    system automation
    in

    Source URL: https://embracethered.com/blog/posts/2024/trust-no-ai-prompt-injection-along-the-cia-security-triad-paper/ Source: Embrace The Red Title: Trust No AI: Prompt Injection Along the CIA Security Triad Paper Feedly Summary: Happy to share that I authored the paper “Trust No AI: Prompt Injection Along The CIA Security Triad”, based on research conducted over the past 18 months. You can download it from arxiv. The…

  • Simon Willison’s Weblog: Quoting Johann Rehberger

    —

    by

    system automation
    in

    Source URL: https://simonwillison.net/2024/Dec/17/johann-rehberger/ Source: Simon Willison’s Weblog Title: Quoting Johann Rehberger Feedly Summary: Happy to share that Anthropic fixed a data leakage issue in the iOS app of Claude that I responsibly disclosed. 🙌 👉 Image URL rendering as avenue to leak data in LLM apps often exists in mobile apps as well — typically…

  • Simon Willison’s Weblog: Security ProbLLMs in xAI’s Grok: A Deep Dive

    —

    by

    system automation
    in

    Source URL: https://simonwillison.net/2024/Dec/16/security-probllms-in-xais-grok/#atom-everything Source: Simon Willison’s Weblog Title: Security ProbLLMs in xAI’s Grok: A Deep Dive Feedly Summary: Security ProbLLMs in xAI’s Grok: A Deep Dive Adding xAI to the growing list of AI labs that shipped feature vulnerable to data exfiltration prompt injection attacks, but with the unfortunate addendum that they don’t seem to…

  • Embrace The Red: Security ProbLLMs in xAI’s Grok: A Deep Dive

    —

    by

    system automation
    in

    Source URL: https://embracethered.com/blog/posts/2024/security-probllms-in-xai-grok/ Source: Embrace The Red Title: Security ProbLLMs in xAI’s Grok: A Deep Dive Feedly Summary: Grok is the chatbot of xAI. It’s a state-of-the-art model, chatbot and recently also API. It has a Web UI and is integrated into the X (former Twitter) app, and recently it’s also accessible via an API.…