Experimental News Clipping Site

Tag: prompt-injection

  • Embrace The Red: Security ProbLLMs in xAI’s Grok: A Deep Dive

    —

    by

    system automation
    in

    Source URL: https://embracethered.com/blog/posts/2024/security-probllms-in-xai-grok/ Source: Embrace The Red Title: Security ProbLLMs in xAI’s Grok: A Deep Dive Feedly Summary: Grok is the chatbot of xAI. It’s a state-of-the-art model, chatbot and recently also API. It has a Web UI and is integrated into the X (former Twitter) app, and recently it’s also accessible via an API.…

  • The Register: Microsoft dangles $10K for hackers to hijack LLM email service

    —

    by

    system automation
    in

    Source URL: https://www.theregister.com/2024/12/09/microsoft_llm_prompt_injection_challenge/ Source: The Register Title: Microsoft dangles $10K for hackers to hijack LLM email service Feedly Summary: Outsmart an AI, win a little Christmas cash Microsoft and friends have challenged AI hackers to break a simulated LLM-integrated email client with a prompt injection attack – and the winning teams will share a $10,000…

  • Embrace The Red: Terminal DiLLMa: LLM-powered Apps Can Hijack Your Terminal Via Prompt Injection

    —

    by

    system automation
    in

    Source URL: https://embracethered.com/blog/posts/2024/terminal-dillmas-prompt-injection-ansi-sequences/ Source: Embrace The Red Title: Terminal DiLLMa: LLM-powered Apps Can Hijack Your Terminal Via Prompt Injection Feedly Summary: Last week Leon Derczynski described how LLMs can output ANSI escape codes. These codes, also known as control characters, are interpreted by terminal emulators and modify behavior. This discovery resonates with areas I had…

  • Simon Willison’s Weblog: 0xfreysa/agent

    —

    by

    system automation
    in

    Source URL: https://simonwillison.net/2024/Nov/29/0xfreysaagent/#atom-everything Source: Simon Willison’s Weblog Title: 0xfreysa/agent Feedly Summary: 0xfreysa/agent Freysa describes itself as “the world’s first adversarial agent game". On 22nd November they released an LLM-driven application which people could pay to message (using Ethereum), with access to tools that could transfer a prize pool to the message sender, ending the game.…

  • Simon Willison’s Weblog: Leaked system prompts from Vercel v0

    —

    by

    system automation
    in

    Source URL: https://simonwillison.net/2024/Nov/25/leaked-system-prompts-from-vercel-v0/#atom-everything Source: Simon Willison’s Weblog Title: Leaked system prompts from Vercel v0 Feedly Summary: Leaked system prompts from Vercel v0 v0 is Vercel’s entry in the increasingly crowded LLM-assisted development market – chat with a bot and have that bot build a full application for you. They’ve been iterating on it since launching…

  • Hacker News: Garak, LLM Vulnerability Scanner

    —

    by

    system automation
    in

    Source URL: https://github.com/NVIDIA/garak Source: Hacker News Title: Garak, LLM Vulnerability Scanner Feedly Summary: Comments AI Summary and Description: Yes **Summary:** The text describes “garak,” a command-line vulnerability scanner specifically designed for large language models (LLMs). This tool aims to uncover various weaknesses in LLMs, such as hallucination, prompt injection attacks, and data leakage. Its development…

  • Hacker News: The Beginner’s Guide to Visual Prompt Injections

    —

    by

    system automation
    in

    Source URL: https://www.lakera.ai/blog/visual-prompt-injections Source: Hacker News Title: The Beginner’s Guide to Visual Prompt Injections Feedly Summary: Comments AI Summary and Description: Yes Summary: The text discusses security vulnerabilities inherent in Large Language Models (LLMs), particularly focusing on visual prompt injections. As the reliance on models like GPT-4 increases for various tasks, concerns regarding the potential…

  • Schneier on Security: Prompt Injection Defenses Against LLM Cyberattacks

    —

    by

    system automation
    in

    Source URL: https://www.schneier.com/blog/archives/2024/11/prompt-injection-defenses-against-llm-cyberattacks.html Source: Schneier on Security Title: Prompt Injection Defenses Against LLM Cyberattacks Feedly Summary: Interesting research: “Hacking Back the AI-Hacker: Prompt Injection as a Defense Against LLM-driven Cyberattacks“: Large language models (LLMs) are increasingly being harnessed to automate cyberattacks, making sophisticated exploits more accessible and scalable. In response, we propose a new defense…

  • Simon Willison’s Weblog: Quoting Question for Department for Science, Innovation and Technology

    —

    by

    system automation
    in

    Source URL: https://simonwillison.net/2024/Nov/1/prompt-injection/#atom-everything Source: Simon Willison’s Weblog Title: Quoting Question for Department for Science, Innovation and Technology Feedly Summary: Lord Clement-Jones: To ask His Majesty’s Government what assessment they have made of the cybersecurity risks posed by prompt injection attacks to the processing by generative artificial intelligence of material provided from outside government, and whether…