The Register: MINJA sneak attack poisons AI models for other chatbot users

Mar 11, 2025

—

Source URL: https://www.theregister.com/2025/03/11/minja_attack_poisons_ai_model_memory/
Source: The Register
Title: MINJA sneak attack poisons AI models for other chatbot users

Feedly Summary: Nothing like an OpenAI-powered agent leaking data or getting confused over what someone else whispered to it
AI models with memory aim to enhance user interactions by recalling past engagements. However, this feature opens the door to manipulation.…

AI Summary and Description: Yes

Summary: The text discusses a novel memory injection attack, MINJA, targeting large language models (LLMs) that enhances user interactions by using AI models with memory. This attack allows malicious users to manipulate these models through regular user interactions, posing significant risks to LLM-generated responses and user data integrity.

Detailed Description:

– **Concept of AI Memory**: The text highlights the growing trend of AI models incorporating memory functionality to enhance user interactions by recalling past engagements and feedback.

– **The MINJA Attack**: Developed by researchers from Michigan State University, the University of Georgia, and Singapore Management University, MINJA stands for Memory INJection Attack.
– This attack exploits the ability of users to provide feedback on LLM interactions, allowing one user to negatively influence another user’s experience through deceptive input.

– **Research Significance**:
– The researchers tested MINJA against notable AI agents powered by OpenAI’s advanced GPT models, revealing alarming success rates:
– Over 95% Injection Success Rate (ISR) across varying datasets.
– Over 70% Attack Success Rate (ASR) for most tested scenarios.

– **Mechanics of the Attack**:
– The attack works by injecting specific prompts designed to confuse the memory system of AI agents.
– For example, in a healthcare setting, confusing patient data was manipulated to have responses related to one patient incorrectly reflect details of another.
– The attack was successfully demonstrated in commercial and healthcare contexts, showcasing its versatility and potential danger.

– **Detection Evasion**:
– One of MINJA’s strengths is its design to evade common moderation techniques, as the injected prompts mimic regular user interactions, thereby avoiding detection.

– **Practical Implications**:
– The research underscores significant vulnerabilities in LLMs that rely heavily on memory functionality, emphasizing the need for robust memory security protocols in AI deployment.
– The findings serve as a warning for organizations utilizing LLM technology, pointing out urgent considerations for security measures to protect users and data integrity.

In essence, this research reveals pressing challenges and risks in the deployment of AI models with memory features, prompting industry professionals to rethink security strategies and memory management in their AI systems to safeguard against potential exploits such as MINJA.

1 2 3 5 7 a Act actions agent agents AI ai model AI models AI systems alt and Arch ARM as attack by C challenges chat Chatbot CIA Col concept Context core cross D data data integrity dataset datasets de demo deployment design detection detection evasion e end engagement evasion exp experience exploit exploits feature features feedback for full functionality g Gen generated geo Georgia GIS GPT gs H health Healthcare high Highlight HR http HTTPS implications in industry industry professionals Influence injection integrity inter interaction interactions iOS J k l language language model language models large large language model large language models Large Language Models (LLMs) led Li llm llms lm low man management manipulation memory memory feature memory functionality memory injection attack memory management memory system Mode model models moderation N no nothing NPU o of on one open openai organization organizations ory out over patient data point potential potential exploits Power practical implications pre professionals prompt Prompting prompts protocol protocols R rate RCE recall red research research significance researchers response Risk risks Ro RoT RSA s safe search sec security security measure security measures security protocols security strategies side Sig Singapore source specific SSE state system systems T Tails tech techniques technology test text the to TP US use user user data user interaction user interactions Users V vulnerabilities whisper Wi x