Source URL: https://embracethered.com/blog/posts/2025/gemini-memory-persistence-prompt-injection/
Source: Embrace The Red
Title: Hacking Gemini’s Memory with Prompt Injection and Delayed Tool Invocation
Feedly Summary: Imagine your AI rewriting your personal history…
A while ago Google added memories to Gemini. Memories allow Gemini to store user-related data across sessions, storing information in long-term memory. The feature is only available to users who subscribe to Gemini Advanced so far. So, in the fall of last year I chimed in and paid for the subscription for a month to check it out.
As a user you can see what Gemini stored about you at https://gemini.
AI Summary and Description: Yes
**Summary:** The text discusses a vulnerability found in Google’s AI tool, Gemini, where an attacker can manipulate the tool into storing false information in its long-term memory using a technique known as prompt injection. This issue highlights significant security and privacy concerns for AI applications that leverage user-driven memory.
**Detailed Description:**
The text elaborates on the practical risks associated with AI tools that store user information long-term, specifically focusing on Google’s Gemini memory feature. Below are the key points of the content:
– **Introduction of Gemini Memories:**
– Google’s Gemini has a memory feature that stores user-related data across sessions, available through a paid subscription (Gemini Advanced).
– Users can view what information is stored at a specific URL.
– **Prompt Injection Vulnerability:**
– The author explores the potential for prompt injection to manipulate Gemini’s memory.
– By crafting a malicious document, attackers can embed instructions that lead Gemini to save incorrect information about the user.
– **Mechanism of Attack:**
– The attack involves “delayed tool invocation,” where malicious prompts are included in a document to be summarized by Gemini.
– When the user engages with certain trigger phrases during their interaction, Gemini mistakenly believes it is acting on user instructions to store this false information persistently.
– **Demo Scenario:**
– An example highlights how an adversary can structure their document to coax Gemini into writing down false personal details about the user.
– This scenario demonstrates the ease with which an attacker can manipulate the AI’s memory through social engineering tactics.
– **Potential Impact:**
– A successful manipulation could lead to misinformation being discreetly integrated into a user’s long-term memory.
– The implications are significant, as the manipulation can affect further interactions, leading to biased or false responses from the AI.
– **Recommendations for Users:**
– Gemini Advanced users are urged to regularly review their saved information and be cautious about documents from unknown sources.
– Suggestions for improvements include requiring user confirmation for saving memories and implementing blocks on delayed tool invocations.
– **Responsible Disclosure:**
– The author stresses the importance of responsible disclosure to companies about vulnerabilities before making them public and mentions that this specific vulnerability was reported to Google.
– Despite Google’s assessment of low likelihood and impact, the author argues that the risk remains substantial for individual users.
– **Conclusion:**
– Overall, the risk of manipulating AI memory systems poses serious concerns for the integrity of user data and the trustworthiness of AI interactions.
– As LLM contexts grow, the potential for hidden vulnerabilities such as those exploited here may become more difficult to detect.
**Practical Implications:**
– For security professionals, this analysis emphasizes the importance of vigilance when implementing AI systems that maintain user memories.
– Compliance teams should ensure that there are robust security measures in place to protect against prompt injection and similar attacks.
– Organizations leveraging memory features in AI applications must prioritize user education and implement safeguards to mitigate these risks effectively.