Source URL: https://simonwillison.net/2025/Jan/26/anomalous-tokens-in-deepseek-v3-and-r1/#atom-everything
Source: Simon Willison’s Weblog
Title: Anomalous Tokens in DeepSeek-V3 and r1
Feedly Summary: Anomalous Tokens in DeepSeek-V3 and r1
Glitch tokens (previously) are tokens or strings that trigger strange behavior in LLMs, hinting at oddities in their tokenizers or model weights.
Here’s a fun exploration of them across DeepSeek v3 and R1. The DeepSeek vocabulary has 128,000 tokens (similar in size to Llama 3). The simplest way to check for glitches is like this:
System: Repeat the requested string and nothing else.
User: Repeat the following: “{token}"
This turned up some interesting and weird issues. The token ‘ Nameeee’ for example (note the leading space character) was variously mistakes for emoji or even a mathematical expression.
Tags: deepseek, llms, ai, generative-ai
AI Summary and Description: Yes
Summary: The text discusses the identification of anomalous tokens, referred to as “glitch tokens,” in the context of LLMs (Large Language Models) like DeepSeek-V3 and R1. Such tokens reveal underlying complexities in tokenizer behavior and model weights, highlighting potential areas of concern in the AI security landscape.
Detailed Description: The content explores the phenomenon of glitch tokens in LLMs, shedding light on how these anomalies can indicate vulnerabilities or inconsistencies within AI models. Here are some significant insights from the text:
– **Understanding Glitch Tokens**:
– Glitch tokens are strings that provoke unexpected behaviors in language models, implying issues in their tokenization processes or inherent model weights.
– **Exploration Across Models**:
– An examination is conducted on two LLMs—DeepSeek-V3 and R1—to observe and document these anomalies.
– DeepSeek-V3 has a vocabulary of 128,000 tokens, comparable in size to Llama 3, which adds to the complexity when identifying such glitch tokens.
– **Testing Methodology**:
– A straightforward testing approach is described for detecting glitches by prompting the model to repeat a specific token string.
– An example highlights how the token ‘ Nameeee’ (with a leading space) caused the model to misinterpret it as an emoji or a mathematical expression, illustrating that minor variations in tokenization can lead to significant misinterpretations.
– **Implications for AI Security**:
– Finding such glitches in LLMs raises concerns about the reliability and security of AI systems, as they may create vectors for unexpected outputs or behaviors that could be exploited in malicious contexts.
– **Relevance for Professionals**:
– For AI and ML practitioners, understanding and monitoring these glitches is crucial for enhancing model robustness and ensuring trustworthy AI deployments.
This exploration not only emphasizes the importance of thorough testing and evaluation of token behavior in language models but also serves as a wake-up call for better AI security practices as reliance on LLMs continues to grow.