Source URL: https://simonwillison.net/2025/Jan/26/paul-gauthier/
Source: Simon Willison’s Weblog
Title: Quoting Paul Gauthier
Feedly Summary: In my experience with AI coding, very large context windows aren’t useful in practice. Every model seems to get confused when you feed them more than ~25-30k tokens. The models stop obeying their system prompts, can’t correctly find/transcribe pieces of code in the context, etc.
Developing aider, I’ve seen this problem with gpt-4o, Sonnet, DeepSeek, etc. Many aider users report this too. It’s perhaps the #1 problem users have, so I created a dedicated help page.
— Paul Gauthier
Tags: aider, ai-assisted-programming, generative-ai, long-context, ai, llms
AI Summary and Description: Yes
Summary: The text highlights a significant limitation in AI models, specifically regarding their inability to effectively manage very large context windows beyond 25-30k tokens. This issue has been observed across various AI coding platforms and is recognized as a major obstacle for users, leading to the creation of supportive resources.
Detailed Description: The content discusses the challenges faced by AI coding models when they are provided with excessively large context windows. Notably, the insights stem from firsthand experience with various AI tools, including gpt-4o, Sonnet, and DeepSeek. The key points of the discussion include:
– **Context Window Limitation**: AI models struggle with context windows over approximately 25-30k tokens, resulting in significant confusion in processing prompts.
– **User Experiences**: Many users working with AI-assisted programming tools have reported similar issues, indicating a widespread problem in the community.
– **System Prompt Compliance**: AI’s ability to adhere to initial system prompts declines as context windows increase, leading to unexpected results in tasks such as code transcription.
– **Help Resources**: Acknowledging the issue’s prominence, the author has created a dedicated help page to assist users in navigating this limitation.
This information is particularly relevant for professionals in the AI and software development sectors, as it underscores a critical area for improvement in AI model design and implementation. Addressing the limitations of long context windows could enhance user experience and efficiency in AI-assisted applications.