Source URL: https://simonwillison.net/2025/Jan/3/asking-them-to-write-better-code/
Source: Simon Willison’s Weblog
Title: Can LLMs write better code if you keep asking them to “write better code”?
Feedly Summary: Can LLMs write better code if you keep asking them to “write better code”?
Really fun exploration by Max Woolf, who started with a prompt requesting a medium-complexity Python challenge – “Given a list of 1 million random integers between 1 and 100,000, find the difference between the smallest and the largest numbers whose digits sum up to 30" – and then continually replied with "write better code" to see what happened.
It works! Kind of… it’s not quite as simple as "each time round you get better code" – the improvements sometimes introduced new bugs and often leaned into more verbose enterprisey patterns – but the model (Claude in this case) did start digging into optimizations like numpy and numba JIT compilation to speed things up.
I used to find the thing where telling an LLM to "do better" worked completely surprising. I’ve since come to terms with why it works: LLMs are effectively stateless, so each prompt you execute is considered as an entirely new problem. When you say "write better code" your prompt is accompanied with a copy of the previous conversation, so you’re effectively saying "here is some code, suggest ways to improve it". The fact that the LLM itself wrote the previous code isn’t really important.
I’ve been having a lot of fun recently using LLMs for cooking inspiration. "Give me a recipe for guacamole", then "make it tastier" repeated a few times results in some bizarre and fun variations on the theme!
Via @minimaxir.bsky.social
Tags: max-woolf, prompt-engineering, ai-assisted-programming, generative-ai, ai, llms, python
AI Summary and Description: Yes
Summary: The text explores whether or not large language models (LLMs) can improve the quality of code through iterative prompts, specifically asking them to “write better code.” It highlights the nuances of this process, including the potential for introducing new bugs and the benefits of optimizations such as using numpy and numba JIT compilation.
Detailed Description: The exploration of how LLMs can respond to iterative prompts for coding improvements presents valuable insights into the capabilities and limitations of AI in software development contexts. Key points include:
– **Iterative Improvement**: The technique of repeatedly asking an LLM to “write better code” can lead to enhancements, but the outcome is not consistently positive; improvements might come with unintended consequences such as new bugs or overly complex solutions.
– **Model Awareness**: The stateless nature of LLMs means that each response is treated as a fresh inquiry, relying on context supplied by previous interactions. This methodological shift influences how effectively the model can iterate on solutions.
– **Optimization Strategies**: In one instance, the model began to leverage advanced coding techniques, such as numpy and numba JIT compilation, which are known to significantly enhance performance in Python programming.
– **Fun with Variations**: The author also draws a parallel with using LLMs for creative tasks (like cooking), illustrating that the iterative refining of prompts can yield fascinating, unexpected outcomes.
Insights for professionals in AI, cloud, and infrastructure security might include:
– **Understanding Limitations**: Recognizing that while AI tools can aid in code generation, the need for human oversight remains crucial to catch bugs and ensure security.
– **Prompt Engineering**: The practice of refining prompts to extract better results can be applied not just in coding but across various AI applications in enterprise settings.
– **Adverse Effects of Automation**: As organizations integrate AI-driven solutions like LLMs into their workflows, awareness of potential pitfalls (such as programming bugs or overly complex solutions) is essential for maintaining software security and efficiency.
In conclusion, the exploration highlights both the promise and challenges of using LLMs in practical programming tasks, emphasizing the need for careful management in automated processes that have security implications.