Source URL: https://simonwillison.net/2025/Feb/19/s3-triggers/#atom-everything
Source: Simon Willison’s Weblog
Title: Using S3 triggers to maintain a list of files in DynamoDB
Feedly Summary: Using S3 triggers to maintain a list of files in DynamoDB
I built an experimental prototype this morning of a system for efficiently tracking files that have been added to a large S3 bucket by maintaining a parallel DynamoDB table using S3 triggers and AWS lambda.
I got 80% of the way there with this single prompt to my custom Claude Project:
Python CLI app using boto3 with commands for creating a new S3 bucket which it also configures to have S3 lambada event triggers which moantian a dynamodb table containing metadata about all of the files in that bucket. Include these commands
create_bucket – create a bucket and sets up the associated triggers and dynamo tables
list_files – shows me a list of files based purely on querying dynamo
ChatGPT then took me to the 95% point. The code Claude produced included an obvious bug, so I pasted the code into o3-mini-high on the basis that “reasoning" is often a great way to fix those kinds of errors:
Identify, explain and then fix any bugs in this code:
code from Claude pasted here
… and aside from adding a couple of time.sleep() calls to work around timing errors with IAM policy distribution, everything worked!
Getting from a rough idea to a working proof of concept of something like this with less than 15 minutes of prompting is extraordinarily valuable.
This is exactly the kind of project I’ve avoided in the past because of my almost irrational intolerance of the frustration involved in figuring out the individual details of each call to S3, IAM, AWS Lambda and DynamoDB.
Tags: lambda, uv, ai, llms, ai-assisted-programming, generative-ai, s3, aws, nosql, o3, chatgpt, claude
AI Summary and Description: Yes
Summary: The text discusses the development of an experimental system using AWS services such as S3, DynamoDB, and Lambda, which highlights the efficiency of leveraging AI tools for programming. This project is relevant for cloud computing and AI security professionals interested in automation, rapid development, and maintaining secure environments when interfacing with cloud services.
Detailed Description: The text outlines the author’s experience building a prototype system that tracks files in an Amazon S3 bucket through the use of DynamoDB for metadata storage. The author leveraged AI models (Claude and ChatGPT) to accelerate the coding process and troubleshoot errors. Here are the significant elements discussed:
– **Use of AWS Services**:
– **S3 Triggers**: The author utilized S3 triggers to monitor changes in the S3 bucket.
– **DynamoDB**: A parallel DynamoDB table is maintained for storing metadata about the files added to the S3 bucket.
– **AWS Lambda**: This serverless computing service orchestrates the trigger events from S3.
– **AI-Driven Development**:
– The author emphasizes the value of AI-assisted programming, noting how it helped transform an initial idea into a working proof of concept in under 15 minutes.
– AI tools were used for both code generation and debugging, showcasing their utility in streamlining the development process.
– **Improvement and Debugging**:
– The text describes the process of fixing a bug in the generated code by leveraging another AI model, which is indicative of the collaborative nature of working with AI tools.
– The introduction of `time.sleep()` calls to address problems related to IAM policy distribution underlines the practical challenges developers face and how AI can assist in overcoming them.
– **Impact on Project Management**:
– The author expresses a newfound ability to tackle projects that previously seemed daunting, implying that such AI tools reduce the friction associated with detailed programming tasks.
Overall, this discussion has implications for cloud computing security and compliance professionals, as understanding how to securely automate and efficiently manage cloud resources is crucial. The ease of development using AI tools may lead to both increased productivity and potential security gaps if the systems are not implemented with appropriate security controls in place. This underscores the importance of integrating security practices within the rapid development frameworks enabled by modern AI and cloud technologies.