Source URL: https://lethain.com/our-own-agents-our-own-tools/
Source: Irrational Exuberance
Title: Our own agents with their own tools.
Feedly Summary: Entering 2025, I decided to spend some time exploring the topic of agents.
I started reading Anthropic’s Building effective agents,
followed by Chip Huyen’s AI Engineering.
I kicked off a major workstream at work on using agents, and I also decided to do a personal experiment of sorts.
This is a general commentary on building that project.
What I wanted to build was a simple chat interface where I could write prompts, select models,
and have the model use tools as appropriate.
My side goal was to build this using Cursor and generally avoid writing code directly as much
as possible, but I found that generally slower than writing code in emacs while relying
on 4o-mini to provide working examples to pull from.
Similarly, while I initially envisioned building this in fullstack TypeScript via Cursor,
I ultimately bailed into a stack that I’m more comfortable, and ended up using
Python3, FastAPI, PostgreSQL, and SQLAlchemy with the async psycopg3 driver.
It’s been a… while… since I started a brand new Python project, and used this project
as an opportunity to get comfortable with Python3’s async/await mechanisms along
with Python3’s typing along with mypy.
Finally, I also wanted to experiment with Tailwind,
and ended up using TailwindUI’s components to build the site.
The working version supports everything I wanted: creating chats with models, and allowing those models
to use function calling to use tools that I provide. The models are allowed to call any number of tools
in pursuit of the problem they are solving.
The tool usage is the most interesting part here for sure.
The simplest tool I created was a get_temperature tool that provided a fake temperature for your
location. This allowed me to ask questions like “What should I wear tomorrow in San Francisco, CA?”
and get a useful respond.
The code to add this function to my project was pretty straightforward, just three lines of Python
and 25 lines of metadata to pass to the OpenAI API.
def tool_get_current_weather(location: str|None=None, format: str|None=None) -> str:
“Simple proof of concept tool."
temp = random.randint(40, 90) if format == ‘fahrenheit’ else random.randint(10, 25)
return f"It’s going to be {temp} degrees {format} tomorrow."
FUNCTION_REGISTRY[‘get_current_weather’] = tool_get_current_weather
TOOL_USAGE_REGISTRY[‘get_current_weather’] = {
"type": "function",
"function": {
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
},
"format": {
"type": "string",
"enum": ["celsius", "fahrenheit"],
"description": "The temperature unit to use. Infer this from the users location.",
},
},
"required": ["location", "format"],
},
}
}
After getting this tool, the next tool I added was a simple URL retriever tool, which allowed the agent
to grab a URL and use the content of that URL in its prompt.
The implementation for this tool was similarly quite simple.
def tool_get_url(url: str|None=None) -> str:
if url is None:
return ”
url = str(url)
response = requests.get(url)
soup = BeautifulSoup(response.content, ‘html.parser’)
content = soup.find(‘main’) or soup.find(‘article’) or soup.body
if not content:
return str(response.content)
markdown = markdownify(str(content), heading_style="ATX").strip()
return str(markdown)
FUNCTION_REGISTRY[‘get_url’] = tool_get_url
TOOL_USAGE_REGISTRY[‘get_url’] = {
"type": "function",
"function": {
"name": "get_url",
"description": "Retrieve the contents of a website via its URL.",
"parameters": {
"type": "object",
"properties": {
"url": {
"type": "string",
"description": "The complete URL, including protocol to retrieve. For example: \"https://lethain.com\"",
}
},
"required": ["url"],
},
}
}
What’s pretty amazing is how much power you can add to your agent by adding such a trivial tool as retrieving a URL.
You can similarly imagine adding tools for retrieving and commenting on Github pull requests and so, which could
allow a very simple agent tool like this to become quite useful.
Working on this project gave me a moderately compelling view of a near-term future where most engineers have
simple application like this running that they can pipe events into from various systems (email, text,
Github pull requests, calendars, etc), create triggers that map events to templates that feed into prompts,
and execute those prompts with tool-aware agents.
Combine that with ability for other agents to register themselves with you and expose the tools that they
have access to (e.g. schedule an event with tool’s owner), and a bunch of interesting things become very accessible
with a very modest amount of effort:
You could schedule events between two busy people’s calendars, as if both of them had an assistant managing their calendar
Reply to your own pull requests with new blog posts, providing feedback on typos and grammatical issues
Crawl websites you care about and identify posts you might be interested in
Ask the model to generate a system model using lethain:systems,
run that model, then chart the responses
Add a “planning tool” which allows the model to generate a plan to guide subsequent steps in a complex task.
(e.g. getting my calendar, getting a friend’s calendar, suggesting a time we could meet)
None of these are exactly lifesaving, but each is somewhat useful, and I imagine there are many
more fairly obvious ideas that become easy once you have the necessary scaffolding to make this
sort of thing easy.
Altogether, I think that I am convinced at this points that agents, using current foundational models,
are going to create a number of very interesting experiences that improve our day to day lives in
small ways that are, in aggregate, pretty transformational.
I’m less convinced that this is the way all software should work going forward though,
but more thoughts on that over time.
(A bunch of fun experiments happening at work, but early days on those.)
AI Summary and Description: Yes
**Summary:** The text provides insights into the author’s experimentation with building AI agents and tools, highlighting the process of creating a chat interface that utilizes various models and integrates different functionalities. This exploration into agents signifies a trend towards enhancing daily productivity and offers valuable implications for AI and software security professionals by underscoring the potential for security considerations in the design of such tools.
**Detailed Description:** The author elaborates on their journey of creating a project centered around AI agents, detailing both the technical aspects of development and the conceptual motivations behind it. Key points include:
– **Conceptual Framework:** The author aims to build a chat interface that enables interaction with AI models through prompts and tool usage.
– **Technological Stack:** Initially aimed for a tech stack using Cursor and TypeScript, they transitioned to using Python3, FastAPI, PostgreSQL, and SQLAlchemy, primarily to leverage their comfort with Python.
– **Tool Development:**
– A **weather retrieval tool** was introduced. The simplicity of the implementation (just a few lines of code) highlights how accessible it is to integrate additional functionalities into agents.
– A **URL retrieval tool** was also created, which demonstrates the potential to extract and utilize web content in prompts, showcasing how the integration of various tools can extend an agent’s functionality.
– **Future Implications:**
– The discussion includes insights into how engineers might leverage such AI agents in their daily tasks to automate processes, enhancing productivity.
– Possible applications mentioned include:
– Scheduling events between calendars.
– Responding to GitHub pull requests automatically.
– Crawling websites for relevant content.
– **Transformational Potential:** While the author expresses tempered enthusiasm regarding the overall direction of software operating through such agents, they recognize the incremental benefits that can emerge from integrating these tools into everyday workflows.
– **Security Considerations:** The author hints at the importance of ensuring that such AI tools maintain security and compliance, especially as they begin interacting with sensitive data (like calendar events, private emails, etc.).
In conclusion, the exploration of building AI agents not only showcases the potential for enhancing everyday workflows but also prompts important considerations for security and compliance as these tools become more prevalent in professional contexts.