Source URL: https://simonwillison.net/2025/Mar/1/llm-anthropic/#atom-everything
Source: Simon Willison’s Weblog
Title: llm-anthropic #24: Use new URL parameter to send attachments
Feedly Summary: llm-anthropic #24: Use new URL parameter to send attachments
Anthropic released a neat quality of life improvement today. Alex Albert:
We’ve added the ability to specify a public facing URL as the source for an image / document block in the Anthropic API
Prior to this, any time you wanted to send an image to the Claude API you needed to base64-encode it and then include that data in the JSON. This got pretty bulky, especially in conversation scenarios where the same image data needs to get passed in every follow-up prompt.
I implemented this for llm-anthropic and shipped it just now in version 0.15.1 (here’s the commit) – I went with a patch release version number bump because this is effectively a performance optimization which doesn’t provide any new features, previously LLM would accept URLs just fine and would download and then base64 them behind the scenes.
In testing this out I had a really impressive result from Claude 3.7 Sonnet. I found a newspaper page from 1900 on the Library of Congress (the “Worcester spy.") and fed a URL to the PDF into Sonnet like this:
llm -m claude-3.7-sonnet \
-a ‘https://tile.loc.gov/storage-services/service/ndnp/mb/batch_mb_gaia_ver02/data/sn86086481/0051717161A/1900012901/0296.pdf’ \
‘transcribe all text from this image, formatted as markdown’
I haven’t checked every sentence but it appears to have done an excellent job, at a cost of 16 cents.
As another experiment, I tried running that against my example people template from the schemas feature I released this morning:
llm -m claude-3.7-sonnet \
-a ‘https://tile.loc.gov/storage-services/service/ndnp/mb/batch_mb_gaia_ver02/data/sn86086481/0051717161A/1900012901/0296.pdf’ \
-t people
That only gave me two results – so I tried an alternative approach where I looped the OCR text back through the same template, using llm logs –cid with the logged conversation ID and -r to extract just the raw response from the logs:
llm logs –cid 01jn7h45x2dafa34zk30z7ayfy -r | \
llm -t people -m claude-3.7-sonnet
… and that worked fantastically well! The result started like this:
{
"items": [
{
"name": "Capt. W. R. Abercrombie",
"organization": "United States Army",
"role": "Commander of Copper River exploring expedition",
"learned": "Reported on the horrors along the Copper River in Alaska, including starvation, scurvy, and mental illness affecting 70% of people. He was tasked with laying out a trans-Alaskan military route and assessing resources.",
"article_headline": "MUCH SUFFERING",
"article_date": "1900-01-28"
},
{
"name": "Edward Gillette",
"organization": "Copper River expedition",
"role": "Member of the expedition",
"learned": "Contributed a chapter to Abercrombie’s report on the feasibility of establishing a railroad route up the Copper River valley, comparing it favorably to the Seattle to Skaguay route.",
"article_headline": "MUCH SUFFERING",
"article_date": "1900-01-28"
}
Full response here.
Tags: llm, anthropic, generative-ai, projects, ai, llms
AI Summary and Description: Yes
Summary: The release of a new feature in the Anthropic API improves efficiency by allowing users to send URLs for images/documents instead of sending bulky base64-encoded data. This update enhances performance in LLM interactions and facilitates better data handling during conversations.
Detailed Description: The text discusses a significant enhancement made by Anthropic to its API. Here are the key points:
– **New URL Parameter**: The release introduces an option to specify a public-facing URL for image or document blocks within the Claude API. This enhances functionality by eliminating the need for base64 encoding, which was previously required to transmit image data.
– **Performance Optimization**: The change primarily serves as a performance improvement rather than a feature addition. Although the API previously accepted URLs and processed them behind the scenes, this update simplifies the user experience.
– **End-user Experience**: Users can now efficiently handle image or document data in conversation scenarios. As the sender, you can rely on external links rather than sending larger payloads of encoded data, making interactions smoother and faster.
– **Practical Application**: In the example provided, the user tested the new feature by successfully transcribing text from a PDF of a historical newspaper page using the Claude 3.7 Sonnet model. It demonstrated not only the capability of handling URLs effectively but also showcased the model’s accuracy and efficiency.
– **Improved Data Retrieval**: Further experimentation involved using logged conversation ID data to refine and enhance outputs, showing the system’s flexibility in processing tasks and leveraging previous interactions.
This update underscores the ongoing enhancements in generative AI tools and their increasing integration capabilities, promoting better user experience in AI interactions. Security and compliance professionals may find this relevant as efficiency improvements can help streamline operations that involve handling sensitive data in conversation-driven contexts, thereby potentially minimizing risks associated with large data transfers and complex API interactions.