Source URL: https://simonwillison.net/2024/Nov/4/haiku/#atom-everything
Source: Simon Willison’s Weblog
Title: Claude 3.5 Haiku
Feedly Summary: Anthropic released Claude 3.5 Haiku today, a few days later than expected (they said it would be out by the end of October).
I was expecting this to be a complete replacement for their existing Claude 3 Haiku model, in the same way that Claude 3.5 Sonnet eclipsed the existing Claude 3 Sonnet while maintaining the same pricing.
Claude 3.5 Haiku is different. First, it doesn’t (yet) support image inputs – so Claude 3 Haiku remains the least expensive Anthropic model for handling those.
Secondly, it’s not priced the same as the previous Haiku. That was $0.25/million input and $1.25/million for output – the new 3.5 Haiku is 4x that at $1/million input and $5/million output.
Given that Anthropic claim that their new Haiku out-performs their older Claude 3 Opus (still $15/m input and $75/m output!) this price isn’t disappointing, but it’s a small surprise nonetheless.
Accessing Claude 3.5 Haiku with LLM
I released a new version of my llm-claude-3 plugin with support for the new model. You can install (or upgrade) the plugin and run it like this:
llm install –upgrade llm-claude-3
llm keys set claude
# Paste API key here
llm -m claude-3.5-haiku ‘describe memory management in Rust’
Here’s the output from that prompt.
Comparing prices
I added the new price to my LLM pricing calculator, which inspired me to extract this comparison table for the leading models from Gemini, Anthropic and OpenAI. Here they are sorted from least to most expensive:
Model
1m token input
1m tokens output
Gemini 1.5 Flash-8B
$0.04
$0.15
Gemini 1.5 Flash
$0.07
$0.30
GPT-4o Mini
$0.15
$0.60
Claude 3 Haiku
$0.25
$1.25
Claude 3.5 Haiku
$1.00
$5.00
Gemini 1.5 Pro
$1.25
$5.00
GPT-4o
$2.50
$10.00
Claude 3.5 Sonnet
$3.00
$15.00
Claude 3 Opus
$15.00
$75.00
Gemini 1.5 Flash-8B remains the model to beat on pricing: it’s 1/6th of the price of the new Haiku – far less capable, but still extremely useful for tasks such as audio transcription.
Also notable from Anthropic’s model comparison table: Claude 3.5 Haiku has a max output of 8,192 tokens (same as 3.5 Sonnet, but twice that of Claude 3 Opus and Claude 3 Haiku). 3.5 Haiku has a training cut-off date of July 2024, the most recent of any Anthropic model. 3.5 Sonnet is April 2024 and the Claude 3 family are all August 2023.
Tags: ai, openai, generative-ai, llms, llm, anthropic, claude, gemini
AI Summary and Description: Yes
Summary: The release of Anthropic’s Claude 3.5 Haiku highlights significant changes in model pricing, features, and capabilities in the landscape of AI language models. This release is particularly relevant for professionals involved in AI and cloud computing, as it underscores the ongoing shifts in AI model development and its economic implications.
Detailed Description: Anthropic’s Claude 3.5 Haiku was released, slightly delayed, and presents noteworthy distinctions from its predecessor, Claude 3 Haiku. The incremental advancements and pricing adjustments are crucial for understanding the competitive landscape of large language models (LLMs).
– **Model Release and Characteristics**:
– Claude 3.5 Haiku does not support image inputs, distinguishing it as the most affordable option for current models that handle text.
– The pricing has significantly increased: from $0.25/million input and $1.25/million output for Claude 3 Haiku to $1/million input and $5/million output for the new version.
– **Performance and Pricing Context**:
– Despite the higher costs, Anthropic claims Claude 3.5 Haiku outperforms the more expensive Claude 3 Opus model, which charges $15/million input and $75/million output.
– Price comparisons reveal that Claude 3.5 Haiku, while costly, is still competitive among other leading models such as OpenAI’s GPT-4o and Gemini.
– **LLM Comparison Insights**:
– A comparative pricing table was shared, ranking various models, showing that Gemini 1.5 Flash-8B remains the most affordable.
– Claude 3.5 Haiku has a maximum output capacity of 8,192 tokens, a feature that may optimize its utility for various applications.
– **Future Developments**:
– The training cut-off date for Claude 3.5 Haiku is notable, set to July 2024, which indicates it incorporates recent knowledge compared to other models with earlier cut-off dates.
This release serves as a critical touchpoint for security and compliance professionals to consider the implications of evolving model capabilities and their associated costs, especially when integrating AI solutions into cloud infrastructures and related applications. Furthermore, the competitive pricing landscape necessitates ongoing vigilance regarding AI security practices, user data protection, and regulatory compliance as organizations adopt these advanced models.