Source URL: https://simonwillison.net/2025/Aug/9/ethan-mollick/#atom-everything
Source: Simon Willison’s Weblog
Title: Quoting Ethan Mollick
Feedly Summary: The issue with GPT-5 in a nutshell is that unless you pay for model switching & know to use GPT-5 Thinking or Pro, when you ask “GPT-5” you sometimes get the best available AI & sometimes get one of the worst AIs available and it might even switch within a single conversation.
— Ethan Mollick, highlighting that GPT-5 (high) ranks top on Artificial Analysis, GPT-5 (minimal) ranks lower than GPT-4.1
Tags: gpt-5, ethan-mollick, generative-ai, ai, llms
AI Summary and Description: Yes
Summary: The text discusses the inconsistency of output quality from GPT-5, particularly emphasizing that users may not receive the expected performance unless they pay for specific model options. This highlights a significant concern for developers and security professionals in understanding and mitigating risks associated with AI performance variability.
Detailed Description:
The provided text outlines a critical observation regarding the functionality of GPT-5, especially in relation to its model switching capabilities. Here are the major points of significance:
– **Model Switching**: Users might receive outputs from different model versions (high vs. minimal performance) without clear indication, impacting the reliability of AI applications.
– **Performance Variability**: The statement points out that GPT-5 can generate outputs that range significantly in quality—sometimes outperforming even earlier models like GPT-4.1, and at other times producing lower-quality responses.
– **User Awareness**: The necessity for users to understand which model they are interacting with and whether they are utilizing a version with enhanced capabilities (like GPT-5 Thinking or Pro) is essential to avoid undesirable outcomes.
– **Implications for Development**: For designers and developers of AI systems, this variability poses security and reliability concerns—particularly in applications where the AI’s decisions could have significant ramifications (e.g., legal documentation, healthcare advice).
– **Cost Concerns**: The mention of needing to “pay for model switching” also hints at business implications, where the cost of using advanced AI capabilities might be a barrier for some users, potentially influencing how AI technology is adopted across different sectors.
In summary, this commentary by Ethan Mollick sheds light on an emerging challenge in generative AI—ensuring consistent quality and user understanding amidst rapidly changing capabilities. Security professionals in the field must consider these nuances when incorporating AI into their systems, especially concerning data integrity, user trust, and operational reliability.