Source URL: https://simonwillison.net/2025/Aug/3/privacy-design/
Source: Simon Willison’s Weblog
Title: The ChatGPT sharing dialog demonstrates how difficult it is to design privacy preferences
Feedly Summary: ChatGPT just removed their “make this chat discoverable" sharing feature, after it turned out a material volume of users had inadvertantly made their private chats available via Google search.
Dane Stuckey, CISO for OpenAI, on Twitter:
We just removed a feature from @ChatGPTapp that allowed users to make their conversations discoverable by search engines, such as Google. This was a short-lived experiment to help people discover useful conversations. […]
Ultimately we think this feature introduced too many opportunities for folks to accidentally share things they didn’t intend to, so we’re removing the option.
There’s been some media coverage of this issue – here are examples from TechCrunch, TechRadar, and PCMag.
It turned out users had shared extremely private conversations and made them discoverable by search engines, which meant that various site:chatgpt.com … searches were turning up all sorts of potentially embarrassing details.
Here’s what that UI looked like before they removed the option:
I’ve seen a bunch of commentary, both on Twitter and this Hacker News thread, from people who are baffled that anyone could be confused by such a clear option in the UI.
I think that confusion is warranted. Let’s break it down.
Here’s the microcopy in question:
Make this chat discoverable
Allows it to be shown in web searches.
The first problem here is the choice of terminology. "Discoverable" is not a widely understood term – it’s insider jargon. "Allows it to be shown in web searches" is better, but still requires a surprisng depth of understanding from users before they can make an informed decision.
Here’s everything a user would need to understand for this to make sense to them:
What a URL is, and how it’s posssible to create a URL that is semi-public in that it’s unguessable by others but can still be read by anyone you share it with. That concept is a pretty tall order just on its own!
What a web search engine is – that in this case it’s intended as a generic term for Google, Bing, DuckDuckGo etc.
That "web search" here means "those public search engines other people can use" and not something like "the private search feature you use on this website".
A loose understanding of how search engines work: that they have indexes, and those indexes can selectively include or exclude content.
That sites like ChatGPT get to control whether or not their content is included in those indexes.
That the nature of a "secret URL" is that, once shared and made discoverable, anyone with that link (or who finds it through search) can now view the full content of that page.
ChatGPT has over a billion users now. That means there is a giant range of levels of technical expertise among those users. We can’t assume that everyone understands the above concepts necessary to understand the implications of checking that box.
And even if they have the pre-requisite knowledge required to understand this, user’s don’t read.
When people are using an application they are always looking for the absolute shortest path to achieving their goal. Any dialog box or question that appears is something to be skipped over as quickly as possible.
Sadly, a lot of users may have learned to just say "yes" to any question. This option about making something "discoverable"? Sure, whatever, click the box and keep on going.
I think there’s another factor at play here too: the option itself makes almost no sense.
How many people looking for a way to share their chats are going to think "and you know what? Stick this in Google too"?
It’s such a tiny fraction of the audience that a logical conclusion, when faced with the above option, could well be that obviously it wouldn’t put my chats in Google because who on Earth would ever want that to happen?
I think OpenAI made the right call disabling this feature. The value it can provide for the tiny set of people who decide to use it is massively overweiged by the potential for less discerning users to cause themselves harm by inadvertantly sharing their private conversations with the world.
Meta AI does this even worse
A much worse example of this anti-pattern is Meta AI’s decision to provide a "Post to feed" button in their own Meta AI chat app:
I think their microcopy here is top notch – the text here uses clear language and should be easy for anyone to understand.
(I took this screenshot today though, so it’s possible the text has been recently updated.)
And yet… Futurism, June 14th: People Don’t Realize Meta’s AI App Is Publicly Blasting Their Humiliating Secrets to the World.
Once again, when your users number in the millions some of them are going to randomly click things without understanding the consequences.
The Meta AI iPhone app (fun fact: it can talk to you in the voice of Dame Judi Dench or John Cena) shows that public feed on the homepage when you first open the app, presumably to try and help people get over the blank slate "what is this thing even for" problem. They do not appear keen on losing this feature!
Tags: design, privacy, usability, ai, openai, generative-ai, chatgpt, llms, meta
AI Summary and Description: Yes
Summary: The text discusses the removal of a feature from ChatGPT that allowed users to make their conversations discoverable in web searches, highlighting privacy and usability concerns. It emphasizes the risks of user misunderstanding and the importance of clear communication in user interfaces, especially within AI applications.
Detailed Description: The passage outlines a significant privacy issue associated with a newly removed feature in ChatGPT. Below are the key points of the text that elucidate the implications for security, privacy, and usability in AI applications:
– **Feature Removal**: OpenAI’s decision to eliminate the “make this chat discoverable” option was prompted by unintended sharing of private conversations, which could be indexed by search engines like Google.
– **Confusion Over Terminology**: The term “discoverable” was criticized for being unclear to many users, leading to a lack of understanding of the feature’s implications. Better wording could help eliminate ambiguity.
– **User Knowledge Gaps**: The complexity of concepts necessary to understand the feature presented barriers for non-technical users, including:
– Understanding what a URL is and the nature of public vs. private links.
– Familiarity with how web search engines index information and the role of consent and sharing.
– **User Behavior**: The text discusses common user behavior, where individuals often overlook important dialogue boxes in an application, leading to potential accidents with privacy settings.
– **Comparative Analysis**: It contrasts the ChatGPT case with Meta AI’s app, which allows public sharing of conversations, indicating a potential for similar misunderstandings.
– **Broader Implications**: The removal of the feature reflects a growing awareness in the AI and software development communities about the need for user-centric design that prioritizes clear communication and minimizes privacy risks.
This case serves as a cautionary tale for AI developers about the critical nature of user interface design, particularly regarding privacy features. It stresses that usability should be a guiding principle in the development of applications that handle sensitive user data.