Experimental News Clipping Site

Tag: evaluations

  • Simon Willison’s Weblog: Anthropic: A postmortem of three recent issues

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Sep/17/anthropic-postmortem/ Source: Simon Willison’s Weblog Title: Anthropic: A postmortem of three recent issues Feedly Summary: Anthropic: A postmortem of three recent issues Anthropic had a very bad month in terms of model reliability: Between August and early September, three infrastructure bugs intermittently degraded Claude’s response quality. We’ve now resolved these issues and want…

  • Slashdot: OpenAI Says Models Programmed To Make Stuff Up Instead of Admitting Ignorance

    —

    by

    Kurt Seifried
    in

    Source URL: https://slashdot.org/story/25/09/17/1724241/openai-says-models-programmed-to-make-stuff-up-instead-of-admitting-ignorance?utm_source=rss1.0mainlinkanon&utm_medium=feed Source: Slashdot Title: OpenAI Says Models Programmed To Make Stuff Up Instead of Admitting Ignorance Feedly Summary: AI Summary and Description: Yes Summary: The text discusses OpenAI’s acknowledgment of the issue of “hallucinations” in AI models, specifically how these models frequently yield false outputs due to a training bias that rewards generating…

  • OpenAI : Detecting and reducing scheming in AI models

    —

    by

    Kurt Seifried
    in

    Source URL: https://openai.com/index/detecting-and-reducing-scheming-in-ai-models Source: OpenAI Title: Detecting and reducing scheming in AI models Feedly Summary: Apollo Research and OpenAI developed evaluations for hidden misalignment (“scheming”) and found behaviors consistent with scheming in controlled tests across frontier models. The team shared concrete examples and stress tests of an early method to reduce scheming. AI Summary and…

  • Simon Willison’s Weblog: GPT‑5-Codex and upgrades to Codex

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Sep/15/gpt-5-codex/#atom-everything Source: Simon Willison’s Weblog Title: GPT‑5-Codex and upgrades to Codex Feedly Summary: GPT‑5-Codex and upgrades to Codex OpenAI half-released a new model today: GPT‑5-Codex, a fine-tuned GPT-5 variant explicitly designed for their various AI-assisted programming tools. I say half-released because it’s not yet available via their API, but they “plan to make…

  • Slashdot: Apple Claims ‘Most Significant Upgrade to Memory Safety’ in OS History

    —

    by

    Kurt Seifried
    in

    Source URL: https://apple.slashdot.org/story/25/09/14/228211/apple-claims-most-significant-upgrade-to-memory-safety-in-os-history?utm_source=rss1.0mainlinkanon&utm_medium=feed Source: Slashdot Title: Apple Claims ‘Most Significant Upgrade to Memory Safety’ in OS History Feedly Summary: AI Summary and Description: Yes Summary: Apple has introduced a groundbreaking security feature called Memory Integrity Enforcement (MIE) in its latest devices, which significantly enhances memory safety and aims to defend against sophisticated spyware attacks. This…

  • OpenAI : Why language models hallucinate

    —

    by

    Kurt Seifried
    in

    Source URL: https://openai.com/index/why-language-models-hallucinate Source: OpenAI Title: Why language models hallucinate Feedly Summary: OpenAI’s new research explains why language models hallucinate. The findings show how improved evaluations can enhance AI reliability, honesty, and safety. AI Summary and Description: Yes Summary: The text discusses OpenAI’s research on the phenomenon of hallucination in language models, offering insights into…

  • OpenAI : OpenAI and Anthropic share findings from a joint safety evaluation

    —

    by

    Kurt Seifried
    in

    Source URL: https://openai.com/index/openai-anthropic-safety-evaluation Source: OpenAI Title: OpenAI and Anthropic share findings from a joint safety evaluation Feedly Summary: OpenAI and Anthropic share findings from a first-of-its-kind joint safety evaluation, testing each other’s models for misalignment, instruction following, hallucinations, jailbreaking, and more—highlighting progress, challenges, and the value of cross-lab collaboration. AI Summary and Description: Yes Summary:…

  • Anchore: Meeting 2025’s SBOM Compliance Deadlines: A Practical Implementation Guide Pt. 2

    —

    by

    Kurt Seifried
    in

    Source URL: https://anchore.com/blog/meeting-2025s-sbom-compliance-deadlines-a-practical-implementation-guide-pt-2/ Source: Anchore Title: Meeting 2025’s SBOM Compliance Deadlines: A Practical Implementation Guide Pt. 2 Feedly Summary: If you’re just joining us, this is part 2 of a series on practical implementation of software supply chain security to meet the most recent SBOM compliance requirements. In Part 1, we covered the fundamentals of…

  • Schneier on Security: Encryption Backdoor in Military/Police Radios

    —

    by

    Kurt Seifried
    in

    Source URL: https://www.schneier.com/blog/archives/2025/08/encryption-backdoor-in-military-police-radios.html Source: Schneier on Security Title: Encryption Backdoor in Military/Police Radios Feedly Summary: I wrote about this in 2023. Here’s the story: Three Dutch security analysts discovered the vulnerabilities­—five in total—­in a European radio standard called TETRA (Terrestrial Trunked Radio), which is used in radios made by Motorola, Damm, Hytera, and others. The…