Tag: model evaluation
-
Simon Willison’s Weblog: Let the LLM Write the Prompts: An Intro to DSPy in Compound Al Pipelines
Source URL: https://simonwillison.net/2025/Oct/4/drew-on-dspy/#atom-everything Source: Simon Willison’s Weblog Title: Let the LLM Write the Prompts: An Intro to DSPy in Compound Al Pipelines Feedly Summary: Let the LLM Write the Prompts: An Intro to DSPy in Compound Al Pipelines I’ve had trouble getting my head around DSPy in the past. This half hour talk by Drew…
-
Cloud Blog: How Mr. Cooper assembled a team of AI agents to handle complex mortgage questions
Source URL: https://cloud.google.com/blog/topics/financial-services/assembling-a-team-of-ai-agents-to-handle-complex-mortgage-questions-at-mr-cooper/ Source: Cloud Blog Title: How Mr. Cooper assembled a team of AI agents to handle complex mortgage questions Feedly Summary: In today’s world where instant responses and seamless experiences are the norm, industries like mortgage servicing face tough challenges. When navigating a maze of regulations, piles of financial documents, and the high…
-
Unit 42: Model Namespace Reuse: An AI Supply-Chain Attack Exploiting Model Name Trust
Source URL: https://unit42.paloaltonetworks.com/model-namespace-reuse/ Source: Unit 42 Title: Model Namespace Reuse: An AI Supply-Chain Attack Exploiting Model Name Trust Feedly Summary: Model namespace reuse is a potential security risk in the AI supply chain. Attackers can misuse platforms like Hugging Face for remote code execution. The post Model Namespace Reuse: An AI Supply-Chain Attack Exploiting Model…
-
Simon Willison’s Weblog: Qwen3-30B-A3B-Thinking-2507
Source URL: https://simonwillison.net/2025/Jul/30/qwen3-30b-a3b-thinking-2507/ Source: Simon Willison’s Weblog Title: Qwen3-30B-A3B-Thinking-2507 Feedly Summary: Qwen3-30B-A3B-Thinking-2507 Yesterday was Qwen3-30B-A3B-Instruct-2507. Qwen are clearly committed to their new split between reasoning and non-reasoning models (a reversal from Qwen 3 in April), because today they released the new reasoning partner to yesterday’s model: Qwen3-30B-A3B-Thinking-2507. I’m surprised at how poorly this reasoning mode…
-
Simon Willison’s Weblog: TimeScope: How Long Can Your Video Large Multimodal Model Go?
Source URL: https://simonwillison.net/2025/Jul/23/timescope/#atom-everything Source: Simon Willison’s Weblog Title: TimeScope: How Long Can Your Video Large Multimodal Model Go? Feedly Summary: TimeScope: How Long Can Your Video Large Multimodal Model Go? New open source benchmark for evaluating vision LLMs on how well they handle long videos: TimeScope probes the limits of long-video capabilities by inserting several…
-
Simon Willison’s Weblog: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity
Source URL: https://simonwillison.net/2025/Jul/12/ai-open-source-productivity/#atom-everything Source: Simon Willison’s Weblog Title: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity Feedly Summary: Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity METR – for Model Evaluation & Threat Research – are a non-profit research institute founded by Beth Barnes, a former alignment researcher at…
-
Cloud Blog: How Schroders built its multi-agent financial analysis research assistant
Source URL: https://cloud.google.com/blog/topics/customers/how-schroders-built-its-multi-agent-financial-analysis-research-assistant/ Source: Cloud Blog Title: How Schroders built its multi-agent financial analysis research assistant Feedly Summary: Financial analysts spend hours grappling with ever-increasing volumes of market and company data to extract key signals, combine diverse data sources, and produce company research. Schroders is a leading global active investment manager. Being an active manager…
-
Simon Willison’s Weblog: How often do LLMs snitch? Recreating Theo’s SnitchBench with LLM
Source URL: https://simonwillison.net/2025/May/31/snitchbench-with-llm/#atom-everything Source: Simon Willison’s Weblog Title: How often do LLMs snitch? Recreating Theo’s SnitchBench with LLM Feedly Summary: A fun new benchmark just dropped! Inspired by the Claude 4 system card – which showed that Claude 4 might just rat you out to the authorities if you told it to “take initiative" in…
-
Cloud Blog: Announcing new capabilities for boosted productivity in Colab Enterprise
Source URL: https://cloud.google.com/blog/products/ai-machine-learning/new-productivity-boosting-capabilities-in-colab-enterprise/ Source: Cloud Blog Title: Announcing new capabilities for boosted productivity in Colab Enterprise Feedly Summary: Colab Enterprise is a collaborative, managed notebook environment with the security and compliance capabilities of Google Cloud. Powerful integrated AI, seamless collaboration tools, enterprise readiness, and zero-config flexible compute are some of the many features making Colab…
-
Cloud Blog: Google Cloud and Spring AI 1.0
Source URL: https://cloud.google.com/blog/topics/developers-practitioners/google-cloud-and-spring-ai-10/ Source: Cloud Blog Title: Google Cloud and Spring AI 1.0 Feedly Summary: A big thank you to Fran Hinkelmann and Aaron Wanjala for their contributions and support in making this blog post happen.After a period of intense development, Spring AI 1.0 has officially landed, bringing a robust and comprehensive solution for AI…