Experimental News Clipping Site

Tag: batch size

  • Simon Willison’s Weblog: Defeating Nondeterminism in LLM Inference

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Sep/11/defeating-nondeterminism/#atom-everything Source: Simon Willison’s Weblog Title: Defeating Nondeterminism in LLM Inference Feedly Summary: Defeating Nondeterminism in LLM Inference A very common question I see about LLMs concerns why they can’t be made to deliver the same response to the same prompt by setting a fixed random number seed. Like many others I had…

  • Cloud Blog: How Baseten achieves 225% better cost-performance for AI inference (and you can too)

    —

    by

    Kurt Seifried
    in

    Source URL: https://cloud.google.com/blog/products/ai-machine-learning/how-baseten-achieves-better-cost-performance-for-ai-inference/ Source: Cloud Blog Title: How Baseten achieves 225% better cost-performance for AI inference (and you can too) Feedly Summary: Baseten is one of a growing number of AI infrastructure providers, helping other startups run their models and experiments at speed and scale. Given the importance of those two factors to its customers,…

  • Cloud Blog: Announcing a new monitoring library to optimize TPU performance

    —

    by

    Kurt Seifried
    in

    Source URL: https://cloud.google.com/blog/products/compute/new-monitoring-library-to-optimize-google-cloud-tpu-resources/ Source: Cloud Blog Title: Announcing a new monitoring library to optimize TPU performance Feedly Summary: For more than a decade, TPUs have powered Google’s most demanding AI training and serving workloads. And there is strong demand from customers for Cloud TPUs as well. When running advanced AI workloads, you need to be…

  • Cloud Blog: Train AI for less: Improve ML Goodput with elastic training and optimized checkpointing

    —

    by

    Kurt Seifried
    in

    Source URL: https://cloud.google.com/blog/products/ai-machine-learning/elastic-training-and-optimized-checkpointing-improve-ml-goodput/ Source: Cloud Blog Title: Train AI for less: Improve ML Goodput with elastic training and optimized checkpointing Feedly Summary: Want to save some money on large AI training? For a typical PyTorch LLM training workload that spans thousands of accelerators for several weeks, a 1% improvement in ML Goodput can translate to…

  • Cloud Blog: From LLMs to image generation: Accelerate inference workloads with AI Hypercomputer

    —

    by

    Kurt Seifried
    in

    Source URL: https://cloud.google.com/blog/products/compute/ai-hypercomputer-inference-updates-for-google-cloud-tpu-and-gpu/ Source: Cloud Blog Title: From LLMs to image generation: Accelerate inference workloads with AI Hypercomputer Feedly Summary: From retail to gaming, from code generation to customer care, an increasing number of organizations are running LLM-based applications, with 78% of organizations in development or production today. As the number of generative AI applications…

  • Simon Willison’s Weblog: Nomic Embed Code: A State-of-the-Art Code Retriever

    —

    by

    Kurt Seifried
    in

    Source URL: https://simonwillison.net/2025/Mar/27/nomic-embed-code/ Source: Simon Willison’s Weblog Title: Nomic Embed Code: A State-of-the-Art Code Retriever Feedly Summary: Nomic Embed Code: A State-of-the-Art Code Retriever Nomic have released a new embedding model that specializes in code, based on their CoRNStack “large-scale high-quality training dataset specifically curated for code retrieval". The nomic-embed-code model is pretty large –…