throughput – Experimental News Clipping Site

Cloud Blog: AlloyDB on Axion-powered C4A instances is generally available

Sep 12, 2025

—

by

Source URL: https://cloud.google.com/blog/products/databases/c4a-axion-processors-for-alloydb-now-ga/ Source: Cloud Blog Title: AlloyDB on Axion-powered C4A instances is generally available Feedly Summary: At Google Cloud Next ’25, we announced the preview of AlloyDB on C4A virtual machines, powered by Google Axion processors, our custom Arm-based CPUs. Today, we’re glad to announce that C4A virtual machines are generally available! For transactional…

Cloud Blog: Scaling high-performance inference cost-effectively

Sep 10, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://cloud.google.com/blog/products/ai-machine-learning/gke-inference-gateway-and-quickstart-are-ga/ Source: Cloud Blog Title: Scaling high-performance inference cost-effectively Feedly Summary: At Google Cloud Next 2025, we announced new inference capabilities with GKE Inference Gateway, including support for vLLM on TPUs, Ironwood TPUs, and Anywhere Cache. Our inference solution is based on AI Hypercomputer, a system built on our experience running models like…

Cloud Blog: Fast and efficient AI inference with new NVIDIA Dynamo recipe on AI Hypercomputer

Sep 10, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://cloud.google.com/blog/products/compute/ai-inference-recipe-using-nvidia-dynamo-with-ai-hypercomputer/ Source: Cloud Blog Title: Fast and efficient AI inference with new NVIDIA Dynamo recipe on AI Hypercomputer Feedly Summary: As generative AI becomes more widespread, it’s important for developers and ML engineers to be able to easily configure infrastructure that supports efficient AI inference, i.e., using a trained AI model to make…

Cloud Blog: BigQuery under the hood: The power of the Column Metadata index aka CMETA

Sep 8, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://cloud.google.com/blog/products/data-analytics/understanding-the-bigquery–column-metadata-cmeta-index/ Source: Cloud Blog Title: BigQuery under the hood: The power of the Column Metadata index aka CMETA Feedly Summary: While petabyte-scale data warehouses are becoming more common, getting the performance you need without escalating costs and effort remain key challenges, even in a modern cloud data warehouse. While many data warehouse platform…

Cloud Blog: How Baseten achieves 225% better cost-performance for AI inference (and you can too)

Sep 4, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://cloud.google.com/blog/products/ai-machine-learning/how-baseten-achieves-better-cost-performance-for-ai-inference/ Source: Cloud Blog Title: How Baseten achieves 225% better cost-performance for AI inference (and you can too) Feedly Summary: Baseten is one of a growing number of AI infrastructure providers, helping other startups run their models and experiments at speed and scale. Given the importance of those two factors to its customers,…

AWS News Blog: New general-purpose Amazon EC2 M8i and M8i Flex instances are now available

Aug 28, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://aws.amazon.com/blogs/aws/new-general-purpose-amazon-ec2-m8i-and-m8i-flex-instances-are-now-available/ Source: AWS News Blog Title: New general-purpose Amazon EC2 M8i and M8i Flex instances are now available Feedly Summary: M8i and M8i-flex instances powered by Intel Xeon processors offer up to 15% better price performance, 20% higher performance, and 2.5 times more memory throughput compared to previous generations. AI Summary and Description:…

Cloud Blog: vLLM Performance Tuning: The Ultimate Guide to xPU Inference Configuration

Aug 25, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://cloud.google.com/blog/topics/developers-practitioners/vllm-performance-tuning-the-ultimate-guide-to-xpu-inference-configuration/ Source: Cloud Blog Title: vLLM Performance Tuning: The Ultimate Guide to xPU Inference Configuration Feedly Summary: Additional contributors include Hossein Sarshar, Ashish Narasimham, and Chenyang Li. Large Language Models (LLMs) are revolutionizing how we interact with technology, but serving these powerful models efficiently can be a challenge. vLLM has rapidly become…

AWS News Blog: AWS Weekly Roundup: Amazon Aurora 10th anniversary, Amazon EC2 R8 instances, Amazon Bedrock and more (August 25, 2025)

Aug 25, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://aws.amazon.com/blogs/aws/aws-weekly-roundup-amazon-aurora-10th-anniversary-amazon-ec2-r8-instances-amazon-bedrock-and-more-august-25-2025/ Source: AWS News Blog Title: AWS Weekly Roundup: Amazon Aurora 10th anniversary, Amazon EC2 R8 instances, Amazon Bedrock and more (August 25, 2025) Feedly Summary: As I was preparing for this week’s roundup, I couldn’t help but reflect on how database technology has evolved over the past decade. It’s fascinating to see…

AWS News Blog: Best performance and fastest memory with the new Amazon EC2 R8i and R8i-flex instances

Aug 19, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://aws.amazon.com/blogs/aws/best-performance-and-fastest-memory-with-the-new-amazon-ec2-r8i-and-r8i-flex-instances/ Source: AWS News Blog Title: Best performance and fastest memory with the new Amazon EC2 R8i and R8i-flex instances Feedly Summary: R8i and R8i-flex instances powered by Intel Xeon processors offer up to 15% better price performance, 20% higher performance, and 2.5 times more memory throughput compared to previous generations. AI Summary…

Cloud Blog: Rightsizing LLM Serving on vLLM for GPUs and TPUs

Aug 19, 2025

—

by

Kurt Seifried

in Uncategorized

Source URL: https://cloud.google.com/blog/topics/developers-practitioners/rightsizing-llm-serving-on-vllm-for-gpus-and-tpus/ Source: Cloud Blog Title: Rightsizing LLM Serving on vLLM for GPUs and TPUs Feedly Summary: Additional contributors include Hossein Sarshar and Ashish Narasimham. Large Language Models (LLMs) are revolutionizing how we interact with technology, but serving these powerful models efficiently can be a challenge. vLLM has rapidly become the primary choice for…

Tag: throughput