Tag: Trillium
-
Cloud Blog: vLLM Performance Tuning: The Ultimate Guide to xPU Inference Configuration
Source URL: https://cloud.google.com/blog/topics/developers-practitioners/vllm-performance-tuning-the-ultimate-guide-to-xpu-inference-configuration/ Source: Cloud Blog Title: vLLM Performance Tuning: The Ultimate Guide to xPU Inference Configuration Feedly Summary: Additional contributors include Hossein Sarshar, Ashish Narasimham, and Chenyang Li. Large Language Models (LLMs) are revolutionizing how we interact with technology, but serving these powerful models efficiently can be a challenge. vLLM has rapidly become…
-
Cloud Blog: Rightsizing LLM Serving on vLLM for GPUs and TPUs
Source URL: https://cloud.google.com/blog/topics/developers-practitioners/rightsizing-llm-serving-on-vllm-for-gpus-and-tpus/ Source: Cloud Blog Title: Rightsizing LLM Serving on vLLM for GPUs and TPUs Feedly Summary: Additional contributors include Hossein Sarshar and Ashish Narasimham. Large Language Models (LLMs) are revolutionizing how we interact with technology, but serving these powerful models efficiently can be a challenge. vLLM has rapidly become the primary choice for…
-
Cloud Blog: An efficient path to production AI: Kakao’s journey with JAX and Cloud TPUs
Source URL: https://cloud.google.com/blog/products/infrastructure-modernization/kakaos-journey-with-jax-and-cloud-tpus/ Source: Cloud Blog Title: An efficient path to production AI: Kakao’s journey with JAX and Cloud TPUs Feedly Summary: When your messaging platform serves 49 million people – 93% of South Korea’s population – every technical decision carries enormous weight. The engineering team at Kakao faced exactly this challenge when their existing…
-
Cloud Blog: Announcements for AI Hypercomputer: The latest infrastructure news for ML practitioners
Source URL: https://cloud.google.com/blog/products/ai-machine-learning/q2-2025-ai-hypercomputer-updates/ Source: Cloud Blog Title: Announcements for AI Hypercomputer: The latest infrastructure news for ML practitioners Feedly Summary: Curious about the latest in AI infrastructure from Google Cloud? Every three months we share a roundup of the latest AI Hypercomputer news, resources, events, learning opportunities, and more. Read on to learn new ways…
-
Cloud Blog: 25+ top gen AI how-to guides for enterprise
Source URL: https://cloud.google.com/blog/products/ai-machine-learning/top-gen-ai-how-to-guides-for-enterprise/ Source: Cloud Blog Title: 25+ top gen AI how-to guides for enterprise Feedly Summary: The best way to learn AI is by building. From finding quick ways to deploy open models to building complex, multi-agentic systems, it’s easy to feel overwhelmed by the sheer volume of resources out there. To that end,…
-
Cloud Blog: Accelerate your gen AI: Deploy Llama4 & DeepSeek on AI Hypercomputer with new recipes
Source URL: https://cloud.google.com/blog/products/ai-machine-learning/deploying-llama4-and-deepseek-on-ai-hypercomputer/ Source: Cloud Blog Title: Accelerate your gen AI: Deploy Llama4 & DeepSeek on AI Hypercomputer with new recipes Feedly Summary: The pace of innovation in open-source AI is breathtaking, with models like Meta’s Llama4 and DeepSeek AI’s DeepSeek. However, deploying and optimizing large, powerful models can be complex and resource-intensive. Developers and…
-
Cloud Blog: AI Hypercomputer developer experience enhancements from Q1 25: build faster, scale bigger
Source URL: https://cloud.google.com/blog/products/compute/ai-hypercomputer-enhancements-for-the-developer/ Source: Cloud Blog Title: AI Hypercomputer developer experience enhancements from Q1 25: build faster, scale bigger Feedly Summary: Building cutting-edge AI models is exciting, whether you’re iterating in your notebook or orchestrating large clusters. However, scaling up training can present significant challenges, including navigating complex infrastructure, configuring software and dependencies across numerous…
-
Cloud Blog: From LLMs to image generation: Accelerate inference workloads with AI Hypercomputer
Source URL: https://cloud.google.com/blog/products/compute/ai-hypercomputer-inference-updates-for-google-cloud-tpu-and-gpu/ Source: Cloud Blog Title: From LLMs to image generation: Accelerate inference workloads with AI Hypercomputer Feedly Summary: From retail to gaming, from code generation to customer care, an increasing number of organizations are running LLM-based applications, with 78% of organizations in development or production today. As the number of generative AI applications…
-
Cloud Blog: 229 things we announced at Google Cloud Next 25 – a recap
Source URL: https://cloud.google.com/blog/topics/google-cloud-next/google-cloud-next-2025-wrap-up/ Source: Cloud Blog Title: 229 things we announced at Google Cloud Next 25 – a recap Feedly Summary: Google Cloud Next 25 took place this week and we’re all still buzzing! It was a jam-packed week in Las Vegas complete with interactive experiences, including more than 10 keynotes and spotlights, 700 sessions,…
-
Cloud Blog: New GKE inference capabilities reduce costs, tail latency and increase throughput
Source URL: https://cloud.google.com/blog/products/containers-kubernetes/understanding-new-gke-inference-capabilities/ Source: Cloud Blog Title: New GKE inference capabilities reduce costs, tail latency and increase throughput Feedly Summary: When it comes to AI, inference is where today’s generative AI models can solve real-world business problems. Google Kubernetes Engine (GKE) is seeing increasing adoption of gen AI inference. For example, customers like HubX run…