Tag: parallelism
-
Cloud Blog: 5 best practices for Managed Lustre on Google Kubernetes Engine
Source URL: https://cloud.google.com/blog/products/containers-kubernetes/gke-managed-lustre-csi-driver-for-aiml-and-hpc-workloads/ Source: Cloud Blog Title: 5 best practices for Managed Lustre on Google Kubernetes Engine Feedly Summary: Google Kubernetes Engine (GKE) is a powerful platform for orchestrating scalable AI and high-performance computing (HPC) workloads. But as clusters grow and jobs become more data-intensive, storage I/O can become a bottleneck. Your powerful GPUs and…
-
Cloud Blog: vLLM Performance Tuning: The Ultimate Guide to xPU Inference Configuration
Source URL: https://cloud.google.com/blog/topics/developers-practitioners/vllm-performance-tuning-the-ultimate-guide-to-xpu-inference-configuration/ Source: Cloud Blog Title: vLLM Performance Tuning: The Ultimate Guide to xPU Inference Configuration Feedly Summary: Additional contributors include Hossein Sarshar, Ashish Narasimham, and Chenyang Li. Large Language Models (LLMs) are revolutionizing how we interact with technology, but serving these powerful models efficiently can be a challenge. vLLM has rapidly become…
-
Cloud Blog: Rightsizing LLM Serving on vLLM for GPUs and TPUs
Source URL: https://cloud.google.com/blog/topics/developers-practitioners/rightsizing-llm-serving-on-vllm-for-gpus-and-tpus/ Source: Cloud Blog Title: Rightsizing LLM Serving on vLLM for GPUs and TPUs Feedly Summary: Additional contributors include Hossein Sarshar and Ashish Narasimham. Large Language Models (LLMs) are revolutionizing how we interact with technology, but serving these powerful models efficiently can be a challenge. vLLM has rapidly become the primary choice for…
-
Cloud Blog: An efficient path to production AI: Kakao’s journey with JAX and Cloud TPUs
Source URL: https://cloud.google.com/blog/products/infrastructure-modernization/kakaos-journey-with-jax-and-cloud-tpus/ Source: Cloud Blog Title: An efficient path to production AI: Kakao’s journey with JAX and Cloud TPUs Feedly Summary: When your messaging platform serves 49 million people – 93% of South Korea’s population – every technical decision carries enormous weight. The engineering team at Kakao faced exactly this challenge when their existing…
-
Cloud Blog: Supercharge your AI: GKE inference reference architecture, your blueprint for production-ready inference
Source URL: https://cloud.google.com/blog/topics/developers-practitioners/supercharge-your-ai-gke-inference-reference-architecture-your-blueprint-for-production-ready-inference/ Source: Cloud Blog Title: Supercharge your AI: GKE inference reference architecture, your blueprint for production-ready inference Feedly Summary: The age of AI is here, and organizations everywhere are racing to deploy powerful models to drive innovation, enhance products, and create entirely new user experiences. But moving from a trained model in a…
-
Cloud Blog: Announcements for AI Hypercomputer: The latest infrastructure news for ML practitioners
Source URL: https://cloud.google.com/blog/products/ai-machine-learning/q2-2025-ai-hypercomputer-updates/ Source: Cloud Blog Title: Announcements for AI Hypercomputer: The latest infrastructure news for ML practitioners Feedly Summary: Curious about the latest in AI infrastructure from Google Cloud? Every three months we share a roundup of the latest AI Hypercomputer news, resources, events, learning opportunities, and more. Read on to learn new ways…
-
Cloud Blog: Your guide to taking an open model from discovery to a production-ready endpoint on Vertex AI
Source URL: https://cloud.google.com/blog/products/ai-machine-learning/take-an-open-model-from-discovery-to-endpoint-on-vertex-ai/ Source: Cloud Blog Title: Your guide to taking an open model from discovery to a production-ready endpoint on Vertex AI Feedly Summary: Developers building with gen AI are increasingly drawn to open models for their power and flexibility. But customizing and deploying them can be a huge challenge. You’re often left wrestling…
-
Cloud Blog: AI/ML-ready Apache Spark with Dataproc
Source URL: https://cloud.google.com/blog/products/data-analytics/dataproc-features-enable-aiml-ready-apache-spark/ Source: Cloud Blog Title: AI/ML-ready Apache Spark with Dataproc Feedly Summary: Apache Spark is the cornerstone for large-scale data processing, model training, and inference for AI/ML workloads. Yet, the complexities of environment configuration, dependency management, and MLOps integration can slow you down. To accelerate your AI/ML journey, Dataproc now delivers powerful, ML-ready…
-
Cloud Blog: Save early and often with multi-tier checkpointing to optimize large AI training jobs
Source URL: https://cloud.google.com/blog/products/ai-machine-learning/using-multi-tier-checkpointing-for-large-ai-training-jobs/ Source: Cloud Blog Title: Save early and often with multi-tier checkpointing to optimize large AI training jobs Feedly Summary: As foundation model training infrastructure scales to tens of thousands of accelerators, efficient utilization of those high-value resources becomes paramount. In particular, as the cluster gets larger, hardware failures become more frequent (~…
-
Cloud Blog: Accelerate your gen AI: Deploy Llama4 & DeepSeek on AI Hypercomputer with new recipes
Source URL: https://cloud.google.com/blog/products/ai-machine-learning/deploying-llama4-and-deepseek-on-ai-hypercomputer/ Source: Cloud Blog Title: Accelerate your gen AI: Deploy Llama4 & DeepSeek on AI Hypercomputer with new recipes Feedly Summary: The pace of innovation in open-source AI is breathtaking, with models like Meta’s Llama4 and DeepSeek AI’s DeepSeek. However, deploying and optimizing large, powerful models can be complex and resource-intensive. Developers and…