Source URL: https://cloud.google.com/blog/products/ai-machine-learning/ai-edge-portal-brings-on-device-ml-testing-at-scale/
Source: Cloud Blog
Title: Google AI Edge Portal: On-device machine learning testing at scale
Feedly Summary: Today, we’re excited to announce Google AI Edge Portal in private preview, Google Cloud’s new solution for testing and benchmarking on-device machine learning (ML) at scale.
Machine learning on mobile devices enables amazing app experiences. But how will your model truly perform across the vast, diverse, and ever-changing landscape of mobile devices? Manually testing at scale – across hundreds of device types – is a laborious task that often requires a dedicated device lab. It’s slow, prohibitively expensive, and often out of reach to most developers, leaving you guessing about performance on users’ devices and risking delivering a subpar user experience.
Google AI Edge Portal solves the above challenges, enabling you to benchmark LiteRT models so you can find the best configuration for large-scale deployment of ML models across devices. Now, you can:
Simplify & accelerate testing cycles across the diverse hardware landscape: Effortlessly assess model performance across hundreds of representative mobile devices in minutes.
Proactively assure model quality & identify issues early: Pinpoint hardware-specific performance variations or regressions (like on particular chipsets or memory-constrained devices) before deployment.
Lower device testing cost & access latest hardware: Test on diverse and continually growing fleet of physical devices (currently 100+ device models from various Android OEMs) without the expense and complexity of maintaining your own lab.
Unlock powerful, data-driven decisions & business intelligence: Google AI Edge Portal delivers rich performance data and comparisons, providing the crucial business intelligence needed to confidently guide model optimization and validate deployment readiness.
Fig. 1. Interactive dashboard to gain insights on model performance across devices
In this post, we’ll share how our partners are already using Google AI Edge Portal, the user journey, and how you can get started.
What our partners are saying
We’ve been fortunate to work with several innovative teams during the early development of Google AI Edge Portal. Here’s what a few of them had to say about its potential:
How Google AI Edge Portal helps you benchmark your LiteRT models
Upload & configure: Upload your model file via the UI or point to it in your Google Cloud Storage bucket.
Select accelerators: Specify testing against CPU or GPU (with automatic CPU fallback). NPU support is planned for future releases.
Select devices: Choose target devices from our diverse pool using filters (device tier, brand, chipset, RAM) or select curated lists with convenient shortcuts.
Fig. 2. Create a New Benchmark Job on 100+ Devices. (Note: GIF is accelerated and edited for brevity)
From there, submit your job and await completion. Once ready, explore the results in the Interactive Dashboard:
Compare configurations: Easily visualize how performance metrics (e.g., average latency, peak memory) differ when using different accelerators across all tested devices.
Analyze device impact: See how a specific model configuration performs across the range of selected devices. Use histograms and scatter plots to quickly identify performance variations tied to device characteristics.
Detailed metrics: Access a detailed, sortable table showing specific metrics (initialization time, inference latency, memory usage) for each individual device, alongside its hardware specifications.
Fig. 3. View Benchmark Results on the interactive Dashboard. (Note: GIF is accelerated and edited for brevity)
Help us shape the future of Google AI Edge Portal
Your feedback is crucial as we expand availability and enhance capabilities based on developer needs. In the future, we are keen to explore integrating features such as:
Bulk inference & evaluation: Run your models with custom datasets on diverse devices to validate functional correctness and enable qualitative GenAI evaluations.
LLM benchmarking: Introduce dedicated workflows and metrics specifically tailored for benchmarking the unique characteristics of large language models on edge devices.
Model optimization tools: Explore integrated tooling to potentially assist with tasks like model conversion and quantization within the portal.
Expanded platform & hardware support: Work towards supporting additional accelerators like NPUs, and other platforms beyond Android in the future.
aside_block
Join the Google AI Edge Portal private preview
Google AI Edge Portal is available starting today in private preview for allowlisted Google Cloud customers. During this private preview period, access is provided at no charge, subject to the preview terms.
This preview is ideal for developers and teams building mobile ML applications with LiteRT who need reliable benchmarking data across diverse Android hardware and are willing to provide feedback to help shape the product’s future. To request access, complete our sign-up form here to express interest. Access is granted via allowlisting.
We are committed to making Google AI Edge Portal a valuable tool for the entire on-device ML community and we look forward to your feedback and collaboration!
AI Summary and Description: Yes
**Summary:** The text announces Google AI Edge Portal, a new Google Cloud solution designed to facilitate benchmarking and testing of on-device machine learning models across diverse mobile devices. It addresses the challenges of traditional manual testing, offering a streamlined approach to assess performance, ensure quality, and optimize deployment. This innovation is particularly relevant for developers focused on improving machine learning applications in mobile environments.
**Detailed Description:**
The announcement introduces the Google AI Edge Portal, focusing on its capabilities and benefits for developers engaged in machine learning (ML) for mobile devices. Here are the main points of significance:
– **Benchmarking and Testing Solution**: The portal provides a managed environment for testing machine learning models, particularly LiteRT models, across a wide variety of mobile devices without requiring developers to maintain expensive device labs.
– **Performance Assessment**:
– **Scale and Variety**: It enables testing across hundreds of device types rapidly, allowing developers to obtain data on model performance in a matter of minutes.
– **Hardware-Specific Insights**: The platform helps identify hardware-specific performance variations, ensuring that potential issues can be caught early in the development cycle.
– **Cost Reduction**: By utilizing Google’s existing fleet of over 100 devices from various Android manufacturers, developers can lower their testing costs significantly. This reduces the complexities and expenses associated with maintaining their own hardware labs.
– **Data-Driven Decisions**: The portal provides comprehensive performance data and business intelligence. Developers can visualize comparison metrics, which is crucial for optimizing models and validating their readiness for deployment.
– **User Interface and Dashboard**: The interactive dashboard allows for easy exploration of benchmark results. Users can:
– **Configure Tests**: Upload models, select device configurations, and choose specific accelerators like CPUs or GPUs.
– **Analyze Metrics**: The platform supports visualizations of latency, memory usage, and other performance metrics across different devices.
– **Future Features**: The announcement hints at plans to enhance the portal with:
– Bulk inference and evaluation capabilities.
– Specific benchmarking workflows for large language models (LLMs).
– Optimization tools for model conversion and quantization.
– Expanded hardware support, including upcoming functionalities for NPUs.
– **Private Preview Access**: The portal is currently available in private preview for selected Google Cloud customers, inviting feedback for further development.
In summary, Google AI Edge Portal represents a significant advancement in ML application development for mobile platforms, emphasizing the need for robust, scalable solutions that address performance validation across a vast device ecosystem. This tool is poised to be invaluable for developers aiming to improve user experience by ensuring their models function optimally on diverse hardware.