Source URL: https://cloud.google.com/blog/products/business-intelligence/how-lookers-semantic-layer-enhances-gen-ai-trustworthiness/
Source: Cloud Blog
Title: How Looker’s semantic layer enables trusted AI for business intelligence
Feedly Summary: In the AI era, where data fuels intelligent applications and drives business decisions, demand for accurate and consistent data insights has never been higher. However, the complexity and sheer volume of data coupled with the diversity of tools and teams can lead to misunderstandings and inaccuracies. That’s why trusted definitions managed by a semantic layer become indispensable. Armed with unique information about your business, with standardized references, the semantic layer provides a business-friendly and consistent interpretation of your data, so that your AI initiatives and analytical endeavors are built on a foundation of truth and can drive reliable outcomes.
Looker’s semantic layer acts as a single source of truth for business metrics and dimensions, helping to ensure that your organization and tools are leveraging consistent and well-defined terms. By doing so, the semantic layer offers a foundation for generative AI tools to interpret business logic, not simply raw data, meaning answers are accurate, thanks to critical signals that map to business language and user intent, reducing ambiguity. LookML (Looker Modeling Language) helps you create the semantic model that empowers your organization to define the structure of your data and its logic, and abstracts complexity, easily connecting your users to the information they need.
A semantic layer is particularly important in the context of gen AI. When applied directly to ungoverned data, gen AI can produce impressive, but fundamentally inaccurate and inconsistent results. It sometimes miscalculates important variables, improperly groups data, or misinterprets definitions, including when writing complex SQL. The result can be misguided strategy and missed revenue opportunities.
In any data-driven organization, trustworthy business information is non-negotiable. Our own internal testing has shown that Looker’s semantic layer reduces data errors in gen AI natural language queries by as much as two thirds. According to a recent report by Enterprise Strategy Group, ensuring data quality and consistency proved to be the top challenge for organizations’ analytics and business intelligence platform. Looker provides a single source of truth, ensuring data accuracy and delivering trusted business logic for the entire organization and all connected applications.
aside_block
The foundation of trustworthy Gen AI
To truly trust gen AI, it needs to be anchored to a robust semantic layer, which acts as your organization’s data intelligence engine, providing a centralized, governed framework that defines your core business concepts and helping to ensure a single, consistent source of truth.
The semantic layer is essential to deliver on the promise of trustworthy gen AI for BI, offering:
Trust: Reduce gen AI “hallucinations" by grounding AI responses in governed, consistently defined data.
Deep business context: AI and data agents should know your business as well as your analysts do. You can empower those agents with an understanding of your business language, metrics, and relationships to accurately interpret user queries and deliver relevant answers.
Governance: Enforce your existing data security and compliance policies within the gen AI environment, protecting sensitive information and providing auditable data access.
Organizational alignment: Deliver data consistency across your entire organization, so every user, report and AI-driven insight are using the same definitions and terms and referring to them the same way.
LookML improves accuracy and reduces large language model guesswork
The semantic layer advantage in the gen AI era
LookML, Looker’s semantic modeling language, is architected for the cloud and offers a number of critical values for fully integrating gen AI in BI:
Centralized definitions: Experts can define metrics, dimensions, and join relationships once, to be re-used across all Looker Agents, chats and users, ensuring consistent answers that get everyone on the same page.
Deterministic advanced calculations: Ideal for complex mathematical or logistical operations, Looker eliminates randomness and provides predictable and repeatable outcomes. Additionally, our dimensionalized measures capability aggregates values so you can perform operations on them as a group, letting you perform complex actions quickly and simply.
Software engineering best practices: With continuous integration and version control, Looker ensures code changes are frequently tested and tracked, keeping production applications running smoothly.
Time-based analysis: Built-in dimension groups allow for time-based and duration-based calculations.
Deeper data drills: Drill fields allow users to explore data in detail through exploration of a single data point. Data agents can tap into this capability and assist users to dive deeper into different slices of data.
With the foundation of a semantic layer, rather than asking an LLM to write SQL code against raw tables with ambiguous field names (e.g., order.sales_sku_price_US), the LLM is empowered to do what it excels at: searching through clearly defined business objects within LookML (e.g., Orders > Total Revenue). These objects can include metadata and human-friendly descriptions (e.g., "The sum of transaction amounts or total sales price"). This is critical when business users speak in the language of business — “show me revenue” — versus the language of data — ”show me sum of sales (price), not quantity.” LookML bridges the data source and what a decision-maker cares about, so an LLM can better identify the correct fields, filters, and sorts and turn data agents into intelligent ad-hoc analysts.
LookML offers you a well-structured library catalog for your data, enabling an AI agent to find relevant information and summaries, so it can accurately answer your question. Looker then handles the task of actually retrieving that information from the right place.
The coming together of AI and BI promises intelligent, trustworthy and conversational insights. Looker’s semantic layer empowers our customers to gain benefit from these innovations in all the surfaces where they engage with their data. We will continue to expand support for a wide variety of data sources, enrich agent intelligence, and add functionality to conversational analytics to make data interaction as intuitive and powerful as a conversation with your most trusted business advisor.
To gain the full benefits of Looker’s semantic layer and Conversation Analytics, get started here. To learn more about the Conversational Analytics API, see our recent update from Google Cloud Next, or sign up here for preview access.
AI Summary and Description: Yes
Summary: The text discusses the importance of a semantic layer in ensuring data quality and consistency within AI-driven applications, specifically emphasizing its role in enhancing the reliability and accuracy of generative AI outputs. It outlines how Looker’s semantic layer facilitates trustworthy insights by providing standardized definitions and governance, enabling organizations to minimize data errors and improve business intelligence.
Detailed Description:
– The text highlights the rising demand for accurate data insights in the context of AI, where the complexity and volume of data can lead to misinterpretations without foundational governance.
– It emphasizes the necessity of a semantic layer to manage and define business metrics consistently, which ultimately influences AI initiatives and analytical efforts.
– Looker’s semantic layer is presented as a solution that ensures all users and applications operate using the same terminology, thus supporting reliable outputs from generative AI tools. Key points include:
– **Reduction of AI Errors**: The semantic layer reduces misunderstandings and inaccuracies in data interpretation during AI usage, minimizing errors in generative AI queries.
– **Improved Data Quality**: Internal testing indicated that Looker’s semantic layer can decrease data errors by up to two-thirds in generative AI applications.
– **Trust & Governance**: Empowering generative AI with a governed semantic structure reinforces compliance with existing data security policies and helps secure sensitive information.
– **Centralized Definitions**: LookML allows specialists to create reusable metrics and definitions, ensuring uniformity in AI responses and enhancing overall clarity in data points.
– **Support for Complex Calculations**: By providing deterministic outcomes and facilitating advanced calculations within a structured framework, LookML optimizes efficiency in outputs derived from AI analyses.
– **Enhanced Understanding**: The semantic layer empowers AI agents to interpret business inquiries more accurately by framing the data in a way that resonates with business terminology, thus bridging the gap between technical and user-friendly language.
– The text underscores the strategic significance of Looker’s features in modern data environments, integrating AI into business intelligence to produce intelligent and trustworthy insights. The growth of conversational analytics and the overall synergy between AI and business intelligence are suggested as areas for continued innovation and development.
The implications for security, privacy, and compliance professionals include navigating the complexities of data governance while implementing solutions like Looker’s semantic layer to enhance accuracy and trust in AI-driven data insights.