Source URL: https://cloud.google.com/blog/products/ai-machine-learning/how-signal-iduna-supercharges-customer-service-with-gen-ai/
Source: Cloud Blog
Title: How SIGNAL IDUNA supercharges customer service with gen AI
Feedly Summary: Today’s insurance customers expect more: simple digital services, instant access to service representatives when they want to discuss personal matters, and quick feedback on submitted invoices. Meeting these demands has become increasingly difficult for insurers due to rising inquiry volumes, a shortage of skilled workers, and the loss of expertise as employees retire.
Recognizing the growing need for immediate and accurate responses, SIGNAL IDUNA, a leading German full-service insurer, particularly prominent in health insurance, introduced a cutting-edge AI knowledge assistant, powered by Google Cloud generative AI.
“We’ve pioneered to unlock the power of human-AI collaboration: To redefine process efficiency by bringing together technology and subject matter experts to deliver exceptional customer experiences,” said Johannes Rath, board member for Customer, Service, and Transformation at SIGNAL IDUNA.
SIGNAL IDUNA, in collaboration with Google Cloud, BCG and Deloitte, has developed an AI knowledge assistant that empowers service agents to quickly and accurately resolve complex customer inquiries. This innovative solution uses Google Cloud AI, including Google’s multimodal Gemini models, to help agents find relevant documents and provide comprehensive answers 30% faster — ultimately, enhancing customer satisfaction.
aside_block
The Challenge: Meeting modern expectations
Like many organizations in the insurance sector, SIGNAL IDUNA faced significant operational burdens. The complexity of insurance products, along with the growing demand for immediate and accurate responses, often leads to bottlenecks that can impact service experiences.
For example, prior to introducing its AI knowledge assistant, service agents had to manually search thousands of internal documents for hundreds of different tariffs to find the information needed to answer questions or resolve customer issues — including, insurance conditions, tariff information, guidelines, and standard operating procedures. As a result, 27% of inquiries required further escalation to other departments or specialists, resulting in delayed resolutions, increased costs, and potential damage to reputation.
Though complex, SIGNAL IDUNA prioritized this process as one of its top gen AI use cases, developing an AI assistant to help agents provide quick and accurate answers to customer inquiries, particularly those about health insurance. The AI knowledge assistant is grounded in more than 2,000 internal documents for more than 600 different tariffs, allowing agents to ask questions in natural language and receive accurate answers, significantly reducing the time spent searching for relevant information.
A deep dive into SIGNAL IDUNA’s gen AI system
Working with Google Cloud, BCG, and Deloitte, SIGNAL IDUNA built a sophisticated generative AI architecture using Google Cloud’s AI platform, Vertex AI, and utilized Gemini 1.5 Pro’s long-context capabilities to develop an AI knowledge assistant that can provide quick and accurate access to the right information within a vast collection of documents. The system employs multiple steps to aggregate and process extensive information from diverse sources, ensuring agents can access the complete context necessary to effectively address customer inquiries.
Here’s a breakdown of the key steps:
An end-to-end architecture diagram
1. Data pre-processing and extractionThe knowledge base is built from various document types, which are typically in PDF format, including policy documents, operating procedures, and general terms and conditions.
SIGNAL IDUNA utilizes a hybrid approach that combines Layout Parser in Google Cloud Document AI and PDFPlumber to parse these PDFs and extract the text content. While the Layout Parser is responsible for extracting the text segments, SIGNAL IDUNA enhances the extraction of tables with PDFPlumber if the quality of the PDFs allows. The extracted texts are then cleaned, chunked by Google’s Gecko multilingual embedding model, and enhanced with additional metadata, enabling the ability to process and analyze the information later effectively.
For storing the vectorized texts, Google Cloud SQL for PostgreSQL is used with the pgvector PostgreSQL extension, which provides a highly effective vector database solution for our needs. By storing vectorized text chunks in Cloud SQL, SIGNAL IDUNA benefits from its scalability, reliability, and seamless integration with other Google Cloud services, while pgvector empowers efficient similarity search functionality.
2. Query augmentationQuery augmentation generates multiple queries to improve the formulation of user questions for both document retrieval from the vector store and answer generation. The original question is reformulated into several variants, creating three versions in total: the original query, a rewritten query, and an imitation query. These are used then to retrieve relevant documents and generate the final answer.
For the rewritten query, the system uses Gemini Pro 1.5 to correct spelling errors in the original question. Additionally, the query is expanded by adding synonyms for predefined terms and tagging specific terms (e.g., “remedies," "assistive devices," "wahlleistung/selective benefits") with categories. The system also uses information about selected tariffs to enrich the query. For example, tariff attributes, such as brand or contract type, are extracted from a database and appended to the query in a structured format. These specific adjustments make it possible to handle special tariff codes and add further context based on tariff prefixes.
The imitation query uses Gemini Pro 1.5 to rephrase the question to mimic the language of technical insurance documents, improving the semantic similarity with the source material. It considers conversation history and handles age formatting.
3. RetrievalFirst, the system checks the query cache, which stores previously answered questions and their corresponding correct answers. If the question, or one very similar to it, has already been successfully resolved, the cached answer is retrieved, helping to provide a rapid answer. This efficient approach ensures quick access to information and avoids redundant processing.
The accuracy of the cache is maintained through a user feedback loop, which identifies correctly answered questions to be stored in the cache through upvotes. A downvote on a cached answer triggers an immediate cache invalidation, ensuring only relevant and helpful responses are served. This dynamic approach improves the efficiency and accuracy of the system over time. If no matching questions are found in the query cache, the retrieval process falls back on the vector store, ensuring that the system can answer novel questions.
After retrieving any relevant information chunks from the query cache or vector store, the system uses the Vertex AI ranking API to rerank them. This crucial process analyzes various signals to refine the results, prioritizing relevance and ensuring the most accurate and helpful information is presented.
Ensuring complete and accurate answers is paramount during retrieval, and SIGNAL IDUNA found that some queries required information beyond what was available in the source documents. To address this issue, the system uses keyword-based augmentations to supplement the final prompt, providing a more comprehensive context for generating responses.
4. GenerationThe answer generation process involves three key components: the user’s question with multiple queries, retrieved chunks of relevant information, and augmentations that add further context. These elements are combined to create the final response using a complex prompt template.
Delivering a near real-time experience is crucial for service agents, so SIGNAL IDUNA also streams the generated response. During development, minimizing latency based on the input posed a significant technical hurdle. To address this issue, SIGNAL IDUNA reduced processing times using asynchronous APIs to help stream data and handle multiple requests. Currently, the system has achieved an average response time of approximately 6 seconds, and SIGNAL IDUNA is experimenting with newer faster models to reduce this time even further.
5. EvaluationRigorous evaluation is essential for optimizing Retrieval Augmented Generation (RAG) systems. SIGNAL IDUNA uses the Gen AI evaluation service in Vertex AI to automate the assessment of both response quality and the performance of all process components, such as retrieval. A comprehensive question set, created with input from SIGNAL IDUNA’s service agents, forms the basis of these automated tests.
The evaluation results flow seamlessly into Vertex AI Experiments and Google Cloud BigQuery. This enables SIGNAL IDUNA to visualize performance trends and gain actionable insights using dashboards with Looker on Google Cloud.
Here’s a closer look at how Looker helps evaluate the AI knowledge assistant:
Chunk retrieval: First, SIGNAL IDUNA evaluates retrieval of relevant information chunks. Metrics at this stage help assess how effectively the model identifies and gathers the necessary information from the source data. This includes tracking gen AI metrics, such as recall, precision, and F1-score, to pinpoint areas for improvement in the retrieval process. This is crucial as retrieving the correct information is the foundation of a good generated response.
Document reranking: Once the relevant chunks are retrieved, they’re reranked to prioritize the most pertinent information. The Looker dashboard allows monitoring the effectiveness of this reranking process.
Generated vs. expected response comparison: The final stage involves comparing the generated response with the expected response. SIGNAL IDUNA evaluates the quality, accuracy, and completeness of the generated output, utilizing large language models (LLMs) to score the similarity between the generated response and the expected response.
Explanation generation: To understand the reasoning behind an LLM’s evaluation, SIGNAL IDUNA generates explanations for its judgments. This provides valuable insights into the strengths and weaknesses of the generated responses, helping the developers identify specific areas for improvement.
This multi-stage evaluation approach provides SIGNAL IDUNA a holistic view of the model’s performance, enabling data-driven optimization at each stage. The Looker dashboard plays a vital role in visualizing these metrics, making it easier for the developers to identify areas where the model excels and where it needs improvement.
Real-world impact: AI-powered efficiency and productivity
To determine whether the AI assistant provided measurable added value for its workforce, SIGNAL IDUNA conducted an experiment with a total of 20 employees (internal and with external providers). During the experiment, customer requests were processed with and without the AI knowledge assistant to assess its impact.
One of the key benefits observed was a reduction in processing time. Searching across numerous data sources used to be a time-consuming process. The experiment showed that using the AI knowledge assistant reduced the core processing time (information search and response formulation) by approximately 30% and increased the quality of the response based on expert evaluations. The time saved was particularly notable for employees with less than two years of experience in health insurance.
In addition, the AI knowledge assistant significantly increased the case closure rate. Health insurance is a very complex field, and the use of external service providers means that not every employee can always answer every customer question. With support from the AI knowledge assistant, SIGNAL IDUNA’s case closure rate increased by approximately 24 percentage points, rising from 73% to almost 98%.
Scaling for the Future
“Together with Google, we at SIGNAL IDUNA have successfully applied gen AI to one of our core business processes” Stefan Lemke, CIO at SIGNAL IDUNA, said. “Now, it’s time to scale this powerful technology across our entire organization. We’re not just scaling a tool, we’re scaling innovation, learning, and the possibilities of what we can achieve.”
Gen AI offers enormous potential for optimizing processes and developing innovative solutions. With its innovative approach — business teams experimenting with the technology in a decentralized manner and developing customized applications — SIGNAL IDUNA is primed to pioneer the next generation of insurance solutions and services.
At the same time, SIGNAL IDUNA is establishing central standards to scale insights gained across the company and tap into the combined power of its teams, resources, and lines of business. This strategic decision has helped create valuable resources like code libraries, infrastructure blueprints, and centrally offered services.
By combining agility with established standards and best practices, SIGNAL IDUNA can now react quickly to new requirements, setting a new standard for efficiency and customer satisfaction.
This project was delivered by the following core team members, Max Tschochohei, Anant Nawalgaria, and Corinna Ludwig by Google, and Christopher Masch, Michelle Mäding from SIGNAL IDUNA
AI Summary and Description: Yes
Summary: The text showcases how SIGNAL IDUNA, in collaboration with Google Cloud, has implemented a generative AI knowledge assistant to meet the modern expectations of insurance customers. This AI solution significantly enhances operational efficiency by enabling service agents to resolve inquiries faster and with greater accuracy, thereby improving overall customer satisfaction.
Detailed Description: The analysis reveals the growing necessity for insurance providers to adapt to the increasing demands of customers, particularly in terms of immediate and accurate service. SIGNAL IDUNA’s initiative to create an AI knowledge assistant exemplifies the merging of advanced technology with insurance expertise to streamline communication and service delivery.
– **Key Innovations:**
– **Generative AI Assistant**: Utilizes generative AI from Google Cloud to assist service agents in answering complex customer inquiries.
– **Efficiency Gains**: The AI solution improves response times by approximately 30%, allowing agents to provide comprehensive answers faster, which is crucial in health insurance.
– **Operational Challenges Addressed**:
– Rising inquiry volumes and a lack of skilled workforce have made it challenging for insurers to provide timely responses.
– The AI assistant addresses these issues by reducing the need for manual document searches, thus speeding up the resolution of customer inquiries.
– **Technical Implementation**:
– The system is built on Google Cloud’s Vertex AI and employs sophisticated data extraction methods from various document formats.
– A hybrid approach combines multiple tools for document parsing and ensures scalable data storage with optimized search capabilities.
– **Process Workflow**:
– The AI system uses a structured process that entails data pre-processing, query augmentation for effective search and generation of responses, retrieval of information, and rigorous evaluation of the answer quality.
– **Impact Measurement**:
– An experiment indicated a notable reduction in processing time and an increase in case closure rates, highlighting the tangible benefits of the AI assistant.
– The increase in case closure rates from 73% to 98% underscores the assistant’s effectiveness in facilitating quicker resolutions.
– **Future Aspirations**:
– SIGNAL IDUNA aims to scale this generative AI technology across its organization, which could revolutionize how insurance solutions and services are delivered.
– The strategic rollout intends to combine agility with established standards and practices to further enhance operational efficiencies and customer satisfaction.
Overall, this case study not only highlights the practical application of generative AI in the insurance industry but also sets a compelling example for the integration of advanced AI technologies in service-oriented sectors to meet evolving customer needs.