Source URL: https://slashdot.org/story/24/09/25/0452207/microsoft-claims-its-new-tool-can-correct-ai-hallucinations?utm_source=rss1.0mainlinkanon&utm_medium=feed
Source: Slashdot
Title: Microsoft Claims Its New Tool Can Correct AI Hallucinations
Feedly Summary:
AI Summary and Description: Yes
Summary: Microsoft has unveiled a new service called Correction that automatically revises factually incorrect AI-generated texts by leveraging both small and large language models. Designed to enhance content accuracy in AI applications, especially in fields like medicine, it faces scrutiny regarding its efficacy and potential to introduce new issues related to trust and explainability in AI systems.
Detailed Description:
– **Service Overview**: Microsoft’s Correction is a feature aimed at improving the accuracy of AI-generated text. It utilizes a dual-model approach:
– **Classifier Model**: Detects potentially incorrect, fabricated, or irrelevant text segments (commonly referred to as hallucinations).
– **Language Model**: Works alongside the classifier to rewrite detected hallucinations based on specified grounding documents.
– **Integration with Azure AI**: The Correction service is part of Microsoft’s Azure AI Content Safety API, currently in preview and applicable across various text-generating AI models, including Meta’s Llama and OpenAI’s GPT-4o.
– **Intended Use Cases**:
– The service aims to benefit application developers in high-stakes fields, such as medicine, where the accuracy of AI-generated responses is paramount.
– **Expert Opinions**:
– Some experts have raised concerns regarding the tool:
– **Os Keyes** points out that while Correction might reduce some existing issues, it could also create new challenges, indicating that its hallucination detection capabilities might themselves suffer from hallucinations.
– **Mike Cook** warns that increasing perceived safety from 90% to 99% doesn’t tackle the underlying problems inherent in AI reliance, emphasizing that errors will persist in the difficult-to-detect 1%.
– **Significance for Security and Compliance**:
– Professionals in AI security and compliance should be aware of the implications of such tools not only for accuracy but also for the broader trust and ethical considerations they introduce.
– The development underscores the importance of rigorous testing and validation in AI systems to minimize risks associated with misinformation and enhance overall reliability.
This newly introduced service by Microsoft reflects ongoing efforts to improve AI-generated content’s accuracy and reliability while simultaneously highlighting the need for critical analysis of such advancements within the security and compliance landscape.