Tech giant Google is prioritising speed over safety in its rollout of AI-generated medical summaries, putting users at risk by obscuring crucial warnings that the health advice it serves may be dangerously inaccurate, an investigation has found.
The company’s AI Overviews, which appear at the very top of search results, are designed to provide instant answers. However, experts warn this convenience comes at a cost when dealing with sensitive health queries, as the feature often buries vital disclaimers. According to findings presented to The Guardian, no clear safety warning appears when a user is first presented with AI-generated medical advice.
To see any disclaimer at all, a user must click a “Show more” button for additional information, and even then the warning is placed at the very end of the expanded text in a smaller, lighter font. It states: “This is for informational purposes only. For medical advice or a diagnosis, consult a professional. AI responses may include mistakes.”
A Design for Danger
This approach has drawn sharp criticism from AI ethicists and medical professionals. Gina Neff, a professor of responsible AI at Queen Mary University of London, said the “problem with bad AI Overviews is by design,” arguing they are engineered for speed, not accuracy, leading to mistakes that can be dangerous in a health context.
The concerns are not theoretical. Specific examples of inaccurate and potentially harmful advice generated by the system have been documented. In one instance, Google wrongly advised people with pancreatic cancer to avoid high-fat foods, advice which charities state is the opposite of recommended nutritional guidance and could increase mortality risk.
In another, queries about liver function tests returned misleading universal numerical ranges without accounting for critical variables like age, sex, or ethnicity, potentially leading users to incorrectly believe abnormal results were normal. Searches related to women’s cancer tests have also yielded “completely wrong” information that could cause individuals to dismiss genuine symptoms, while summaries for mental health conditions like psychosis and eating disorders have been known to offer incorrect and “very dangerous advice” that could deter people from seeking help.
Pat Pataranutaporn, an assistant professor and AI researcher at the Massachusetts Institute of Technology (MIT), said the absence of upfront disclaimers creates critical dangers. “First, even the most advanced AI models today still hallucinate misinformation or exhibit sycophantic behaviour, prioritising user satisfaction over accuracy. In healthcare contexts, this can be genuinely dangerous,” he told The Guardian.
Sonali Sharma, a researcher at Stanford University’s centre for AI in medicine and imaging (AIMI), highlighted the psychological impact of the feature’s prominent placement. “For many people, because that single summary is there immediately, it basically creates a sense of reassurance that discourages further searching,” she said. The overviews can mix partially correct and incorrect information, making it extremely difficult for a layperson to discern accuracy.
Broader Context of Waning Warnings
The issue forms part of a wider decline in safety safeguards across AI platforms. Research by Sonali Sharma and colleagues found a significant drop in medical disclaimers across various generative AI models between 2022 and 2025, with fewer than 1% of outputs including a warning by 2025. Further compounding concerns, one study revealed Google AI Overviews are increasingly sourcing health information from YouTube videos rather than established medical websites, raising questions about the authority of its sources.
Patient advocacy groups have voiced alarm. Tom Bishop, head of patient information at the blood cancer charity Anthony Nolan, called for the disclaimer to be “right at the top” in the same size font as the advice itself. “We know misinformation is a real problem, but when it comes to health misinformation, it’s potentially really dangerous,” he said. Charities including Pancreatic Cancer UK, the British Liver Trust, and Mind have warned that inaccurate summaries could delay vital doctor visits and escalate patient harm.
Google’s Response and Partial Rollback
In response to the criticism, Google has removed AI Overviews for some, but not all, medical searches. Specific queries like “what is the normal range for liver blood tests” no longer generate the AI summary feature. The company stated that an internal review by clinicians found most AI Overview health content to be accurate and supported by reliable sources, though it does not comment on individual removals.
A Google spokesperson challenged the characterisation of its systems, stating: “It’s inaccurate to suggest that AI Overviews don’t encourage people to seek professional medical advice. In addition to a clear disclaimer, AI Overviews frequently mention seeking medical attention directly within the overview itself, when appropriate.” The company also described some shared examples of harmful advice as “incomplete screenshots” that nonetheless linked to reputable sources.
This scrutiny follows a Guardian investigation in January which first revealed people were being put at risk by false health information in AI Overviews. Despite the company’s adjustments and assurances, experts maintain that the fundamental design—which requires users to click through to find a subdued warning—fails to provide the necessary safeguard for individuals seeking critical, and potentially life-altering, health information online.
