For the past three years, healthcare headlines have been dominated by the promise of artificial intelligence. AI will transform radiology. AI will detect cancer earlier. AI will solve physician burnout. AI will democratize diagnostics, personalise treatment and reduce healthcare costs simultaneously.
However, as time has passed, trust issues have begun to surface.
This is because, while hospitals and health systems have welcomed AI adoption as inevitable progress, many healthcare professionals are secretly concerned about overreliance on AI and issues related to legal exposure and the gradual erosion of clinical judgment.
Clinicians Wary Of ‘Magic Box’ AI Functions
It’s true that many perceive AI as something of a ‘magic box’. You put in the question and out comes the answer. What happens in between, no-one knows!
Many clinicians, while not being anti-AI, are cautious of a system which gives a result without fully explaining its reasoning process.For example, if a system flags a lesion but cannot explain why in clinically meaningful terms, trust is left fragile.
Clinicians are trained not only to diagnose, but to justify reasoning. Black-box, or ‘magic box’, AI disrupts that tradition.
This growing discomfort is especially visible in radiology. Here, AI tools are now widely promoted for mammography, chest imaging and cancer screening. While some studies show improved detection rates when AI supports clinicians, controversy continues around false positives, hidden bias and workflow dependency.
Clinicians are fearing that they may become overly dependent on systems which they do not fully understand.
A Weakening Of Diagnostic Intuition?
Then there’s the issue of ‘using what your mother gave you’… in the case of AI, your brain and your God-given ability to reason and think.
AI promises relief by reducing workload and accelerating interpretation. However, critics worry that excessive reliance could eventually weaken diagnostic intuition itself. The criticism here is that AI might be teaching us to not think or reason or use intuition. Is this really a good thing?
“We risk creating a generation of physicians trained to supervise algorithms more than think independently,” says Professor Elaine Murray, a healthcare ethics researcher specialising in medical automation.
Early Indications Of Automation Fatigue?
At the same time, after years of chatbot hype, algorithmic recommendation systems and growing awareness of AI calculations and diagnoses, patients are beginning to develop what psychologists and technologists have called ‘automation fatigue.’
The phenomenon extends beyond healthcare. Across industries, consumers are becoming overwhelmed by constant exposure to automated systems which promise convenience but often feel impersonal, opaque or unreliable.
Says the Philips Future Health Index 2025: “Patients need reassurance that AI will augment – not displace – the human touch that defines personalised care.”
A 2025 WalkMe workplace study found that 71% of employees believed AI tools were evolving too quickly for their liking, while nearly half reported feeling anxious rather than excited about AI integration.
These are high figures which suggest a general sense of mistrust concerning AI.
In medicine, this ‘automation fatigue’ carries deeper emotional consequences. This is because healthcare decisions involve vulnerability, and must be based on trust.
Patients increasingly want reassurance that a human physician remains involved in their care, even when AI systems are technically accurate. Regardless of whether or not AI works, many people battle to psychologically accept machine-mediated healthcare decisions.
The Philips Future Health Index, which surveyed more than 16 000 patients and 1 900 healthcare professionals across 16 countries, found that while 96% of clinicians expressed confidence in AI’s diagnostic support capabilities, patients remained significantly more cautious. More than half of patients said they feared AI could eventually replace their doctor.
Another US survey found that while 63% of healthcare providers believed AI could improve patient outcomes, only 48% of patients agreed. Among patients over 45, optimism dropped to just 33%.
This skepticism is increasingly shaping patient behaviour. A recent UK study from King’s College London revealed that one in seven people now consult AI chatbots instead of seeing a doctor, largely because of healthcare access delays. Yet the same study found significant concern about safety and accountability, with public opinion on AI in clinical decision-making almost evenly divided.
In this way, patients are caught between two conflicting impulses: frustration with overstretched healthcare systems and discomfort with increasingly automated medicine.
A push back against ‘efficient, emotionally impersonal systems’
Across industries, people are also beginning to push back against systems that feel efficient but emotionally impersonal.
Healthcare amplifies this sensitivity as a diagnosis is not merely a technical fact. It is intrinsically tied to emotions and psychological well-being. Patients want reassurance, interpretation and empathy alongside clinical accuracy.
For decades, lab reports, scans and physician interpretation represented certainty, expertise and institutional trust. Experts are saying that AI disrupts this relationship because it changes who, or what, patients believe is making decisions about their bodies.
Increasingly, patients are asking questions which barely existed a decade ago. Was my scan reviewed by a human? Did the doctor agree with the algorithm? Is this diagnosis statistically generated? Can I appeal an AI assessment?
Considering Future Complications
In summary, the question on everyone’s lips is: Who is liable when diagnostic AI gets something wrong?
If an algorithm misses early cancer signs on a mammogram, responsibility becomes murky. Is the physician liable for trusting the software? Is the hospital liable for implementing it? Is the developer liable for the algorithm itself? These questions remain only partially resolved.
Legal experts increasingly warn that hospitals adopting AI diagnostics may underestimate future litigation exposure. Ironically, even highly accurate systems may create liability problems if clinicians cannot adequately explain how decisions were made.
The issue becomes even more sensitive when healthcare AI bias enters the discussion. AI systems are only as reliable as the datasets used to train them.
Experts argue that the next phase of AI healthcare may depend less on technological capability and more on social legitimacy. The winners in diagnostics AI won’t simply be the companies with the most powerful algorithms. They will be the ones capable of making automation feel interpretable, accountable, human-centered and trustworthy. Because in the end, it’s all about trust.