Artificial intelligence (AI) is becoming part of healthcare decision-making, from disease detection to treatment recommendations. As these systems enter routine care, a harder policy question follows: when an AI-assisted decision goes wrong, who is accountable? A common assumption is that responsibility rests with the clinician making the final call. But treating ‘human-in-the-loop’ as a sufficient safeguard can confuse clinical judgement with technical accountability, leaving doctors responsible for failures they were never equipped to detect.
In a 2023 randomised study published in the Journal of the American Medical Association, 457 US clinicians assessed patient case vignettes. Systematically biased AI recommendations reduced diagnostic accuracy by 11.3 percentage points from baseline. Image-based explanations did not significantly offset the effect: accuracy remained 9.1 percentage points below baseline. A human decision-maker was present throughout, yet oversight did not reliably correct the system’s errors
This is already a practical question for India. Government figures reported in December 2025 show that AI-assisted diabetic-retinopathy screening had reached 38 facilities across 11 states, benefiting 7,100 patients, while an AI-based tuberculosis screening tool had screened more than 162,000 people.
The policy question therefore extends beyond the final clinical decision: who has the function, information and authority to prevent or detect a problem at each stage of the AI lifecycle?
Accountability Starts Before the Bedside
Developers and technical teams determine training data, testing methods and the boundaries of a model’s validated use. Clinicians may contribute essential expertise without controlling these decisions. Responsibility cannot simply pass downstream to whoever uses the system. Meaningful clinical oversight requires training, information about limitations and authority to challenge recommendations. Where these are absent, the institution’s decision to deploy the system also warrants scrutiny.
After deployment, institutions should oversee clinical integration, staff training, local performance and incident escalation. Developers should investigate technical performance, communicate limitations and updates, and identify changes requiring revalidation.
A 2025 study published in JMIR Medical Informatics, evaluating three AI systems for diabetic-retinopathy screening in public-health settings in Punjab, reported substantial variation in diagnostic performance, with sensitivity ranging from 60 to 80 percent and specificity from 14 to 96 percent. One system was withdrawn from the evaluation after interim analysis identified low specificity, which can generate false positives and unnecessary referrals. The withdrawal shows how testing can detect problems before wider use. Decisions about acceptable performance and deployment are themselves consequential governance decisions.
Where Liability Meets the AI Lifecycle
India’s emerging governance architecture addresses several parts of this problem. The Digital Personal Data Protection Act and Rules provide a framework for processing digital personal data, with phased commencement. The Central Drugs Standard Control Organisation regulates software qualifying as a medical device, while the Indian Council of Medical Research identifies accountability as an ethical principle. The Strategy for Artificial Intelligence in Healthcare for India (SAHI) provides sectoral direction, and the Benchmarking Open Data Platform for Health AI (BODH) supports model evaluation.
The India AI Governance Guidelines, issued in November 2025, propose graded liability based on function, risk and due diligence. Translating that principle into healthcare requires explicit duties across development, validation, deployment and monitoring. Accountability establishes who must explain, investigate and correct failures; legal liability additionally depends on applicable duties, evidence and causation.
A February 2026 Reuters investigation into the TruDi surgical navigation system illustrates the difficulty of investigating harm across this chain. It identified at least 100 unconfirmed reports of malfunctions and adverse events submitted to the United States Food and Drug Administration after an AI upgrade, including reports involving at least 10 injured patients. Most alleged that the system misinformed surgeons about the location of their instruments. Integra LifeSciences said there is no credible evidence of a causal connection between the system, AI and the alleged injuries; the reports do not establish that AI caused the harm. Investigating such cases requires evidence about design, validation, software changes and clinical use.
Making Accountability Traceable
When harm occurs, records should establish which system and model version were used, what it recommended and how the clinical decision followed. Investigators also need access to validation evidence, known limitations and material updates. Institutions should assign responsibility for incident review and authority to suspend use, with clear arrangements for obtaining evidence from developers.
Traceability cannot settle every question of liability, but it can prevent accountability from defaulting to whoever is easiest to identify. Clinicians remain responsible for their professional judgements; technical failures they could not reasonably detect require scrutiny elsewhere in the chain.
Oversight must also be feasible. Smaller institutions may lack expertise for specialised testing and monitoring, while the consequences of failure vary substantially: a scheduling tool should not face the same scrutiny as a system recommending cancer treatment. BODH provides shared infrastructure for testing and benchmarking models against diverse, real-world health data while protecting patient datasets. This could reduce institutional testing burdens, but local monitoring remains necessary to assess performance within particular workflows and patient populations.
The Policy Test
As healthcare AI scales, the policy test is whether duties are documented, failures can be investigated and corrective action has a clear owner. Human oversight should be supported in practice, and responsibility should follow the decisions each actor controls. Systems entering clinical care should carry clear lines of responsibility with them.