The interview partners
Prof. Dr. Patrick Da-Cruz is a professor of business administration and health management in the School of Health Management at Neu-Ulm University of Applied Sciences (HNU) and the academic director of the MBA program in Leadership and Management in Healthcare.
Prior to joining HNU, Dr. Da-Cruz worked at leading strategy consulting firms in the pharmaceutical and healthcare sectors and held executive positions at companies in the healthcare industry both in Germany and abroad.
Daniel Kleiboldt is a legal engineer who helps medical practices, medical care centers, and hospitals implement AI systems in a regulatory-compliant manner. He studied law at the University of Münster and spent over ten years working in labor law and compliance for a global corporation. Because he wanted to understand what he was regulating, he subsequently trained as a software engineer. Today, he bridges the gap between law and technology—not as a commentator, but as a practitioner.
LinkedIn (opens in a new window) Website (opens in a new window)
Where does the greatest illusion of liability lie today when it comes to the use of AI in medicine? Should doctors even trust AI decisions when they are ultimately the ones responsible?
Daniel Kleiboldt: A radiologist has an AI generate the report, reads it over, and signs it. Three months later, it turns out that the AI missed a tumor. His first reaction: “The system failed.” His lawyer’s reaction: “You failed.”
That is the biggest misconception about liability. Many assume that since AI makes the decisions, the responsibility lies with the manufacturer. This is legally incorrect and practically dangerous. AI is a tool, and the responsibility for treatment remains with the physician. Should one trust AI recommendations? Yes, but critically. The second-opinion model is a helpful analogy, but it has legal limitations. A fellow physician bears their own professional liability and can explain their error. AI can’t do that. This makes clinicians’ own assessments and documentation all the more essential. There is also another aspect that is often overlooked in everyday practice. Patients have a legitimate interest in knowing whether AI plays a role in their diagnosis. Anyone who ignores this has not fully considered their duty to inform patients.
Who is actually liable when AI makes a mistake—the doctor, the manufacturer, or no one?
Daniel Kleiboldt: In most cases, the doctor. This is the logical consequence of the responsibility for treatment. Two scenarios must be distinguished. If the AI produces incorrect results due to a system error, the manufacturer’s product liability may apply. The new EU Product Liability Directive, which explicitly covers software and AI systems and must be transposed into German law by December 2026, has been in effect since December 2024. Until then, the current Product Liability Act (ProdHaftG) will continue to apply, but the direction is clear, and manufacturer liability will expand significantly as a result. If, on the other hand, a physician uncritically accepts a plausible diagnosis or overlooks a questionable finding because it is embedded in the workflow, this constitutes medical malpractice. In the same incident, two lines of liability may run concurrently: medical liability and product liability. How responsibility is allocated in such chains of shared fault has yet to be fully determined by the courts. This, in itself, is a risk.
What does the 2024 ruling by the Regional Court of Kiel specifically mean for private practitioners and medical center operators?
Daniel Kleiboldt: First, an important clarification. The ruling by the Regional Court of Kiel (Case No. 6 O 151/23) is not a medical malpractice case. It concerns a business information portal whose AI software automatically analyzed mandatory disclosures from the commercial register and, in doing so, confused two companies with similar names, resulting in the wrong company being ordered to be dissolved due to insolvency. The portal operator argued that he had not been involved in the fully automated process. The court did not accept this. Anyone who deliberately uses AI to generate and publish results is directly liable. This is true regardless of whether they were aware of specific errors. Applied to the medical context, this means that the algorithm makes a mistake, but the report bears your name. The principle applies directly: The doctor believes they are using a tool. The law holds them responsible for its output. “I was just following the system” is not a valid defense. Furthermore, there is a serious debate in legal scholarship about whether the failure to use AI can itself become a liability issue—specifically, when an AI system becomes the accepted standard in a particular area of care. This is not yet established law, but it is a development that healthcare executives should keep on their radar.
What do medical practices overlook when selecting AI providers?
Daniel Kleiboldt: The Terms of Service. It sounds trivial, but it’s the most common blind spot. Many AI providers exclude any liability for errors in content in their terms of service. If the system suggests an incorrect diagnosis and you accept it, you’re on your own. The manufacturer has contractually absolved itself of liability before you even clicked “Generate Findings” for the first time. What I’m seeing here is a systematic pattern. From a marketing perspective, AI is marketed as a way to lighten the load, as an innovation, and as a means of improving efficiency. The language is polished, and the promises are concrete. In the contract, the wording is just as precise—only in the opposite direction. “The final decision rests with the user.” Disclaimers that clearly assign operator status to the physician. Both are understandable in their own right. But there is a gap between the promise and the legal reality that no provider is closing. No DSFA template, no training materials for the practice team, no checklist for implementing human oversight. What I recommend to practices is to take a close look before integration. Before you integrate a system into patient care, read the liability clause and the data protection agreement, and check which functions the system actually performs. The key question is whether AI merely records what the doctor says, or whether it begins to make diagnostic or therapeutic recommendations on its own. Because as soon as it does, the regulatory classification changes fundamentally. The system becomes a medical device, and the operator is subject to the expanded obligations under Section 26 of the AI Act. Trust the technology. But read the contract.
How does the workflow in medical practices need to change to ensure that liability risks remain manageable? What role does continuing education and training play in this?
Daniel Kleiboldt: Three changes that can be implemented today: First, explicitly document the use of AI. Document not only the result, but also the process—that is, which system was used, what the output was, and what conclusions were drawn from it. Second, establish a critical review process in which AI output is treated as input for decision-making, not as the final output. Third, train the team on the specific decision-making points where the human factor is critical, not on the tool itself. The AI Regulation distinguishes between two levels of training. Article 4 requires operators of AI tools to ensure that their staff have sufficient AI competence as early as February 2025. This is not a traditional training requirement subject to fines, but rather an organizational obligation—one that must be taken seriously. Article 26(2) goes even further for high-risk AI systems, requiring that human oversight be entrusted to individuals who can demonstrate that they possess the necessary expertise. The deadlines for the full implementation of operator obligations regarding high-risk AI are currently being revised under the Digital Omnibus. The political agreement points to the end of 2027 for autonomous AI systems and August 2028 for AI embedded in medical devices. Art. 4 gilt davon unabhängig bereits jetzt. If you wait to train your team until the deadlines are set in stone, you’ll end up with too little lead time.
What is the most common avoidable mistake you see in your consulting practice?
Daniel Kleiboldt: Gaps in documentation. Not due to negligence, but to a misunderstanding of what constitutes sufficient documentation today when AI is involved. Doctors document the outcome. Courts assess the process. That is the disconnect. Under Section 630h(3) of the German Civil Code (BGB), it is presumed that a medically necessary procedure that is not documented simply did not take place. This turns the burden of proof on its head. The statement, “AI-assisted analysis of findings obtained, recommendation critically reviewed, own clinical assessment concurs,” fundamentally changes this situation. The difference comes down to a single sentence. It takes ten seconds. What it prevents in the event of harm cannot be measured in seconds.
Thank you very much for the interview!
The content and statements presented in the interviews reflect the perspectives of the interviewees and do not necessarily reflect the editorial staff’s position.
