The interview partners
Prof. Dr. Patrick Da-Cruz is Professor of Business Administration and Healthcare Management at the Faculty of Healthcare Management at Neu-Ulm University of Applied Sciences (HNU) and Academic Director of the MBA program Leadership and Management in Healthcare.
Before joining the HNU, Mr. Da-Cruz worked for renowned strategy consultancies in the pharmaceutical/healthcare sector and in management positions in companies in the healthcare industry in Germany and abroad.
Rudolf Wagner has more than 15 years of experience in all areas of quality and compliance management, team leadership, P&L responsibility and relationship building in the fast-paced environment of the pharmaceutical and medical device industry. He has been Managing Director of ADHOCON UG since 2022.
What is meant by patient safety?
Rudolf Wagner: Patient safety refers to many individual elements in connection with a medical diagnosis and subsequent treatment. In the context of digital health and AI in medicine, it is about protecting patients from avoidable harm during medical care. These are all measures to reduce diagnostic and treatment errors that can occur during treatment. The authorities in almost all countries worldwide have enshrined regulatory requirements such as the EU Medical Device Regulation (EU MDR) and the In Vitro Diagnostics Regulation (EU IVDR) in law, thereby obliging manufacturers of medical devices and healthcare facilities to carry out comprehensive risk analyses and integrate safety measures. These regulations ensure that products come onto the market that are safe and proven to be effective and have been tested by an external body (notified body such as TÜV or DEKRA) and given a CE mark. Even after market launch, these products must be continuously monitored to identify and prevent potential risks. A central element of patient safety is the post-market surveillance system, which ensures continuous monitoring and evaluation of medical devices in clinical use. International standards such as ISO14971 supplement this regulation by providing specific guidelines for risk management. Patient safety is thus ensured through a combination of strict regulatory requirements, their review and approval, but also through technological innovation and best clinical practice.
What role does the topic of patient safety currently play in the German healthcare system?
Rudolf Wagner: Patient safety has a very high priority in Germany, particularly due to the increasing complexity of the healthcare system, the requirements of health insurance companies, a lack of financial resources, a lack of personnel and the advancing digitalization. Regulatory requirements such as the EU MDR and national laws such as the Patients' Rights Act aim to ensure patient safety. The MDR, which has been fully applicable since May 2021, requires manufacturers of medical devices, including software as a medical device (SaMD), which includes AI and is explicitly included, to provide rigorous safety and clinical evidence of positive efficacy. These regulations are designed to minimize risks for patients by placing high demands on the development, approval and monitoring of products. In addition, hospitals are also regulated by law so that they may only use approved and tested medicines and medical devices such as software and AI. In addition, initiatives such as the “Patient Safety Action Plan” and the National Contact Point for Patient Safety promote the prevention of medical errors. This initiative implements standardized safety protocols and achieves continuous improvements in patient safety. The integration of new technologies and the assurance for every patient that they meet the high safety requirements are challenges that must be closely monitored by healthcare facilities, regulatory authorities and society.
What traditional measures can be considered to increase patient safety?
Rudolf Wagner: Traditional measures to increase patient safety include standardized protocols, training of medical staff and quality assurance processes at developers and healthcare facilities. These measures aim to minimize treatment errors and ensure a high level of safety and quality of diagnosis and treatment.
An important approach here is risk management, which requires the systematic identification, assessment and mitigation of risks for products such as hardware, medical devices, software, AI and medicines. In the EU, the MDR requires manufacturers to implement a risk management system that must be regularly reviewed and adapted to ensure that products remain safe and effective.
In clinical practice, measures such as the introduction of checklists, which are used particularly in surgery and intensive care medicine, and the establishment of error reporting and learning systems help to improve patient safety.
However, the most important thing is that all regulatory and legal requirements are monitored and actively enforced by the authorities. This is the only way to identify manufacturers and healthcare facilities that use dangerous, unauthorized products and software and eliminate the risk of harm. Regular audits and training help to raise awareness of safety risks and ensure that medical staff are able to recognize and address potential dangers at an early stage.
Can artificial intelligence (AI) improve patient safety?
Rudolf Wagner: In principle, yes, but only if this AI has been tested and approved in advance for safety and effectiveness in accordance with the applicable regulations and laws.
AI offers great potential to improve patient safety, in particular by analyzing large amounts of data and - if approved by the authorities and notified bodies - by providing accurate decision support for medical staff. AI systems can be used to support complex diagnoses, predict drug interactions and detect potential complications at an early stage, e.g. through so-called digital twins and the results of clinical trials.
Regulatory requirements such as the EU MDR and the FDA's Software as a Medical Device (SaMD) guidelines place strict requirements on the safety and effectiveness of AI applications before they can be used clinically. These regulations require manufacturers to validate and continuously monitor AI systems to ensure their safety and effectiveness. In addition, AI algorithms must be regularly reviewed and adapted to ensure that they are based on current medical knowledge and data and function reliably.
One example of the use of AI to improve patient safety is the use of machine learning algorithms that recognize patterns in patient data that may escape human understanding. This can lead to more precise diagnoses and personalized treatment plans that are tailored to the individual needs of patients.
What risks, if any, does the use of AI pose to patient safety?
Rudolf Wagner: Although AI has the potential to improve patient safety, its use also harbors risks. One of the main dangers is the risk of incorrect decisions caused by insufficiently trained or faulty AI. Such errors could lead to incorrect diagnoses or inappropriate treatment decisions, which could jeopardize patient safety. For example, some approved AI systems have been trained on the basis of only 1500 patient data. This is likely to result in severe limitations, meaning that incorrect conclusions may be drawn in marginal areas, i.e. for patients with rare secondary diseases.
Another risk is the dependence of medical staff on AI systems, which could mean that important clinical judgments are less critically scrutinized. Where today a doctor might consult a colleague, a misplaced trust in AI and its quick and confident response could prevent doubt from arising in the first place.
Regulatory requirements such as the EU MDR and FDA guidelines aim to minimize such risks by imposing strict requirements for the validation and continuous monitoring of AI applications. However, over 200,000 software apps and thousands of AI systems are already in use today without approval or review by the authorities. Even ChatGPT, which has no approval for medical or clinical use, is increasingly being used secretly by doctors.
In addition, data protection and security risks pose a challenge, especially as AI systems process large amounts of sensitive patient data. The MDR requires comprehensive measures to protect this data, including clear responsibilities and transparent handling of data processing procedures. Finally, there is a risk that AI systems that are not regularly updated and monitored will lose accuracy and relevance over time. This could lead to a deterioration in the quality of medical care.
Thank you very much for the interview!
The content and statements presented in the interviews reflect the perspective of the interviewees and do not necessarily reflect the position of the editorial team.
