Laying the Groundwork for Evidence-Based AI in Healthcare and Robust Health Technology Assessment

As digital health technologies evolve, artificial intelligence (AI) is playing an increasingly important role in clinical care. While many AI-based medical technologies have received regulatory approval, their real-world clinical effectiveness and cost-effectiveness remain insufficiently validated to support reimbursement decisions under Taiwan’s National Health Insurance (NHI) program.

To close this evidence gap, the Ministry of Health and Welfare established the Center for Clinical AI Impact Evaluation under the Department of Information Management. The Center leads multi-center randomized controlled trials (RCTs) to assess the clinical utility and health technology value of AI tools and to inform potential reimbursement strategies. It convenes experts in clinical trials, biostatistics, epidemiology, health informatics, and health economics to ensure rigorous evaluation and actionable evidence that can support health policy and implementation.

Establishment Objectives

As smart healthcare continues to evolve, artificial intelligence (AI) is playing an increasingly prominent role in clinical diagnostics. Unlike conventional medical devices, AI tools do not rely on physical consumables; rather, they generate diagnostic results through computational algorithms alone. This fundamental distinction poses unique challenges for reimbursement decision-making by national health insurance systems, public health agencies, and private insurers, as existing payment models are often not well-suited to algorithm-based technologies.

To accurately assess the clinical and economic value of AI technologies, rigorous health technology assessment (HTA) grounded in scientific evidence is essential. High-quality evidence must be generated through well-designed clinical trials that directly compare AI-assisted diagnostics with standard-of-care approaches. Such studies require large sample sizes, multi-institutional collaboration, and interdisciplinary expertise to ensure methodological rigor, reproducibility, and generalizability.

However, most AI developers are small- to medium-sized enterprises (SMEs) with limited financial and operational resources, making it challenging for them to conduct large-scale, multi-center randomized controlled trials (RCTs). As a result, many AI tools have yet to undergo the level of rigorous clinical validation necessary for broad clinical adoption and global regulatory acceptance.

To strengthen Taiwan’s leadership in smart healthcare, the Ministry of Health and Welfare (MOHW) established the Center for Clinical AI Impact Evaluation, led by Dr. Chien-Chang Lee, MD, ScD, Director of the Department of Information Management. Leveraging globally recognized methodologies for clinical trials, the Center brings together experts in clinical research, biostatistics, epidemiology, health informatics, and artificial intelligence to conduct scientifically rigorous, multi-center evaluations. This initiative aims to generate high-quality evidence on the real-world effectiveness and impact of AI tools, supporting informed decision-making for clinical integration and reimbursement.

Figure 1. Key Challenges in the Clinical Use of AI-Based Medical Technologies After Regulatory Approval

Core Strategies

Establishing a Multi-Center Consortium of Academic Medical Centers

Conducting randomized clinical trials for AI-based medical devices requires the integration of specialized applications within hospital information systems and the secure processing of large volumes of patient encounter data. Entrusting these critical systems and sensitive data to commercial contract research organizations (CROs) may introduce significant cybersecurity and privacy risks. Therefore, the optimal model for evaluating trials of AI medical technologies involves a consortium of academic medical centers that can oversee the study design, implementation, and analysis while maintaining control over information infrastructure and patient data security.

A rigorous impact evaluation demands multidisciplinary expertise. The first step is to establish a formal consortium among participating institutions. A centralized Institutional Review Board (IRB) process should then be implemented to ensure consistent ethical oversight across all sites. A methodological advisory group of epidemiologists, biostatisticians, and data scientists should guide trial design, randomization, and study protocols. Finally, study outcomes should be analyzed by biostatisticians or epidemiologists, with the results compiled into formal reports to inform reimbursement decisions under the NHI system. Where applicable, health economists should conduct economic evaluations to support evidence and value-based policy development.

Figure 2. Three Strategic Pillars of the Center for Clinical AI Impact Evaluation

Randomized Clinical Trial Design

The design of RCTs for AI-enabled technologies differs significantly from that of traditional drug trials. First, AI trials often involve randomization at the system or institutional level, such as entire hospitals or clinics, rather than at the individual patient level. Second, because AI tools are typically used for diagnostic support, they may not directly influence clinical outcomes unless combined with a therapeutic or interventional component.

To address these methodological challenges, the Department of Information Management at the Ministry of Health and Welfare (MOHW) has supported the establishment of the Center for Clinical AI Impact Evaluation. The Center serves as a national consulting body for clinical trial design and offers technical assistance in trial planning. Working in collaboration with clinical investigators, the Center develops integrated protocols that align diagnostic tools with interventional workflows to form coherent and actionable trial designs.

Three trial designs are particularly well-suited for evaluating AI-enabled interventions:

• Controlled Before-After Trial (CBAT): This design compares outcomes before and after implementation of the AI tool, with a concurrent control group receiving standard care. One group is exposed to the intervention, while a control group undergoes only routine diagnostic procedures.
   
• Cluster Randomized Controlled Trial (CRCT): In this model, entire clusters (e.g., hospitals or clinical sites) are randomized to different study arms. This approach reduces contamination between groups and simulates real-world implementation environments.
   
• Step-Wedge Pragmatic Controlled Trial: In this sequential design, clusters are gradually transitioned from control to intervention status over time. This model allows all participating sites to eventually receive the intervention while enabling temporal comparisons of impact.

Figure 3. Three AI-Relevant Randomized Trial Designs Funded by the Ministry of Health and Welfare

Future Outlook

The Office of Medical Informatics of the Ministry of Health and Welfare is steadfastly advancing the highest standards of research design, boldly improving efforts to improve AI technologies through randomized clinical trials held to the same rigor as pharmaceutical studies. In 2024, five leading medical centers were selected to participate in a six-month pilot phase under this initiative. The human and material resources committed by each center far exceeded the Ministry’s funding, demonstrating Taiwan’s medical community’s unwavering commitment for international competitiveness and excellence. These trials are slated for completion by the end of 2025, at which point Taiwan will present a substantial body of evidence to the global community, further solidifying its reputation as a “Smart Healthcare Island.”

Moreover, to ensure the long-term sustainability of the trial centers beyond the lifespan of government grants, Director Paul Lee has introduced an innovative economic model. Hospitals will offer customized services to device manufacturers and research groups on a fee-for-service basis, creating a self-reinforcing data economy. This mechanism will institutionalize the trial centers as core components of Taiwan’s premier healthcare infrastructure, safeguarding their continued operation regardless of fluctuations in public funding. By establishing this forward-looking framework, Taiwan is laying a robust foundation for the future of AI-driven clinical research, reinforcing its leadership in the global evolution of smart healthcare.

Figure 4. Five Demonstration Hospitals for Successful Deployment of the AI Impact Evaluation Center Across Taiwan

Project Team and Advisors

Project Initiator

Chien-Chang Lee, M.D., Sc.D.

Chief Information Officer (CIO) for the Taiwan Ministry of Health and Welfare (MOHW) 
Professor of Emergency Medicine 
Deputy Director of the Center of Intelligent Healthcare at National Taiwan University (NTU) Hospital

Acknowledgements

We sincerely acknowledge Meng-Ru Shen, MD, PhD, Convener of the Center for Clinical AI Impact Evaluation and President of National Cheng Kung University (Tainan, Taiwan); Lee-Jen Wei, PhD, Professor of Biostatistics at Harvard University (Boston, MA, USA); Kenneth Mandl, MD, MPH, Professor of Pediatrics and Biomedical Informatics at Harvard University and Director of the Computational Health Informatics Program at Boston Children’s Hospital (Boston, MA, USA); and Yu Shyr, PhD, Professor of Biostatistics, Biomedical Informatics, and Health Policy at Vanderbilt University and Director of the Vanderbilt Center for Quantitative Sciences (Nashville, TN, USA), as distinguished external expert reviewers for their invaluable professional guidance and contributions to our initiative in evaluating clinical AI impact.

Meng-Ru Shen, MD, PhD	Lee-Jen Wei, PhD	Kenneth Mandl, MD, MPH	Yu Shyr, PhD
Convener of the Center for Clinical AI Impact Evaluation President of National Cheng Kung University	Professor of Biostatistics at Harvard University	Donald A.B. Lindberg Professor of Pediatrics and Biomedical Informatics at Harvard University Director of the Computational Health Informatics Program at Boston Children’s Hospital	Harold L. Moses Chair in Cancer Research Chair, Department of Biostatistics Professor of Biostatistics, Biomedical Informatics, and Health Policy Director, Center for Quantitative Sciences Vanderbilt University Medical Center

International Partner Matching Platform

We support international organizations and innovators in identifying and connecting with the most appropriate partners in Taiwan, including the officially designated Centers for Clinical AI Impact Evaluation. To learn more, please click here.