A Structured Comparison of the Coalition for Health AI Responsible AI Guide and South Korea's Trustworthy AI Guideline for Health Care AI Assurance: Comparative Framework Analysis.

Background: Trustworthy artificial intelligence (AI) in health care requires assurance frameworks that translate ethical principles into measurable governance and evaluation practices. While a growing number of AI assurance frameworks have been proposed, they differ substantially in governance structure, institutional embedding, and implementation mechanisms, reflecting differences in intended purpose and use. To date, few studies have applied standardized, rubric-based evaluation criteria to systematically compare how assurance instruments with different institutional origins operationalize ethical principles across the AI lifecycle. Objective: This study aimed to develop and apply a structured, rubric-based evaluation instrument to compare 2 health care AI assurance instruments, including the Coalition for Health AI (CHAI) responsible AI guide, a voluntary consortium-based instrument, and South Korea's Trustworthy AI guideline, a government-issued instrument. Methods: A 7-dimension evaluation rubric was developed based on a synthesis of established international AI assurance and governance instruments. The rubric covered core principles, AI lifecycle coverage, governance context, stakeholder breadth, operational maturity, instrument design and tools, and public accessibility. Seven independent evaluators with expertise in health care AI governance assessed each instrument using a 5-point ordinal rating scale (1=absent-5=comprehensive). Each evaluator independently scored the materials using a standardized rubric. Discrepancies were resolved through structured consensus discussions, with reference to rubric definitions and source documents. Final scores were determined based on documented evidence, requiring full consensus rather than averaging. Interrater reliability was assessed using Fleiss kappa. Results: Both instruments demonstrated strong alignment in core principles (CHAI: 4; Trustworthy AI Guideline: 5) and stakeholder breadth (both: 4). The government-issued Trustworthy AI Guideline exhibited broader AI lifecycle coverage (5 vs 4), a more formalized governance context (5 vs 3), and higher operational maturity (4 vs 2), reflecting stepwise oversight and formal embedded oversight mechanisms supported by legislation. In contrast, the voluntary CHAI instrument demonstrated greater emphasis on instrument design and implementation tools (4 vs 3) and higher public accessibility (5 vs 3), driven by open-access resources such as assurance standards guides and applied model cards. Interrater agreement of independent ratings was moderate to substantial (Fleiss kappa=0.47-0.64; P<.001), indicating consistent scoring patterns among evaluators. Conclusions: This comparative analysis indicates that voluntary and government-issued AI assurance instruments operationalize trustworthy AI principles in distinct but complementary ways. Voluntary instruments emphasize flexible tools and accessible implementation resources, while government-issued guidelines embed assurance functions within formal governance and oversight structures. Rather than representing competing models, these approaches address different assurance needs across the AI lifecycle. By identifying concrete areas of alignment and divergence, this study supports a more coherent comparison of assurance practices and highlights potential opportunities for alignment across documentation structures and evaluation approaches that can support safe, equitable, and scalable deployment of health care AI across diverse institutional contexts.