Blogs

Every health system wants to adopt AI responsibly. Few have figured out how to evaluate it efficiently.

Clinical AI is moving into healthcare faster than governance processes were designed to accommodate. AI review committees are balancing growing queues of technologies, limited internal resources, and pressure from clinicians who are eager to deploy tools that can meaningfully improve patient care. At many organizations, the question is no longer whether to adopt AI, it's how to evaluate it thoroughly without slowing innovation to a crawl.

When CHAI recently brought together 15 health systems to examine how organizations evaluate AI during procurement, one finding stood out. The longest part of the review process wasn't legal review or technical assessment, it was simply gathering the information needed to begin. System leaders reported spending anywhere from two weeks to two months evaluating a single AI solution, with missing documentation and repeated vendor follow-up accounting for much of that time.

The exercise revealed something equally important. Despite differences in organizational size and governance maturity, health systems were asking remarkably similar questions:

• How was the model validated?

• What evidence supports its performance?

• How is it monitored after deployment?

• What are its limitations?

• How will it fit into existing clinical workflows?

• What happens if performance changes over time?

The challenge wasn't that organizations lacked good governance processes. It was that they were asking many of the same questions in different ways, while vendors were often responding through a patchwork of security questionnaires, trust centers, product documentation and follow-up conversations. The information existed, but it wasn't easy to find, compare, or reuse.

That observation has important implications for both healthcare organizations and AI developers. If the industry is converging around a common set of evaluation questions, then transparency isn't simply about disclosure. It's about presenting information in a way that makes those conversations more productive.

One company already putting that idea into practice is clinical AI platform developer Nabla.

During a recent CHAI webinar on AI transparency, Brittney Harrell, Head of Information Security at Nabla, described what it was like to publish the company's model card through the CHAI Registry. Perhaps the most telling part of her experience was that very little of the information had to be created from scratch.

"A lot of that information already existed," she explained. "It just wasn't formal."

That distinction matters.

Many AI developers have already invested heavily in validation studies, security reviews, governance processes and clinical evidence. The challenge is that those materials often live in separate places and are shared only after procurement conversations are well underway. By organizing key information into a standardized format, Nabla found it could answer foundational questions earlier in the process and spend more time discussing implementation, clinical priorities and organizational needs.

Brittney also emphasized that transparency should never be confused with giving away intellectual property. Rather, it means providing enough evidence for healthcare organizations to understand how a solution was developed, where it performs well, what its limitations are and how the company approaches ongoing monitoring and governance. Those conversations become richer when they begin from a shared understanding rather than a blank page.

The CHAI model card consistently served as a useful starting point for evaluation, but it was never intended to answer every question. Instead, it established a common foundation that allowed governance teams to focus on the issues that truly required discussion. Participants found that while model cards covered much of the descriptive information organizations wanted, additional operational topics – including continuous monitoring, incident response, vendor governance, and data stewardship – were equally important to understanding how an AI solution would perform over the course of a long-term partnership.

Shaping the next phase of CHAI's Model Card

The CHAI Model Card Registry was created as a public repository where AI developers can publish standardized model cards and supporting resources, allowing health systems, clinicians, and governance teams to discover and compare information using a consistent framework. Much like a nutrition label helps consumers quickly understand what's inside a product, the Registry provides a structured snapshot of an AI solution's intended use, validation approach, governance practices, and supporting evidence without attempting to replace a comprehensive vendor review.

Importantly, the Registry is not intended to certify products or determine which solution a health system should choose. Every organization has unique clinical priorities, workflows, and risk tolerances that require deeper evaluation. Instead, the Registry is designed to make those conversations more informed from the outset by ensuring that everyone begins with a common set of facts.

As AI governance continues to evolve, so too will expectations around transparency. During the webinar, Brittney predicted that future evaluations will place even greater emphasis on continuous monitoring, interoperability, external validation and ongoing performance measurement as AI systems become more sophisticated and interconnected. Those expectations are unlikely to diminish. They will become part of the baseline for responsible adoption.

The broader lesson from CHAI's work is that responsible AI adoption depends on more than strong governance frameworks or better technology. It depends on reducing unnecessary friction between the organizations building AI and the organizations evaluating it. When health systems can spend less time searching for information and more time discussing evidence, implementation and patient impact, everyone benefits. Transparency becomes more than a reporting exercise. It becomes an enabler of responsible innovation.