Blogs

Hartford HealthCare brings nearly a decade of experience working with AI across research, academic partnerships, startups, and clinical innovation. Rocco Orlando, Strategic Advisor and Emeritus Chief Academic Officer at Hartford HealthCare, brought the health system’s perspective to the development of CHAI’s AI Governance Playbooks, contributing lessons from an established governance program while learning from organizations at different stages of their own AI journeys.

We spoke with him about why effective governance starts with knowing where AI is being used, how Hartford HealthCare is adapting its approach as the number and complexity of tools grow, and how the Playbooks can help health systems prepare for evolving expectations, including Joint Commission’s Responsible Use of AI in Healthcare certification.

How does Hartford HealthCare think about the governance process? How has that evolved?

We’ve been working with AI for almost 10 years, through research, academic partnerships, startups and developing AI models ourselves. We recognized early that while AI offers tremendous benefits, it also introduces risks.

As a large health system, we were seeing a deluge of AI solutions coming from vendors and companies of all sizes. Our first step was creating an inventory: Where is AI being used across Hartford HealthCare, and do we know where it is?

From there, we established a policy that AI tools couldn’t be introduced without going through our governance process. You can’t effectively govern AI if you don’t know where it is, so that inventory is essential.

How do you create meaningful oversight without turning governance into a barrier to innovation?

People want autonomy, and governance can be perceived as something that slows them down. Given the pace of change in AI, you need an approach that is nimble and responsive. Otherwise, you risk hindering people’s ability to innovate and improve care.

We developed an efficient evaluation process, and the CHAI work helped enhance the tools supporting it. Risk assessment is especially important. With the volume of AI entering healthcare, organizations need ways to distinguish between low-, medium- and high-risk applications so teams can focus their attention where it matters most.

Why is knowing where AI is being used only the beginning of the governance process?

Governance also has to account for what happens after a tool enters the organization, including when vendors update it. We learned that firsthand when an established vendor introduced a new AI capability without notifying us. The tool produced an incorrect finding. Fortunately, a physician questioned the result and a radiologist confirmed it was wrong, so there was no patient harm.

The fundamental governance problem wasn’t simply that AI made a mistake. We know AI can be wrong, which is why monitoring matters. The problem was that the tool had entered production without going through our evaluation process.

That experience reinforced that governance has to extend into vendor relationships. We now include contractual expectations that AI vendors inform us about changes and upgrades to their systems.

What did participating in CHAI’s Governance Work Group add to an approach that was already relatively mature?

Even with a mature process, there is enormous value in learning how other organizations are solving the same problems. Through this work, we’ve built relationships with other health systems and shared practices and tools. At one point, another organization had developed risk-assessment tools that were stronger than ours in certain areas. We connected our teams, learned from their approach and improved our own process.

That is one of the strengths of CHAI’s community. Organizations can learn from solutions that are already working rather than solving every problem independently.

How are CHAI’s Governance Playbooks influencing Hartford HealthCare’s process today?

Our governance process evolves almost every month because the technology isn’t standing still. As the Playbooks were being developed, I shared that work with the leader who operationalizes our governance process. Some guidance reinforced what we were already doing, while other areas helped us improve it.

For an organization like Hartford HealthCare, I’d characterize that value as incremental improvement. But for organizations without the same resources or experience, the impact can be much greater.

Community hospitals, rural health systems and other organizations without a deep bench of AI, informatics and IT expertise face many of the same challenges. The Playbooks give them a practical foundation they can adapt instead of requiring them to build a governance approach from scratch.

Joint Commission’s Responsible Use of AI in Healthcare certification spans areas including governance, data management, risk and bias reduction, and monitoring and evaluation. How do you see the Playbooks supporting organizations preparing for those expectations?

I see the Playbooks and Joint Commission certification as very well aligned. Hartford HealthCare is beginning the certification process now, so this is very much front of mind for us.

For organizations considering certification, the Playbooks provide a useful foundation for bringing the right team together and thinking systematically about responsible AI. Governance touches many areas, from assessing risk and bias to cybersecurity, ethics and ongoing performance. Having a structured resource helps organizations understand the questions they need to ask and the processes they need to establish as they prepare for those expectations.

As governance practices become more established, where do you see the next major challenge?

For me, the frontier is monitoring AI performance over time. As the number of AI tools proliferates, monitoring how they’re actually performing becomes incredibly challenging. Vendors can and should play a role, and some of those expectations can be built into contracts, but healthcare organizations still need their own oversight.

AI creates a different challenge from traditional medical technology because these tools aren’t necessarily static. They can be updated, evolve and change over time. Organizations need to know not only whether a tool was safe and effective when introduced, but whether it continues to perform as intended.

I don’t think anyone has completely solved that challenge yet. That’s also why collaboration matters. CHAI creates a community of people confronting these problems, sharing what they learn and helping each other develop better approaches as the technology evolves.