Blogs

As AI moves from isolated pilots into everyday healthcare workflows, responsible adoption increasingly depends on more than deciding whether an individual tool is safe to use. Health systems need processes for documenting how AI works, educating the people who use it, and monitoring whether it continues to perform as intended.

Those needs are also becoming more formalized. Joint Commission’s Responsible Use of AI in Healthcare certification puts areas such as governance, risk management, education, and ongoing monitoring into a broader framework for responsible AI use.

Cleveland Clinic has been developing its approach to AI governance since 2021, building on an existing data governance program and adapting it as the technology has evolved. Jennifer Owens, AIGP, Senior AI Program Administrator at Cleveland Clinic, participated in the development of CHAI’s AI Governance Playbooks, bringing lessons from Cleveland Clinic while learning from organizations with different resources, structures and patient populations.

We spoke with Jennifer about how collaboration has strengthened Cleveland Clinic’s governance approach, why clear guardrails can accelerate rather than inhibit innovation, and what health systems will need to address as responsible AI moves from initial review to ongoing use.

Cleveland Clinic already had an AI governance program when the Governance Playbooks were being developed. What did you hope to learn from other organizations?

We had been doing this work somewhat in a silo for a long time, so I was looking for benchmarking. What are other institutions doing? Where are their strengths and weaknesses? Where can we share best practices?

AI governance has been in place at Cleveland Clinic since about 2021 as an outgrowth of our data governance program. Since then, our approach has come a long way. We’re doing a better job of evaluating vendor-developed and internally developed products against consistent criteria, and we’ve significantly improved our documentation.

The CHAI work gave us an opportunity to look outside our own organization and ask where we could make that process stronger.

How did developing the Playbooks alongside organizations with different resources and structures affect the way you thought about governance?

There’s a big difference between reading a finished framework and participating in the process that creates one. The working group allowed us to test our thinking against people who were actively doing this work in very different environments.

The perspectives from organizations outside large academic medical centers were especially valuable. It’s easy for someone focused on governance full time to think, “Everybody should want to do this.” But at a regional or rural hospital, AI governance might be far down someone’s priority list.

That changes how you think about implementation. The easier we can make responsible governance for organizations with fewer dedicated resources, the more likely they are to participate in the process.

Was there anything you heard through that collaboration that changed Cleveland Clinic’s own approach?

One example was model card documentation. Another organization talked about documenting an AI tool from multiple perspectives rather than relying on a single view of the product.

That shook up how we were thinking about our own documentation in a really helpful way. More broadly, participating in the working group increased my confidence in the final Playbooks because we could see both where our processes were mature and where we still had room to improve.

What does putting the Playbooks into practice look like at Cleveland Clinic?

We’ve been particularly interested in the guidance around responsible AI lifecycle management and risk and impact assessments. It has helped frame cross-functional discussions among AI governance, cybersecurity, risk and compliance and given those teams a shared starting point for planning.

That common foundation is tremendously valuable. Having guidance developed through a cross-functional group of healthcare stakeholders gives you a place to start as you build processes appropriate for your own organization.

How can stronger governance support innovation rather than slow it down?

We’re fortunate to have strong alignment with our digital leadership around the idea that clear governance can actually speed innovation because it makes the rules of the road clear.

Think about planning a trip. If you understand which route to take, where there is construction and what conditions you might encounter, it becomes easier to get from point A to point B. Governance can serve a similar purpose. Clear, current guidance gives people a better understanding of how to move an AI initiative forward responsibly rather than leaving them to figure out the boundaries along the way.

As AI becomes more widely used, why does education need to be part of the governance conversation?

We’re working both to increase baseline AI literacy across our workforce and to make sure people introducing AI tools understand their responsibility for educating the people who will actually use them. It isn’t enough to assume users will understand what an algorithm predicts, what data it is based on or when they need to exercise clinical judgment. We need educational materials and ways to better identify how well individuals understand how a tool is meant to be used.

That’s different from a traditional IT deployment where you might provide training once at deployment. With AI, education and change management need to continue alongside the technology.

How can governance also help clinicians navigate patients’ growing use of AI?

Patients are encountering the same AI technologies as everyone else, including using consumer tools to ask questions about their own health or the health of family members.

Some of the governance principles have helped us think about how clinicians can have those conversations: where AI can be useful, what a tool has actually been validated to do, what responsible use looks like and what medical expertise adds.

I didn’t necessarily expect governance guidance to extend into that clinician-patient conversation, but it has given us a helpful structure for thinking about it.

As organizations consider Joint Commission’s Responsible Use of AI in Healthcare certification, where do you see the biggest challenge ahead?

The certification is something we’re looking at internally at Cleveland Clinic, and I’m particularly interested in the ongoing monitoring component.

There is nothing more important to the continued responsible use of AI in healthcare than understanding how these tools perform over time. Right now, getting a holistic view of AI use across a hospital can be incredibly labor intensive and incomplete. Individual platforms may provide ways to track their own AI tools, but bringing that information together across an organization remains difficult.

That’s an area where there is still significant work to do. As governance matures, we need better ways not only to assess AI before it is introduced, but to understand how it is being used and performing once it becomes part of healthcare delivery.