Blogs

As AI becomes embedded across more healthcare technologies and workflows, health systems face a practical challenge: how do you build governance processes rigorous enough to protect patients, but flexible enough to keep pace with a rapidly changing landscape?

That challenge comes with another layer of complexity for pediatric health systems. Children have historically been underrepresented in the development and evaluation of new technologies, and differences in available data, anatomy and physiology can introduce distinct considerations when AI is applied to pediatric care.

R. Brandon Hunter, MD, FAAP, Assistant Professor of Pediatrics, Division of Critical Care Medicine and Co-Chair, AI Governance Tactical Committee at Texas Children’s Hospital, was a leading voice in the development of CHAI’s AI Governance Playbooks. He brought experience across pediatric innovation, medical devices, generative AI and AI governance to the working group. We spoke with him about what Texas Children’s learned from developing its own governance approach alongside peers, why scalable governance requires flexibility, and how shared guidance can help health systems prepare for emerging expectations around responsible AI.

Why was it important to bring a pediatric perspective into the development of the Governance Playbooks?

My background is in pediatric innovation, including medical devices, generative AI research and parent-facing applications. One lesson I’ve seen play out repeatedly with new technologies is that pediatrics can become an afterthought. It’s a smaller market, and children aren’t always adequately represented in the data or development processes behind new tools. That makes it particularly important for pediatrics to have a voice in conversations about how AI should be validated, governed and used responsibly.

There are also distinct considerations when evaluating AI for children. The amount of medical literature and data available for adults is much larger, and children themselves change considerably over time in terms of anatomy and physiology. Those differences can affect whether an AI tool performs as expected. Being at the table early gives us an opportunity to make sure those realities are considered as governance practices develop.

Texas Children’s already had AI policies in place when this work began. What challenge were you still trying to solve?

We had an AI policy and an initial framework for governing new and existing AI tools, but like many organizations, our approach was still evolving.

Generative AI has created a real shift. Healthcare organizations have had experience governing areas like traditional machine learning and AI in radiology, but now AI capabilities are becoming embedded across an enormous number of digital tools. The volume and ubiquity are fundamentally different.

Participating in the CHAI work helped us bring some of those pieces together and further formalize our approach. It gave us an opportunity to learn alongside other organizations that were grappling with many of the same questions in real time.

Did anything you learned from other organizations change how Texas Children’s approached AI governance in practice?

Absolutely. Initially, I thought a highly centralized approach was the answer. These technologies were new and potentially risky, so it seemed logical to have a small central group review everything.

But through conversations with other organizations, it became clear that this model becomes difficult to sustain as the volume of AI tools increases. We learned from institutions that were distributing some responsibilities while maintaining centralized oversight, and that helped us move toward more of a hybrid model.

Today, we have multidisciplinary governance that brings together clinical, legal, technical, information security and other perspectives to evaluate questions such as how a model performs, how it has been validated, what testing is needed and how it will be monitored. We also have strategic oversight that considers whether a technology is appropriate and whether it makes sense to devote organizational resources to it.

For certain technologies embedded within our EHR, we’ve created a parallel review process involving people who work most closely with those systems. It allows us to maintain consistent oversight while putting some decisions closer to the people with the relevant expertise.

What did developing the playbooks collaboratively provide that a more traditional, top-down framework might not?

When a technology is evolving this quickly, there can be a disconnect between top-down guidance and what people are actually encountering on the front lines by the time that guidance is published. What I appreciated about this process was that organizations actively dealing with these challenges could speak frankly about what they were trying, what was working and what they had learned.

It was also genuinely multidisciplinary and included organizations with very different levels of resources. If a conversation began moving toward an approach that might work for a large academic institution, someone would ask what that meant for a community hospital with only a couple of people responsible for governance.

That gives me confidence in the playbooks because they weren’t developed around one institution’s ideal model. They incorporate practical experience from different parts of healthcare.

As the number of AI tools grows, what makes a governance framework sustainable?

Scalability is one of the biggest challenges. Whatever process an organization establishes has to accommodate not only the tools coming through today, but the volume we expect in the future. At the same time, organizations need enough information to make informed decisions. We want to understand how a model has been validated, how it performs, how updates are handled and what monitoring will look like after implementation.

That balance is difficult because many AI tools can provide real value. We don’t want governance to become so burdensome that it prevents useful technologies from reaching clinicians and patients. But patient safety and privacy must remain our highest priorities, and healthcare organizations have a responsibility to ensure the tools we deploy are safe and effective.

To me, that’s why practical governance frameworks are so important. They give organizations a structured way to ask the right questions without pretending that every AI tool or use case carries the same level of risk.

Joint Commission’s Responsible Use of AI in Healthcare certification addresses areas including governance, data management, information security, safety and performance monitoring. How does the work Texas Children’s has done position you against those expectations?

I think we’re in a strong position because we’ve established a multidisciplinary framework that addresses many of those areas. Governance is only one piece. You also need the right expertise around data management, information security, compliance, safety monitoring and performance. We’ve built a structure that brings those perspectives together, and we’ve started putting AI tools through that process.

The next step is learning from experience. You don’t really know every edge case when you design a new governance process. You establish the framework, start using it and then refine it based on what you encounter. That is especially important in AI because the field changes so quickly. Being prepared doesn’t mean assuming the framework you have today will address every challenge six months from now. It means having a strong foundation and a process for continuously learning and adapting.

What do you think healthcare organizations should keep in mind as AI governance continues to mature?

Healthcare is in an unusual position with AI. These technologies are advancing rapidly and can offer meaningful improvements in efficiency, access to information and healthcare delivery, but the traditional evidence base we are accustomed to may not always exist yet.

That makes it important to avoid either extreme. We shouldn’t leave value on the table simply because a technology is new, but we also shouldn’t assume that promising performance eliminates the need for validation, monitoring and human oversight.

The best approach is to start governing, learn from what happens in practice and keep improving the process. That’s what makes CHAI’s collaborative approach valuable. No organization has every answer right now, but we can learn much faster by bringing together people who are confronting these challenges every day.