Blogs

Last week, CHAI released a series of outputs from its Q1-Q2 collaborative work groups. Included in this series of products is an Implementation Playbook and Testing & Evaluation Framework developed by the Ambient AI Work Group. These two new resources were created through a consensus-driven process that brought together diverse perspectives from across the healthcare ecosystem. Participants included pediatric practices, community health centers, integrated delivery networks, academic medical centers, and ambient AI developers, ensuring the guidance reflects implementation realities across a wide range of clinical settings rather than any single organization's approach.

Over the past several months, leading members collaborated to translate real-world implementation experience with ambient AI technologies into practical, vendor-agnostic guidance that any organization can use to support responsible adoption. Together, these resources provide an evidence-based foundation for healthcare organizations to conduct rigorous evaluation of clinical performance, safety, fairness, privacy and operational impact.

New Ambient AI Resources

Ambient AI Implementation Playbook: A practical, vendor-agnostic guide that equips health systems with consensus-defined best practices for procuring, deploying, governing and monitoring ambient AI technologies throughout the AI lifecycle.

Ambient AI Testing & Evaluation Framework: A living framework, hosted publicly on GitHub, that provides literature-backed methods and metrics to evaluate ambient AI across dimensions including usefulness, safety, fairness, privacy and business value. Organizations are encouraged to adapt the framework to their own environments and contribute new evidence as the field evolves.

Work Group Leads included: Oregon Health & Science University, Suki, Nabla, Infinitus

Much of what the Implementation Playbook covers isn't the AI itself, but the operational and governance questions around it. Across five lifecycle stages – procurement, pre-deployment, piloting, deployment, and monitoring—the work group kept hitting the same hard problems: consent and recording rules that shift with state laws and get increasingly complex in pediatric and behavioral health visits; questions about where patient audio is stored and whether it trains vendor models; per-seat licensing that ignores part-time and trainee providers; the pull toward unsanctioned "shadow AI"; and systems that drift as vendors quietly update them.

The 42 consensus best practices turn those challenges into concrete moves – treating ambient documentation as recording governed by state law (Best Practice (BP) 19), building mid-visit withdrawal-of-consent workflows (BP3), funding ambient AI as shared infrastructure rather than a per-provider "productivity tax" (BP20), and measuring success through clinician wellness and patient experience, not just adoption (BP29, BP41). Together they give organizations a shared starting point to adapt to their own needs rather than build from scratch.

Hear from our work group participants:

“Across my roles as a clinician, educator, researcher, and health policy leader, I kept hearing the same questions about ambient AI from organizations across the country. The CHAI Ambient AI Work Group brought together diverse perspectives to develop practical, consensus-based guidance on governance, consent, implementation, and evaluation – providing a shared foundation so the collective field can innovate responsibly rather than each building from scratch.” – R Logan Jones, M.D., FACP, Associate Professor of Medicine at Oregon Health & Science University

"Being a part of CHAI has been as much about learning as contributing. Working alongside diverse voices in medicine, research, and industry has deepened how I think about responsible AI, while giving me the opportunity to bring a nursing line of sight to conversations and help ensure nursing voices shape guidance that will ultimately be used at the bedside. It's a genuinely collaborative space, and I'm grateful to be part of a process that reflects the full care team." – Christopher H. Lee, MBA, BSN, RN-BC, Clinical Nurse III at UCLA Health

These resources reflect CHAI's broader mission to convene the healthcare community around practical solutions to shared challenges. Through collaborative work groups, clinicians, health systems, technology developers, researchers, policymakers and other stakeholders work together to develop consensus-driven guidance that helps organizations deploy AI responsibly and with confidence. CHAI looks forward to seeing these resources adopted, refined, and expanded by the broader community as ambient AI continues to mature.

Thank you to our members who made this work possible:

• Kang Hsu, MD, Canary Speech

• Meghan Reading Turchioe, PhD, MPH, RN, Columbia University School of Nursing

• Ashley M Hopkins, PhD, Flinders University

• Lucy Sutphen, MD, HealthPoint

• Craig Norquist, MD, HonorHealth

• Joe Derenzo, PMP, Healthcare Performance Group Inc.

• Renee Moss RN, BSN, CPhT, PMP, Healthcare Performance Group Inc.

• Jennifer Shannon MD, Individual Contributor

• Joan Chang, MS, MPhil, Individual Contributor

• Brittney Harrell, CISSP, Nabla

• Jeremy Attermann, MSW, National Council for Mental Wellbeing

• Jeffrey A. Gold, MD, Oregon Health & Science University

• R Logan Jones MD FACP, Oregon Health & Science University

• Taylor N. Anderson, MD, MS, Oregon Health & Science University

• Sudha Jayaraman MD MSc FACS, Suki AI

• Christopher H. Lee, MBA, BSN, RN-BC, UCLA Health

• Hwayoung Cho, PhD, RN, FAMIA, University of Florida College of Nursing

• Shiba Kuanar, University of Minnesota

• Ann Wieben, PhD, RN, NI-BC, FAMIA, University of Wisconsin-Madison