From Pilot to Practice: Building a Competency-Based POCUS Assessment Framework in Family Medicine

Abstract

Introduction Point‑of‑care ultrasound (POCUS) is increasingly essential in clinical practice, yet residency programs lack standardized teaching and competency assessment frameworks. To address this gap, the University of Hawai‘i Family Medicine Residency Program developed and piloted structured obstetric (OB) and musculoskeletal (MSK) POCUS checklists, tailored to Family Medicine through an environmental scan and consensus-based adaptation. The instruments support direct observation, consistent feedback, and competency‑based assessment. This project describes the development, implementation, and early evaluation of this framework.

Objectives To develop, implement, and pilot structured OB and MSK POCUS assessment checklists program, and evaluate their feasibility, usability, and perceived educational impact on learner development within a family medicine residency.

Methods A POCUS‑trained faculty group created structured OB (first‑ and third‑trimester) and MSK (knee, shoulder, hip) checklists by reviewing existing tools, literature, and adapting content for Family Medicine. The checklists aligned with direct‑observation formats and covered interpersonal skills, scan techniques, and medical knowledge, along with learner self‑confidence ratings and observer global assessments. Faculty completed targeted training, including structured learning modules and hands-on practice, to ensure readiness for assessment. The tools were introduced to faculty and residents and piloted at two sites over eight months. Post‑encounter surveys on feasibility, usability, and learning informed iterative PDSA cycles to refine both content and workflow.

Results Five faculty completed 39 checklists (28 OB, 11 MSK) for PGY‑1 to PGY‑3 residents and one fellow;four focused on OB, while one performed MSK assessments exclusively. OB assessments were primarily first‑trimester scans (71%), while MSK assessments focused on shoulder and knee exams. Sixty post-encounter surveys were collected (33 attending, 27 residents). Checklist completion was feasible, with most OB (76%) and MSK (63%) forms finished in under five minutes. Faculty and residents used the tools before, during, and after encounters, with OB checklists were more often utilized during the encounter. Feedback was usually provided post‑encounter, though 52% of residents performing OB POCUS reported receiving feedback at multiple stages. Perceived educational value was high: 76% of OB and 88% of MSK faculty evaluations rated the checklists as very useful, citing its role in assessing competence, identifying knowledge gaps, and structuring feedback. Nearly all residents performing OB POCUS (95%) and MSK POCUS (100%) found the checklists helpful for clarifying expectations, reinforcing systematic scanning, and supporting actionable feedback. Common challenges included time constraints, especially the medical knowledge section and difficulty assessing items not easily observable. Suggested improvements included streamlining content, improving accessibility, and adding visual aids or QR‑linked resources.

Discussion Structured OB and MSK POCUS checklists were feasible to integrate into Family Medicine workflows and were perceived as highly useful for guiding direct observation and feedback. The tools facilitated competency‑based progression for residents. Through iterative PDSA cycles, usability and workflow alignment improved, and both checklists demonstrated strong educational value despite time pressures and variation in faculty involvement. Limitations include small sample size, concentration of assessments among a few faculty, and reliance on perceived utility rather than objective performance metrics. Overall, This pilot provides a scalable model for structured POCUS assessment in Family Medicine. Next steps include gathering data on the tools’ utility for assessing competency and guiding skill development, as well as further faculty development and expansion across additional POCUS domains.