USMLE Step 1 Timing Effects on Performance During COVID

Abstract

Introduction: The United States Medical Licensing Examination (USMLE) Step 1 has historically been a high stakes exam, as residency programs use the Step 1 score, along with other factors, to screen and select medical students for residency positions. JABSOM students must take Step 1 before staring third year clerkships. Upon the onset of COVID in 2020, JABSOM permitted students to take Step 1 into their first semester of clerkships. While there have been studies on the effects of moving Step 1 post-clerkship, there are no studies in the literature on how Step 1 performance is affected if moved into the first semester of the clerkship year. Step 1 will now be pass/fail, but an examination into whether or not delaying Step 1 into the first clerkship semester has implications on performance will be useful should a future COVID wave cause similar testing site closures. This knowledge would be particularly helpful for students who are at risk of failing Step 1.

Objectives: The purpose of this study was to assess whether Step 1 performance varies based on the time elapsed up until the end of the first semester of clerkship year.

Methods: The study included USMLE scores from 64 JABSOM students who took Step 1 in 2020 who were not on extended leave. Regression analysis via Minitab software was used to assess whether or not variation in Step 1 scores could be explained by Step 1 timing, after accounting for standardized test taking ability for which Step 2 Clinical Knowledge (2CK) score was adopted as the closest proxy.

Results: The average Step 1 score was 238 (sd=15). The average and median days elapsed from the initial Step 1 exam period was 46 (sd=41) and 29. Regression analysis showed that Step 1 score significantly (p<0.0005) decreased based on days elapsed from the initial Step 1 exam period but there was no statistical significance (p<0.065) after accounting for Step 2CK score.

Discussion: This study indicates that students with lower Step 1 scores tended to be students with later test dates into the first semester of clerkship but that once standardized test taking ability was accounted for, with Step 2CK being the closest proxy, there was no effect of timing on Step 1 score. It’s likely that students who anticipated lower Step 1 scores, based on practice exams for instance, chose later dates for their Step 1 exam. This would explain why Step 1 score was significantly related to Step 1 date before Step 2CK was taken into account. Current literature indicates that delaying Step 1 post-clerkship is a better or non-inferior approach but does not address the impacts on Step 1 performance that slight delays in Step 1 timing have had during COVID or in general. This study suggests that Step 1 performance is not affected by time taken into the first semester of third year clerkships. Should another COVID wave cause JABSOM to allow students to again take Step 1 into the first semester of third year, students can feel comfortable delaying their exam into the third year if circumstances require it.

Target Audience: Advisors, Medical Students