摘要：Assessing competencies or program learning outcomes in educational programs is often a leadership challenge. This case study reports medical education program’s efforts to document undergraduate competency attainment using a pre-post, third-party, objective testing service that allows for inter-university comparison, a testing service that is being adopted by some certification and accrediting bodies. Students completed a pre-test after program acceptance and a post-test at the end of the last didactic semester (1.5 years later) just prior to their required internships. Scores and subscores were evaluated using t-tests (Holm-adjusted p-values). MANOVA models of sub-competency difference scores were also evaluated. Results indicate competency improvement for each of the 12 areas based on the n = 55 student sample, (p < .001 for all scores). These improvements were independent of ethnicity, age, gender, and grades. The average student improved by 12.85 points (95% CI of 10.52 to 15.18) with the largest improvements in strategic planning and leadership competency areas (21.30 and 18.33 percentage points, respectively). The third-party pre-post has some face validity given that student performance improved after completing a related curriculum as would be expected. Congruent with earlier studies, we find that repeated testing helps document competency attainment and that a single method for assessment is insufficient. We further document limitations of this 3d-party exam.
其他摘要：Abstract Background Assessing competencies or program learning outcomes in educational programs is often a leadership challenge. This case study reports medical education program’s efforts to document undergraduate competency attainment using a pre-post, third-party, objective testing service that allows for inter-university comparison, a testing service that is being adopted by some certification and accrediting bodies. Methods Students completed a pre-test after program acceptance and a post-test at the end of the last didactic semester (1.5 years later) just prior to their required internships. Scores and subscores were evaluated using t-tests (Holm-adjusted p -values). MANOVA models of sub-competency difference scores were also evaluated. Results Results indicate competency improvement for each of the 12 areas based on the n = 55 student sample, ( p < .001 for all scores). These improvements were independent of ethnicity, age, gender, and grades. The average student improved by 12.85 points (95% CI of 10.52 to 15.18) with the largest improvements in strategic planning and leadership competency areas (21.30 and 18.33 percentage points, respectively). Conclusions The third-party pre-post has some face validity given that student performance improved after completing a related curriculum as would be expected. Congruent with earlier studies, we find that repeated testing helps document competency attainment and that a single method for assessment is insufficient. We further document limitations of this 3d-party exam.
关键词：Competency-based medical education (CBME);Competency assessment;Pre-post testing;Quasi-experimental;Peregrine testing
