A study of Two Clinical Performance Scores: Assessing the Psychometric Characteristics of a Combined Score Derived from Clinical Evaluation Forms and OSCEs

Abstract

Background/Purpose - It is important to improve the quality of clinical skill assessments. In addition to using the OSCE, the clinical skills of medical students are assessed with clinical evalu­ation forms (CEFs). The purpose of this study is to examine the psychometric characteristics of an OSCE/CEF composite score. Methods - This study included 2 medical student classes from a large medical school. Students completed approximately 12 OSCEs and were rated over 33 times on a CEF. The reliability of the CEF and OSCE were estimated. A correlation between the mean OSCE score and the mean CEF was calculated and corrected for attenuation. Classical methods were used to examine composite score reliability. Results - For both classes there was a statistically significant correlation between the CEF and OSCE (r = .27 & .42, p = .003 & .0001). The disattenuated correlations were .44 and .68. Weight­ing the OSCE in the composite score as high as .4 was associated with only a small decrease in composite reliability. Conclusion - These results demonstrate that assessment information based on simulated and ac­tual patient encounters can be combined into a composite. Since a composite score may provide a more valid measure of clinical performance, this study supports using a combined CEF and OSCE measure. Keywords: psychometrics, composite score, clinical skills, OSCE, reliability