期刊名称：Journal of Educational Evaluation for Health Professions
电子版ISSN：1975-5937
出版年度：2018
卷号：15
期号：1
页码：1-6
DOI：10.3352/jeehp.2018.15.23
语种：English
出版社：Korea Health Insurance Licensing Examination Institute
摘要：Purpose:This study aimed to assess the agreement between 2 raters in evaluations of students on a prosthodontic clinical practical exam integrated with directly observed procedural skills (DOPS).Methods:A sample of 76 students was monitored by 2 raters to evaluate the process and the final registered maxillomandibular rela_x0002_tion for a completely edentulous patient at Mansoura Dental School,Egypt on a practical exam of bachelor’s students from May 15 to June 28,2017.Each registered relation was evaluated from a total of 60 marks subdivided into 3 score categories:occlusal plane orientation (OPO),vertical dimension registration (VDR),and centric relation registration (CRR).The marks for each category in?cluded an assessment of DOPS.The marks of OPO and VDR for both raters were compared using the graph method to measure re?liability through Bland and Altman analysis.The reliability of the CRR marks was evaluated by the Krippendorff alpha ratio.Results:The results revealed highly similar marks between raters for OPO (mean= 18.1 for both raters),with close limits of agree?ment (0.73 and −0.78).For VDR,the mean marks were close (mean= 17.4 and 17.1 for examiners 1 and 2,respectively),with close limits of agreement (2.7 and −2.2).There was a strong correlation (Krippendorff alpha ratio,0.92;95% confidence interval,0.79– 0.99) between the raters in the evaluation of CRR.Conclusion:The 2 raters’ evaluation of a clinical traditional practical exam integrated with DOPS showed no significant differences in the evaluations of candidates at the end of a clinical prosthodontic course.The limits of agreement between raters could be opti?mized by excluding subjective evaluation parameters and complicated cases from the examination procedure.
关键词：Purpose:This study aimed to assess the agreement between 2 raters in evaluations of students on a prosthodontic clinical practical exam integrated with directly observed procedural skills (DOPS).Methods:A sample of 76 students was monitored by 2 raters to evaluate the process and the final registered maxillomandibular rela_x0002_tion for a completely edentulous patient at Mansoura Dental School,Egypt on a practical exam of bachelor’s students from May 15 to June 28,2017.Each registered relation was evaluated from a total of 60 marks subdivided into 3 score categories:occlusal plane orientation (OPO),vertical dimension registration (VDR),and centric relation registration (CRR).The marks for each category in?cluded an assessment of DOPS.The marks of OPO and VDR for both raters were compared using the graph method to measure re?liability through Bland and Altman analysis.The reliability of the CRR marks was evaluated by the Krippendorff alpha ratio.Results:The results revealed highly similar marks between raters for OPO (mean= 18.1 for both raters),with close limits of agree?ment (0.73 and −0.78).For VDR,the mean marks were close (mean= 17.4 and 17.1 for examiners 1 and 2,respectively),with close limits of agreement (2.7 and −2.2).There was a strong correlation (Krippendorff alpha ratio,0.92;95% confidence interval,0.79– 0.99) between the raters in the evaluation of CRR.Conclusion:The 2 raters’ evaluation of a clinical traditional practical exam integrated with DOPS showed no significant differences in the evaluations of candidates at the end of a clinical prosthodontic course.The limits of agreement between raters could be opti?mized by excluding subjective evaluation parameters and complicated cases from the examination procedure.
