Reliability of Virtual Objective Structured Clinical Examination in Pharmacy Practice Experience Education Using Generalizability Theory

ABSTRACT

Background and Objectives: The virtual Objective Structured Clinical Examination (OSCE) was established using the MyDispense database in various pharmacy schools worldwide. However, rigorous reliability estimation is lacking, which could affect student preparedness for the real-world pharmacy practice experience. Hence, this study aims to estimate the reliability of virtual OSCE using a robust reliability estimation method that has already been successfully established in medical schools. Materials and Methods: The reliability and components of variation were estimated using the generalized theory (G-theory) statistical framework. 4 facets were used to assess the components of variation: students, stations, items and years. The reliability of different stations (from 2-5) in virtual OSCE was estimated, with 182 students registered for pharmacy practice experience in various academic years between 2020 and 2023. The case scenarios were studied for content validity through expert academicians in pharmacy practice from universities other than the study site. Results: The G-coefficient with various virtual OSCE stations was close to the benchmark reliability of 0.70; however, it is still a low-stakes assessment. This study’s low-stakes assessment might be due to the estimation of component variance in more significant (22.60%) regarding the student’s performance varies mainly in different stations across the academic years. The low component variance (1.41%) indicated the negligible difference between the levels of students in various academic years. Conclusion: The number of stations in virtual OSCE strongly influences reliability. Pharmacy curriculum academicians can consider G-theory to estimate the reliability of virtual OSCE in preparing students for a better real-world pharmacy practice experience.

KEYWORDS

• Generalizability Theory
• Pharmacy Practice Experience Education
• Reliability
• Virtual Objective Structured Clinical Examination

Showing references and in-text citations exactly as published.

• DOI10.5530/ijper.2021159Copy

RELATED ARTICLES

Matched on section and article type within this issue of IJPER.

• Article · VOLUME 60, ISSUE 1SThe Role of EPAC in the Pathogenesis of Cardiovascular Complications: A ReviewNandini Dubey, Vishal Kumar Vishwakarma, Aanchal Verma, Ahsas Goyal, Harlokesh Narayan Yadav · pp. s1–s10Research ArticleSame type: Article
• Article · VOLUME 60, ISSUE 1SGlobal Comparison of Gene and Cell Therapy Regulations: A Cross-Regional Analysis of FDA, EMA, TGA, CSFDA, DCGI, PMDA, IFDA and MFDSSeyed Heidar Mohammadi, Gholamhossein Sadeghian, Hoseein Shomali, Mohamadreza Abedi, Behrooz Jannat, Faham Khamesipour · pp. s18–s32Research ArticleSame type: Article
• Article · VOLUME 60, ISSUE 1SReview on Exploring the Combination Effect of Aldosterone along with Plasma Renin with Melatonin and Hydro-Cortisol in Diurnal RhythmMainak Pal, Masud Sk, Saheli Mandal, Dipanjan Mandal · pp. s33–s41Research ArticleSame type: Article
• Article · VOLUME 60, ISSUE 1SRegulatory Pathway for Genotoxic Impurities in Europe, US, Canada, India, Australia and UKKavyashree Kadehalli Ramegowda, Sangita Mishra, Venkatesh Madhugiri Prakash · pp. s42–s51Research ArticleSame type: Article