Nursing Students’ Evaluation Of OSCE At Taibah University Main Campus

Background

The Objective Structured Clinical Examination (OSCE) is a reliable assessment tool used to objectively assess clinical competence. However, there are limited evidence identifying the perception of nursing students in KSA regarding the OSCE. The aim of this study is to assess current OSCE applications based on the students' perspective. Methods: This is a cross-sectional study undertaken at Nursing College, Taibah University main campus, KSA. The study questionnaire was completed by student nurses immediately after the OSCE of various nursing courses. Main outcome measures were students’ view of examination attributes, quality of performance testing, validity and reliability of the process and usefulness of the OSCE compared to other assessment formats.

Results

Despite that students were accepting OSCE with respect to comprehensiveness (61.2%), clarity of instructions and sequence (57.8% and 55.8 %), fairness (55.4%) practicality and usefulness (52.2%), students felt that it was a strong anxiety stimulating experience (64.6%) and intimidating (52.2%). Concerns were expressed regarding the inadequacy of time to complete each stations (57%). A low percentage of examinees believed that stations were prepared in an organized and sequenced manner (41.4%) and the exams provided an authentic clinical experience (39.4%). Although Regular Nursing Program (RNP) assessed the OSCE more positively, clerkship and Multiple Choice Questions (MCQ) were the preferred format of examination for them.

Conclusion

OSCE is accepted as an assessment tools among the students. Further OSCE evaluation process should be taken place and more quality improvement initiatives should be done to address students’ concern and improve OSCE application.

Introduction

The assessment of students' clinical competence is of great importance and there are several measures in evaluating student performance. The objective structured clinical examination (OSCE) is one of these approaches to student assessment in which components of clinical competence reflected in actual clinical scenarios are evaluated in a comprehensive, consistent and structured manner in accordance to learning outcomes specified in the curriculum. Although there are various methods for evaluating students’ competencies, OSCE has been recognized as the most reliable method for evaluation of the clinical skills. Further, its reliability as an effective tool for nursing students’ clinical skills were supported by numerous studies and it has been widely used in the assessments of students’ clinical performance.

The stressful nature of OSCEs and potential unsuccessful outcomes brought negative consequences to the students, therefore, better planning in conducting OSCE is critically important. On the other hand, the various advantages of OSCE such as improving students’ clinical performance, preparing highly qualified and competent graduates, increasing decision making capabilities and enhance teaching level compels it as essential concern to all nursing faculties and clinical instructors in the context of acceptance, effectiveness, comprehensiveness, and accurateness of this clinical evaluation. In this vein, this educational approach requires robust design based on sound pedagogy to assure practice and assessment of holistic nursing care.

The Nursing College at Taibah University has introduced OSCE in 2014 to be part of all clinical courses. Generally, OSCEs are undertaken as students’ midterm and final examinations in the practical part of nursing courses and completed prior to their theoretical final examination. Number of stations and contents varies among courses. Although OSCE is instituted in the university, there is no evidence that current OSCE practices at the College of Nursing are meeting students' clinical competencies needs. In this regard, their insights are deemed important to understand the plausible lapses in development, organization and actual conduction of OSCE, which in turn would provide guidelines for faculty development initiatives and a curricular reform. Moreover, student feedback among these attributes is one of the key elements to enhance OSCE as an evaluation tool. The aim of this study is to assess current OSCE applications based on the students' perspective.

Methods

This study is a descriptive cross sectional design involving both male and female sections of the main campus of the university. Study aim and objectives were explained to all students preparing for their OSCE exam. The study questionnaire was distributed to all nursing students prior to each OSCE exam and informed to drop their responses to the collection box in case they like to take part in the study. Retuning the questionnaire to the collection box was considered as a permission to participate. No personal or any identifiable data were collected. Participation was a voluntary basis and students were assured that those who will decline involvement in the survey will not be penalized.

There are 452 students enrolled at Nursing College. Those who undertake courses involving clinical skills evaluation are 374 students. With a response rate of 66.8%, 500 responded to the study questionnaire. An adapted questionnaire from Pierre et.al. (2004) was the primary tool used to gather relevant data for the study. There was minimal amendment undertaken to the original questionnaire. However, since it was to be used with participants of culture and language different than for whom it was originally developed, the researchers translated the content of the questionnaire into Arabic language. Early face validity was attained after administering the instrument to seventeen (17) students. Data analysis were done using SPSS version 23.

Results

Majority of the respondents agreed that the OSCE was comprehensive covering wide area of knowledge (61.2%) and included broad range of clinical skills (56.6%), yet only thirty-seven percent (37.4%) believed that the stations in the examination would permit them to compensate in some areas of weaknesses. Further, less than half of them (41.4%) agreed that the assessment test helped them to identify weaknesses and gaps in their clinical skills sets. Concerning quality of the performance test, only half of the participants felt they were well oriented about the nature of exam and the required tasks to perform were in accordance to ones been taught in clinical teachings (53.2% and 52.8% respectively). Similar distribution was evident in the item which reflects that the required skills to demonstrate were fair and consistent with the actual curriculum of the course (53.4%).

The OSCE validity and reliability result showed that (52.2%) of the student nurses believed that it was a practical and useful experience for them. Furthermore, they are indefinite whether their scores were an actual reflection of their essential clinical nursing skills (45.8%). A similar low percentage of participating students agreed that marking criteria in the exam is standardized (46%). However, almost two-thirds felt that personality, ethnicity and gender did not affect OSCE scores. When it comes to preferred assessment stand, majority of student nurses perceived that the MCQ and clerkship were the easiest assessment formats (57 and 21% respectively). Most of the respondents agreed that they learned from clerkship (34.4%) and MCQ (32.4%). Clerkship (33.2%) and MCQs (32.8%) were the preferred formats of student nurses in their clinical years with some to a lesser extent agreed that it should be OSCE (23.8%).

It was observed that there was significant difference on OSCE evaluation when grouped according to education level (0.000) and program (0.001). This means that the responses varied significantly and was found out that those who are in level 5 and RNP students have higher OSCE ranking compared to NBP students. Similarly, students' perception on validity and reliability showed variations according to education level and program. This was observed on level 4 and RNP students. Computed r-value for age, GPA, and experience relationships to profile variables indicate almost negligible to moderate correlation, however, only experience (0.029) shows significant relationship on OSCE evaluation; age (0.001) and experience (0.008) on students' observation of validity and reliability. Therefore, the lesser the experience and younger the age, the more that they assessed the OSCE positively. There was no statistical significance in the regression model that was run since the p-value of 0.190.

Discussion

Participants in the study discerned that the OSCE as an assessment approach to the clinical practice had acceptable construct validity. This was evidenced by affirmative responses with regards to fairness, comprehensiveness, authenticity and objectivity of this assessment process. These findings are similar with past studies on OSCEs in the medical literature. The respondents however conveyed concerns about whether the exam would minimize chances of failing or the results were a precise reflection of their clinical skills. Since that OSCE is relatively new to the nursing college, these concerns are important to integrate for future OSCE preparation workshop. Many student nurses felt that the assessment tool was stressful and intimidating experience. The students’ perception with regards to the gap in the structuring and sequencing of stations, students’ awareness of the level of information needed and the time to complete skills at each station may have intensified respondents’ anxiety. These indicators inferred the need to re-examine the examination logistics particularly those sections describing the roles and responsibilities of OSCE committee, orientation and debriefing of examinees, preparation of the exam materials including station profile, the opening statement (scenario or preface) and the candidate instructions; and the recommended standard and guidelines for the delivery of exams which also includes the allowable time to complete skills at each station. The current study come in accordance with earlier international studies revealing that OSCE is a powerful anxiety stimulating experience and that the degree of anxiety lessens as learners advance through the examination. Further, anxiety and lack of confidence were related to insufficient preparation for the assessment, which may have influenced student perception of the exam especially for those examinees who answered the questionnaire after their OSCE.

Therefore, stress-management and fatigue-relieving techniques should be included in the OSCE preparation workshop. In terms of quality of performance testing, almost half of the participants were convinced that tasks of each stations were fair and consistent with the actual curriculum. Possible explanation for those who did not consider that the OSCE reflects the real curriculum is that not all students have the opportunity to apply their new skills in the nursing lab, therefore, accumulated required skills result in skills-gap. However, these parameters can be exemplary cornerstone of enhancement measures of the assessment tool under scrutiny. In this regard a test matrix which was applied only in theoretical exam of the university can be an essential activity in OSCE preparation to ensure that this examination method would be maximally valid based on a blue print. In addition, students' skills log book should be examined by the OSCE panel prior to the exam date to ensure that all students had practiced their acquired skills. Scores for other indicators such as the authenticity of each stations reflecting real clinical setting or life situations and adequacy of time allocation to complete each tasks were considerably low. Gormley, Sterling, Menary, and McKeown (2012) inscribed techniques and principles that can be used by the OSCE team to enhance the realism of this assessment process such as use of standardized patient (SP), video clips of real patients to recreate clinical scenario, hybrid simulations using wide range of manikin devices, modifying the environment surrounding of the SP or manikin and moulage kits. These approaches can be useful to tailor a more realistic clinical experience. Diversely, due to huge number of students per course, the length of time for each stations in the college’s OSCE was between 3 - 5 minutes. This is different from the Saudi Commission for Health Specialties (SCFHS) guidelines of 5-10 minutes for each station when examining undergraduate students.

Furthermore, Khan, Ramachandran, Gaunt, and Pushkar (2013) asserted that an appropriate and realistic time allocation for task at individual stations improve the test validity which generally lasting between 5 and 10 minutes. Correspondingly, Humayun and Haider (2016) propounded that the reliability and validity are both influenced by the number of stations and total length of examination. Therefore, examination length is one of the critical indicators of OSCE that should be augmented. The findings of this study about the rate of various assessment formats to which they have been exposed denoted that MCQs and clerkship are the preferred type of clinical examination of the student participants. Interestingly, these mode of exams had received considerable high scores in the indicators describing the acceptable level of difficulty of exam, fairness and opportunities to learn. Further, majority asserted that the MCQs and clerkship should be employed more often in the clinical years of the program compared to other assessment methods. These results are consistent with earlier studies where MCQs was the easiest and fairest assessment format as perceived by the students.

In the similar sense, student participants in the current study obtained higher grades in the clinical area compared to theoretical examinations. This may be attributed to their preference since it was easier for them to obtain scores through these test methods compared to essay/SAQs and OSCE. This context entails a scientific inquiry to determine the gaps or the differences in the outcome among these assessment methods. It was observed in this study that RNP students have higher assessment on the evaluation of OSCE. Similarly, the younger age assessed the OSCE more positively. This implied that previous experiences of those in NBP both in the academic institutions and hospitals may influenced their perception about the college’s OSCE. In this context, it is noteworthy to assert the urgency of matching up the college’s OSCE with the evidenced based practices of local and international higher academic institutions and university hospitals.

Conclusion

It was observed in this study that OSCE was accepted by the majority of the nursing students they acknowledged its positive impacts on their learning achievements. On the other hand, a considerable proportion of the sample did not support OSCE implementations. In this regard, the OSCE for the Bachelor program of the College of Nursing in the university had been challenging yet the student participants’ acceptance and their participation were motivating. Their feedback was considered by the faculty as key indicators for successful implementation of the assessment tool and also provided impulse for enhancement measures.

Recommendation

The study recommendations include: continuous implementation of OSCE evaluation process; matching up the college’s OSCE with the evidenced based practices of local and international higher academic institutions and university hospitals; and promote students' awareness about OSCE examinations. Further, quality improvement initiatives should be established to address students’ concerns and enhance OSCE applications into Nursing College evaluation system. Ethical approval Research ethics committee of Taibah University has reviewed and approved the conduct of this research study.