Objective Structured Clinical Examination OSCE Nadine J Kaslow

Objective Structured Clinical Examination (OSCE) Nadine J. Kaslow, Ph. D, ABPP nkaslow@emory. edu

Context • ACGME Outcome Project – Toolbox of Assessment Methods • Royal College of Physicians and Surgeons of Canada – CAN MEDS Assessment Tools Handbook

Examples of OSCE Stations 1 1. resuscitation of a child 2. consultation skills 3. physical examination skills 2 3

Context • Competency Assessment Toolkit for Professional Psychology • High fidelity approach for summative assessment of myriad foundational and functional competencies

Context • Competency Assessment Toolkit for Professional Psychology • OSCEs are useful for assessing • Readiness for practicum? • Readiness for internship • Entry level to practice • Advanced credentialing

What is an OSCE? • Objective – all candidates are presented with the same stimuli

What is an OSCE? • Objective • Structured – Specific foundational and functional competencies are tested at each station and the marking schema for each station is structured

What is an OSCE? • Objective • Structured • Clinical Examination – Test of performance of clinical competencies, with an emphasis on skills and attitudes

What is an OSCE? • Evaluation tool that allows people to be observed performing in many different clinical situations

What is an OSCE? • OSCE combines • Multiple observations • Standardization of content • Range of difficulty

What is an OSCE? • Effective method to assess foundational competencies • Professionalism (e) • Scientific knowledge & methods (e) • Reflective practice (e) • Individual & cultural diversity (e) • Relationships (o) • Ethical & legal standards & policy (o)

What is an OSCE? • Effective method to assess functional competencies • Assessment (o) • Intervention (o) • Consultation (e) • Research and evaluation (e)

Methodology • Standardized patients (SP) • Stations • Marking scheme

Standardized Patients (SP) • SPs are lay persons who are trained to portray scripted patient presentations accurately and consistently across many encounters • SP encounters are credible and reliable

Standardized Patients (SP) • SPs are used to teach • Foundational competencies • Interpersonal and communication skills (CIS) • Clinical reasoning • Functional competencies

Standardized Patients (SP) • SP ratings of CIS provide a good proxy for actual patient satisfaction • SP Communication • Increases patient satisfaction • Facilitates doctor-patient relationship • Increases compliance • Decreases malpractice claims

Standardized Patients (SP) • Several encounters are required to obtain a reliable estimate of a person’s competence • SP assessments are frequently organized as a set of cases or stations • This series of encounters is known as OSCE

Implementation • While there are variants on the original OSCE format, key implementation steps are • Train SPs in the issues of encounter and ensure they don’t provide any information unless the person being assessed requests it

Implementation • SP encounters or tasks requiring interpretation of clinical information must be provided at separate stations • These must last 5 -10 minutes and be observed by the assessor • Topics – relationship development, risk assessment, diagnosis, therapeutic technique, consultation strategy

Implementation • At each station, the person being assessed completes notes about the SP

Implementation • The person being assessed moves between stations when an announcer indicates in accord with a specified time sequence • Separate performance scores are generated for the task at each station based on input from the SP, the person being assessed, and the assessor

Implementation • The scores from multiple informants across the stations or tasks are combined by the assessors to reach a final outcome determination

Role of Examiners - YES • Observe the performance of the trainee at a particular task • Score according to the marking scheme (get practice at marking) • Contribute to the good conduct of the examination

Role of Examiners - NO • Re-write the station • Interfere with the SP’s role • Design their own marking scheme • Teach

Psychometrics With appropriate attention to design, OSCEs have acceptable psychometric properties including • • Good inter-rater, inter-station, and split-half reliability • Good generalizability • Strong content, construct and concurrent validity

Psychometrics OSCE scores correlate moderately with other performance indicators and other forms of evaluations •

Psychometrics OSCE has increased reliability and content validity with greater number of stations and similarity between tasks at different stations •

Strengths • Provides for a high fidelity assessment

Professional Authenticity DOES SHOWS HOW KNOWS HOW Skills & Attitudes Knowledge KNOWS Miller et al. , 1990

DOES SHOWS HOW OSCE KNOWS HOW KNOWS Miller et al. , 1990

Strengths • Measures clinical competence crosssectionally using standarized means • Focuses on observable behaviors • Enables fairer peer comparison

Strengths • Allows for assessing complex competencies without endangering patients’ well-being • Encourages a collaborative assessment approach • Has the potential for peer feedback and assessment

Strengths • Has been extensively researched and found to be valuable for summative assessment • Provides valuable information for curriculum review

Challenges • Is challenging to create and administer • Is labor intensive • Is costly • Only cost-effective when many individuals are examined at one administration

Challenges • Requires high demand of other resources • SPs • Assessors • Time commitment • Physical resources

Challenges • Doesn’t provide longitudinal assessment • Doesn’t adequately tap complex skills requiring integrated professional judgment

Example: SP Information • I am a 25 year old who comes to the Emergency Room for treatment of a 2 -3 week old puncture wound that I can see but no one else can • I am having a first episode of psychosis and have numerous alterations in my thinking and function; however it is not immediately obvious that my thoughts are disordered when the student begins taking my history

Example: SP Information • Chief complaint - Stepped on a nail • Opening statement - I stepped on a nail and the wound won’t heal – If pressed to say more… “It’s making me sick and I need help

Example: SP Information • Appearance - alert and oriented, calm, cooperative, poor hygiene (dirty unmatched clothes, hair uncombed, no make up or grooming) • Look like you haven’t showered in days • Have some eye contact but less than normal • Your affect is flat • You talk very little and you have no emotion in your voice when you are talking

Example: SP Information • Thought processes - you are focused on “saving America” and you mostly stay on topic when speaking but sometimes you start talking about one thing and it turns into a related topic but not on point • An example might be “Coke advertisements transmit messages to me. The first coke slogan was ‘Drink Coca-cola’ released in 1886. ” • Refer to three Coke slogans during your interview. If possible say these when you can tangentially connect them to something else you are saying.

Examples

Conclusion • OSCEs are a valuable and well-developed assessment approach in other health professions • It behooves us as psychology to develop and share OSCE scenarios for the essential components of foundational and functional competencies

Conclusion • OSCEs are advantageous to incorporate in our assessment armamentarium for summative evaluations particularly related to key transition phases • Attention needs to be paid to increasing the feasibility of this high fidelity assessment methodology

Questions