RANZCP COMPETENCY-BASED FELLOWSHIP PROGRAM – WBAS AND EPAS John Crawshaw Chair, Board of Education, RANZCP Congress, 2012, Hobart, Tasmania
PRESENTATION OVERVIEW • • • Introduction Formative & Summative Assessments Workplace-based Assessments – WBAs – Why use WBAs? – When to use WBAs? – WBA tools and how to use them Entrustable Professional Activities – EPAs – What is an EPA? – When to entrust an EPA – Standard of entrustment at Stages 1, 2 & 3 – How the EPA Handbook and RANZCP Certificate of Entrustment (COE) work – Process of entrusting an EPA How WBAs may help gauge readiness to entrust an EPA Questions
INTRODUCTION - COMPETENCY-BASED FELLOWSHIP PROGRAM AIMS • Develop a competency-based curriculum and Fellowship training program that - integrates the concept of competent performance into the College’s educational philosophy - integrates contemporary concepts of best practice in adult education into the training program - is developed in line with international specialist medical education best practice - increases flexibility in training - allows for more timely progression
INTRODUCTION CONTINUED • The competency-based 2012 Fellowship training program – is developed to better reflect the complexity of contemporary professional practice of specialist psychiatrists (medical expert; communicator; collaborator; manager; health advocate; scholar and professional) – provides a broad range of clinical experience during training – addresses issues associated with competing demands for service provision and training requirements for trainees – provides for Recognition of Prior Learning (RPL) – maintains high standards of training • Workplace-based assessments align to, and assess, the development of competencies
FORMATIVE ASSESSMENT – WHY USE WBAS? • Competency-based training requires actual demonstration of the knowledge, skills and attitudes needed for safe practice • WBAs – measure what doctors actually do in practice – assess competence in an authentic setting – provide mechanism for structured feedback from supervisor to trainee – help assess trainee progress – encourage concrete and effective feedback – formalise and add structure to current supervisor activities
SUMMATIVE ASSESSMENTS – WHY USE EPAS? • Entrustable Professional Activities (EPAs) – are representative tasks that are only entrusted to competent professionals – are summative assessment tasks in the competencybased Fellowship program – capture and record supervisor judgements about trainee performance and competence
FORMATIVE & SUMMATIVE ASSESSMENTS Formative Summative Assessment for learning Assessment of learning Immediate feedback Evaluates trainee’s learning at a particular point Feedback identifies strengths Feedback: Pass/fail Percentage Brief comments Feedback highlights areas for improvement Must pass to progress Feedback typically delayed Ways to improve performance outlined WBAs EPAs Exams, Psychotherapies, Scholarly Project, ITAs
HOW MANY WBAS AND WHEN? • WBAs are a way to: – Provide feedback for future learning – Help track and assess trainee progress • WBAs may occur – Early, midpoint and towards the end of a rotation – Whenever a trainee is seeking to gain formal feedback • Three WBAs required before each EPA can be attained – But, three WBAs does not mean an EPA has been achieved
WBA TOOLS • Four Board of Education approved WBAs: – Case-based Discussion (Cb. D) – Mini Clinical Evaluation Exercise (Mini-CEX) – Observed Clinical Activity (OCA) – Professional Presentation • Trainees are responsible for – Initiating the WBA process – Retaining the WBA forms after completion – Updating their learning plans
WBA ASSESSMENT • All WBAs are assessed on a 9 -point standard rating scale Trainee Stage Below standard for end of Stage (1 2 3) Meets standard for end of Stage (4 5 6) Above standard for end of Stage (7 8 9) Stage 1 – Basic standard Below standard for Basic trainee At Basic level as described on developmental descriptors Above Basic level. Moving towards the standard of a Proficient trainee Stage 2 – Proficient standard Below the standard of a Proficient trainee. Meets standard of a Basic trainee. Meets the standard of a Proficient trainee. Above the standard of Proficient trainee. Moving towards the standard of an Advanced trainee. Stage 3 – Advanced standard Below standard for Advanced trainee. Meets standard of a Proficient level trainee. Meets the standard of an Advanced trainee. Above the standard for an Advanced trainee.
WBA ASSESSMENT CONTINUED • The mid-point of the scale – ‘Meets standard for end of Stage’ – is the standard trainees should aim to achieve on completion of each Stage • Assessor also provides feedback comments • Not all assessed criteria will be assessed on each WBA form • Choose the appropriate assessment criteria from those listed on the WBA form
WBA ASSESSMENT – DEVELOPMENTAL DESCRIPTORS • Developmental Descriptors: – are behavioural descriptors for the Fellowship Competencies – articulate how the Developmental Trajectory applies to the Fellowship Competencies through the training stages – that articulates what is expected at each training stage is provided for each aspect of practice – are used as a guide when assessing a trainee’s performance on a WBA
USE OF DEVELOPMENTAL DESCRIPTORS IN WBA ASSESSMENT Example: Aspect of Practice: Assessment By the end of each Stage, the trainee’s performance in this aspect of practice can be assessed using the following standards: Stage 1 Basic level Stage 2 Proficient level Stage 3 Advanced level Conducts a standard assessment of a patient with typical psychiatric disorders, but requires supervision to elicit all necessary data and to understand the significance of data obtained. With supervision, performs a detailed and comprehensive assessment of a patient presenting with typical and atypical features Performs a detailed and comprehensive assessment of a patient presenting with complex or multiple problems, or in special groups.
WBA Assessment example – Case-based discussion NOVICE 2 months BASIC 11 months STAGE 1 Trainee 2 Does not formulate using a Bio. Psycho. Social model ? Score Below standard* Meets standard* Above standard* for Stage 1 6. Data synthesis 1 2 3 4 5 6 7 8 9 n/a
WBA TOOL – CASE-BASED DISCUSSION (CBD) • Occurs in regular supervision time • Discussion based on existing case notes • Used to assess – Clinical reasoning – Decision making – Integration of medical knowledge with case management – Ability to document these skills • Assessor feedback provided to trainee is key
CBD PROTOCOL • • • Trainee arranges time with assessor Trainee selects four cases and gives the files to the assessor Assessor chooses one of the four cases Trainee discusses case with assessor for approx. 15– 20 mins Assessor may prompt trainee on further discussion points – guidance on Cb. D tool Assessor provides feedback to trainee Assessor rates performance (9 -point scale) Next learning steps discussed and agreed on Form is signed by assessor and trainee Trainee keeps form and updates learning plan
WBA TOOL – MINI-CLINICAL EVALUATION EXERCISE (MINI-CEX) • Assessor observes a trainee in a clinical encounter with a real patient • Feedback provided on the agreed clinical task being assessed • Ideally, patient is new to trainee, at least in terms of clinical skill being assessed • Assessor should be – Familiar with Mini-CEX process – Familiar with the patient
MINI-CEX • Used to assess – History taking process – History taking content – Mental state exam – Physical exam skills – Communication skills – Data synthesis – Organisation and efficiency • Important: identify assessment focus before commencing exercise
MINI-CEX PROTOCOL • Trainee arranges with assessor to observe Mini-CEX • Trainee and assessor agree on competencies being assessed • Mini-CEX occurs: – – Clinical encounter, observed by assessor (15– 20 mins) Discussion and feedback follows immediately (10– 15 mins) Self-reflection time Assessor rates performance on Mini-CEX form (5– 10 mins) • Assessor takes no part in encounter unless intervention required for patient safety • Trainee and assessor discuss feedback, agree on next steps and sign form • Trainee keeps form and updates learning plan
WBA TOOL– OBSERVED CLINICAL ACTIVITY (OCA) • Similar in format to summative Observed Clinical Interview (OCI) • Occurs over two sessions • Trainee observed for the duration of an initial patient assessment • OCAs are good preparation for summative OCI in Stage 3 • OCA and OCIs are different: OCA is formative but OCI is summative • Trainees should be encouraged to use the OCA tool from Stage 1 as this will assist them to prepare for the OCIs.
WBA – OBSERVED CLINICAL ACTIVITY (OCA) • Examines – History taking and content – Mental state and relevant physical exam skills – Data synthesis – Management plan skills
OBSERVED CLINICAL ACTIVITY (OCA) – PROTOCOL • Trainee arranges with assessor to do OCA • Trainee nominates a clinical case, ideally new to trainee • Full OCA requires two 1 -hour sessions OCA (session 1) Clinical encounter 50 mins Post-encounter feedback 10 mins Thinking/self reflection Trainee’s own time OCA (session 2) Presentation of Assessment & Viva 20 mins Presentation of Plan & Viva 20 mins Feedback 10 mins Total 110 mins
OBSERVED CLINICAL ACTIVITY (OCA) – PROTOCOL CONTINUED • Feedback is given by assessor • Trainee and assessor discuss and agree on next steps, and sign the form • Trainee keeps the form and updates the learning plan
WBA TOOL – PROFESSIONAL PRESENTATION • Used in situations such as – – – Journal club Case presentations Community education presentations Clinical audits Grand rounds In-service presentations, etc. • Assesses aspects of typical presentation skills expected from trainees – – – Introduction Context Analysis and critique Delivery Questions Quality of information
PROFESSIONAL PRESENTATION
PROFESSIONAL PRESENTATION PROTOCOL • • • Trainee selects forum and makes all necessary arrangements Develops presentation Trainee arranges for an assessor to attend Trainee presents presentation of at least 30 mins including questions Assessor observes entire presentation Assessor rates performance Assessor provides feedback to trainee immediately after presentation Next learning steps discussed and agreed on, form signed Trainee retains form in their file
CHILD & ADOLESCENT WBA PILOT STUDY FEEDBACK • Two barriers to uptake – “Buy-in” – Time • But, with more use and familiarity, WBAs should fit into regular supervision time • After initial concern, trainees and supervisors found a rating scale useful • Trainees found them helpful in – gauging their progress – highlighting areas to work on • View that structured process would aid struggling trainees
EPAS – WHAT IS AN EPA? • Entrustable Professional Activity • Specific activities that a trainee must – Demonstrate their ability to perform; and – Perform with sufficient independence with distant (reactive) supervision • Supervisor must be confident that the trainee – Knows when to ask for additional help – Will seek timely assistance • An EPA should be – A task of high importance (core business) – A high-risk or error prone task – A task that is exemplary of a number of Can. MEDS roles
EPAS – STAGE 1 AND 2 GENERAL PSYCHIATRY • Stage 1 General Psychiatry 1. 2. 3. 4. Producing discharge summaries and organising appropriate transfer of care. Initiating an antipsychotic in a patient known to have schizophrenia. Active participation in the multidisciplinary team meeting. Providing an explanation to a family about a young adult’s major mental illness. • Stage 2 General Psychiatry 1. 2. 3. 4. 5. Demonstrating proficiency in all the expected tasks associated with prescription, administration and monitoring of ECT. The application and use of the Mental Health Act. Assessment and management of risk of harm to self and others. The safe and effective use of clozapine in psychiatry. Cultural competence.
EPAS – STANDARDS OF ENTRUSTMENT LOW INDEPENDENCE DEVELOPMENTAL TRAJECTORY HIGH LEVELS SUPERVISION LOW LEVELS SUPERVISION STAGE 1 STAGE 3 STAGE 2 BASIC PGY 1/2 HIGH INDEPENDENCE PROFICIENT ADVANCED JUNIOR CONSULTANT
EPAS – STANDARDS OF ENTRUSTMENT • As with WBAs, EPAs are entrusted at the following levels – Stage 1 – Basic – Stage 2 – Proficient – Stage 3 – Advanced • These standards are described by the Developmental Trajectory which traces the development of a trainee through training • Developmental Descriptors help to define the standards
HOW DO THE EPA HANDBOOK AND RANZCP CERTIFICATE OF ENTRUSTMENT WORK? • All EPAs are detailed in the EPA Handbook, which is available on the CBFP website • EPA Handbook is helpful for supervisors and trainees • The website version is always the most up-to-date • Every EPA also has a RANZCP Certificate of Entrustment (COE) – Summary version of full EPA – Sign-off required on RANZCP COE to certify entrustment
PROCESS OF ENTRUSTING AN EPA • Insert full EPA here
PROCESS OF ENTRUSTING AN EPA
PROCESS OF ENTRUSTING AN EPA • EPA Handbook document contains a full description of the knowledge, skills and attitudes required to gain entrustment of that professional activity • May or may not be attained in dedicated supervision time • Entrustment decision draws on all data available to supervisor – WBA performance – Observation – Information from other staff/allied health/etc
ENTRUSTMENT EXAMPLE: CONSULTATION–LIAISON PSYCHIATRY EPA • Care for a patient with delirium Skill: Negotiates clinical role throughout the course of the delirium episode Trainee may assess patient, liaise with staff, etc. On the following day, trainee discusses this case with a supervisor in a Cb. D as Workplace-based Assessment Skill: Considers the patient’s capacity to consent and any implications Trainee may have consented a different patient with a different issue on another occasion and completed a Cb. D Knowledge: Accesses, appraises and applies best level of evidence. Supervisor has observed trainee working with a number of patients and considers their knowledge to be good.
HOW WBAS HELP GAUGE READINESS TO ENTRUST EPAS • WBAs provide an evidence base • The structured feedback provided by a WBA highlights trainees’ strengths and weaknesses • They can measure progress over time • WBAs can be used to assess difficult or complex tasks
PROCESS OF ENTRUSTING AN EPA • When the supervisor judges the trainee to be competent to perform the EPA with only distant, reactive supervision → entrustment achieved • To document entrustment – Trainee prints the RANZCP COE – Supervisor completes RANZCP COE – RANZCP COE signed by supervisor, trainee and DOT – EPAs attainment will be reported to the College on the In-Training Assessment report at the end of each rotation
PROCESS OF ENTRUSTING AN EPA
QUESTIONS • Any questions? • Further resources: – CBFP website – new and updated: http: //cbfp. ranzcp. org/ – CBFP email: cbfp@ranzcp. org – Information is also available at the CBFP Booth
Скачать презентацию RANZCP COMPETENCY-BASED FELLOWSHIP PROGRAM WBAS AND EPAS
c190bfb8568a54994564bd8b25b0a03b.ppt