CAPTE On-site Reviewer Training MODULE 7 The Visit

CAPTE: On-site Reviewer Training MODULE 7 The Visit Report April 2017 This module discusses the report that the team submits at the conclusion of the on-site visit. The Visit Report is known as the Report of the On-site Review Team or ROSRT. 1

Objectives 1. Describe the purpose of the Visit Report 2. Identify the activities of the team members in creating the Visit Report a. Before the visit b. During the visit c. After the visit 3. Identify the path of the Visit Report from the end of the site visit to presentation at the CAPTE meeting 4. Utilize hints for writing the report in the upcoming training workshop Please take a moment to review the objectives for this module. 2

Purpose of the Visit Report Interviews with On-Site Visit participants Documents evidence (or lack of) to verify and clarify Self-Study Report Review of documentation provided on-site Examination of institutional facilities (library, offices, classrooms, laboratories, etc. ) The Visit Report is the mechanism used to convey the team’s findings to CAPTE. It provides the evidence found on-site that verifies or clarifies the Self-Study Report. Through its description of the team’s findings, it provides an assessment of the quality of the program. It is used by CAPTE, along with all materials provided by the program, to determine compliance with each required element and ultimately to determine the program’s accreditation status. 3

Visit Report & On-site Visit Before Team prepares for visit During Team writes during visit After Team submits, staff edits, & program responds The Visit Report is the main product of the on-site visit. As such, it is important to prepare prior to the visit, work diligently during the visit and submit the Report to the Accreditation staff in a timely manner once you return home. The next few slides will highlight some of the logistics involved in completing the report during each of these periods. 4

Creating the Visit Report Before the Visit • Team Leader assigns specific areas to each team member • Begin to write comments about assigned areas » Identify what needs to be clarified » Identify who or what might confirm IMPORTANT! Find a way to clearly identify previsit written comments. Due to the rapid pace of the on-site visit, it is essential that the team identify what needs to be clarified or confirmed well in advance of the visit. While it is helpful to start writing the Report prior to the visit, it is obviously essential that any pre-visit comment be updated based on what was found on site. Remember that the report is most helpful to CAPTE when it identifies the evidence found on site. Therefore, you will need some mechanism to identify pre-visit comments. Some team members use a different font, a different font color, or highlighting to identify comments written prior to the visit. Use whatever works best for you. 5

Creating the Visit Report Before the Visit • • Make notes on how well the response addresses the required element Compare response to the Evidence of Compliance listing Identify how to best verify/clarify on site Confirm that policies ‘live’ where reported IMPORTANT! Do Self-Study Report responses make sense in light of their mission One of the first things to do as you review the Self-Study Report, especially your assigned areas, will be to assess how well the program addressed each required element. Use the Evidence of Compliance listings to identify any missing, unclear or conflicting information. Consider how you might best clarify or verify information on site. All evidence should be considered in light of the institution’s and program’s mission and the required elements. It is important to keep in mind that this is not an assessment as to how you believe something is best done. The program only needs to demonstrate that it works for them and meets the expectations of the require element. 6

Creating the Visit Report At the Start of Visit • Program provides team – An updated General Information Form – Materials Provided On-site Form – Summary of Person’s Interviewed Form • Update each form as appropriate – Becomes part of your report! – E-mail as attachments with final report » Staff will insert into final ROSRT IMPORTANT! Ensure that team is provided with electronic copies of the forms To facilitate the work of the team, the program provides the 3 forms at the beginning of the visit in both paper and electronic formats. The paper version will facilitate using it while on-site. However, since these forms ultimately become part of the Visit Report, an electronic version is also needed so you can make revisions if necessary and submit to the Accreditation staff along with the Visit Report. The General Information form is the same form that is provided in the Self-Study Report. It provides the team with the most current information. Even if the program has no changes, it still needs to provide the form. You should review the General Information form for accuracy and if necessary make corrections or provide explanations of discrepancies in your written comments. As its name implies, the Summary of Person’s Interviewed, provides a list of those individuals the program has arranged to meet with the team. The team is responsible for checking this against who was actually interviewed and revising the form as needed. The third form provides an official record of the documents the team reviewed on site. The team is to indicate which documents were actually reviewed. If documents not listed on this form are reviewed, the team should add them to the form. Alternatively, the team leader can submit the form to staff and indicate that all materials listed were reviewed – and staff can make the changes in the document. 7

Creating the Visit Report During the Visit • Each evening the team completes those portions of the Visit Report for which evidence is available • Writing continues on Wednesday morning – Goal is for the report to be essentially complete before leaving – Overview Section of report MUST be finalized prior to Oral Report IMPORTANT! Revise pre-visit written comments based on on -site evidence In order to expedite the writing process, each team member is involved in writing the report. Team members do need to have a laptop during the visit. 8

Creating the Visit Report After the Visit • Team Leader mails hard copy of signature page • If necessary, finalize assigned areas immediately after the visit & send to Team Leader • Final editing by Team Leader • Team Leader e-mails report to staff with 3 forms: – General Information – Person’s Interviewed On-site – Materials Reviewed On-site IMPORTANT! Submit report within 10 working days of visit The team leader is responsible for submitting the Visit Report to the Accreditation staff. Typically, the team leader provides the team with an electronic version of the report at the end of the visit, or immediately upon returning home. Although there may be sections that still need to be written, more often the team is only reviewing the report for completeness and accuracy at this point. CAPTE has rules regarding when the report is to be provided to the program so it is very important to adhere to deadlines. If there is an unavoidable delay, the team leader must contact the Accreditation staff. The last page of the report is a signature page for the team. Since the end of the visit may be hectic, the team often signs it at the beginning of visit. Since this is the only document the team submits as a paper copy, it should be mailed immediately after the visit. Waiting until the report is emailed to the Accreditation staff to mail the signature page often results in a delay in providing the program with a hard copy of the report. 9

Creating the Visit Report Accreditation Staff • Inserts forms into final report » General Information Form » Persons Interviewed On-site » Materials Reviewed On-site • Removes evidence of compliance listings or worksheets • Checks grammar & spellings • Contacts Team Leader if clarification is needed • Password protects document and e-mail it to program • Provides feedback to the team regarding the report The Accreditation staff does the final editing of the Visit Report. While the evidence of compliance listings are included in the report to facilitate your writing, staff will remove them along with any notes that might have been written during the development of the report. Since the program has an opportunity to comment directly into the Visit Report, the last thing staff does is to password protect the document in such a way that the program can not change the team’s comments. 10

Path of the Visit Report The program has 30 calendar days from receipt of the Visit Report to respond to it. The Visit Report is provided in electronic and hard formats. Program provides comment in the grey box that follows each team response. The program also can add any supplemental materials. The Visit Report with program comments is then provided to CAPTE. IMPORTANT! Program will not be able to change team comments in the Visit Report As previously indicated, the institution has an opportunity to comment on the Visit Report. This allows them to clarify any mistakes of fact and provide further explanations, if needed. They are also able to provide additional materials as an appendix to the Visit Report. This includes any additional materials the team had requested to be sent to CAPTE. The institution’s response to the Visit Report further assists CAPTE in making accurate judgments. A copy of the Visit Report with the institution’s comments is sent to each team member by the Accreditation staff. The CAPTE Panel member assigned as the primary reviewer to the program usually contacts the team leader and must contact program director prior to the CAPTE meeting. This provides an opportunity for CAPTE to learn of any changes since the visit and obtain clarification, if needed. Please keep in mind that CAPTE can only use information from the team that the program has had an opportunity to respond to. Therefore, discussions between the team leader and CAPTE reviewer must address what is already part of the record – the team leader cannot provide new information. 11

Hints for Writing the Visit Report Relevant Good evidence is Verifiable from Multiple Sources Representative The Visit Report is especially helpful to CAPTE when statements are supported by evidence. Keep the following principles in mind when determining what evidence should be included in the report: • The evidence MUST be relevant to the required element. Sometimes teams make good comments that are not related to compliance as will be demonstrated in a later slide. Staff will often delete these comments during editing. In addition, comments should be relevant to level expected by the required element and/or entry- level practice. For example, comments by employers regarding the lack of skills that are not considered entry-level would not need to be included. • Evidence should be verifiable. Confirmation should be sought from multiple sources whenever possible. Is there written documentation that supports what faculty tell you? Do students and faculty indicate the same information? • Evidence must be representative of what is typically done, not what happened once or twice or what previously occurred but has since changed. Think in terms of assessing student clinical performance. If by the end of a clinical experience the student is consistently demonstrating good body mechanics, reporting this as a problem during the first week of the clinical is not representative of current performance and would not effect the student’s grade. 12

Hints for Writing the Visit Report Providing the evidence that leads to your comments Provide team’s assessment of the quality by Identifying the source(s) of evidence Identifying what has been verified or clarified It is more helpful to describe what you have found on site that has lead to your comments than just providing the assessment. For example, saying the laboratory facilities were impressive - or not impressive - does not give the Commission the details needed to determine compliance. 13

Missing or Incorrect Information Indicate what able to confirm on site Identify what is missing or incorrect Identify source(s) used to confirm When too much to provide in report, request program to submit after the visit Although the evidence of compliance listings identify the evidence that CAPTE believes is necessary to determine compliance, it is possible that compliance can be determined even if all material has not been provided. CAPTE has gone on record to say that if they can determine compliance based on the information they have, programs will not have to submit missing information in a Compliance Report. However, neither the program nor the team will know for sure if CAPTE will need the missing information. Therefore, the Visit Report should identify what is missing or incorrect in the Self-Study Report and then indicate what was confirmed on site and how the confirmation was made. If the related data is too lengthy to be provided in the Visit Report, the program should be requested to send it as additional material as part of its response to the Visit Report. There is a page at the end of the Visit Report for the team to indicate what additional materials, if any, have been requested. In addition, the request should be noted under the appropriate required element indicating what has been requested and why. The team should not request the program to submit any material that does not already exist, unless it is something that should have originally been provided based on the evidence of compliance listings. An important note about meeting minutes, which is requested under many require elements. Should a program not have minutes of faculty meetings, annual retreats, etc. , the team should make every attempt to confirm the Self-Study Report in another manner, for example through interviews with faculty. Although the fact that there are no meeting minutes can be identified under the first relevant required element, it should not be repeated each time. To do so gives the appearance that meeting minutes are required by the required element rather than being one way to confirm the information provided. The same would be true of other evidence that is requested under multiple require elements. 14

Sample Visit Report Responses • The exam for PT 999 was not provided in the Self-Study Report. The program has been requested to submit it as additional materials with their response to the Visit Report. • On-site interviews with faculty confirmed that PT 999 is a 4 credit course, not 3 credits as was reported in the Curriculum Summary Form. • Although faculty meeting minutes were not available, interviews with core faculty confirmed the assessment process described in the Self-Study Report. IMPORTANT! Don’t keep repeating that requested evidence is not available Examples are provided here for how the Visit Report might address missing or incomplete information. 15

AVOID • Comments not related to the required elements – Core faculty are enthusiastic and dedicated. » But are they qualified? ! – Not all topics in the textbook are covered. – Faculty salaries are low. » Note: CAPTE never comments on salaries • Own Bias • Except if it is something the program says it does–in which case must make this clear! – Although the Self-Study Report indicated that interdisciplinary health care is a curricular thread, onsite review of curriculum materials and interviews with core faculty identified related learning experiences only in courses during the first year (PT 766 & PT 768). As was previously mentioned, the Visit Report comments must directly relate to the required elements. Examples are provided in this slide of comments for which there is no related required element. While the required elements require that faculty are qualified to teach what they teach, there is no required element requiring them to be enthusiastic and dedicated. This can be especially problematic when the Oral Report provides the positive assessment of dedicated faculty but the related required element indicates that they are not qualified. Sometimes unrelated comments are made because they relate to the team member’s expectations rather than the required element. Care needs to be taken to ensure that comments are not a reflection of your bias about how something should be done. Finally, CAPTE never comments on faculty salaries. The compliance issue is that they have a sufficient # of qualified faculty. The team should not address faculty salaries either during interviews or in the Visit Report 16

AVOID • Prescriptive Statements – should; needs to; must • Consultative Statements – The librarians provide individualized instruction upon request. This could be more efficient by having a formalized program early in the program. • Wording that appears to indicate you are searching for negatives – “no evidence of a problem was found” Prescriptive statements that indicate what a program must or should do need to be reworded to indicate what the problem is. CAPTE always leaves it up to the program to determine how to fix a problem. For example, if a lab space is too small for the number of students in the program, the program could get a larger lab. It could also hold multiple lab sections or decide to decrease the class size. It is institutional prerogative as to how it wishes to address the issues. Sometimes the team’s comments are provided in an attempt to be helpful to the program. Although consultative comments can be provided during the consultative session, they should not be included in the Visit Report. Finally, the report should not keep repeating that no evidence of a problem was found as it sends the message that the purpose of the on-site visit is to find problems. 17

AVOID • Conflicting statements – Review of the SSR indicated that the courses are well sequenced. Curriculum planning logic is not always evident in terms of sequential and integrated learning experiences. – Overview indicates a problem but response to related required element does not- or visa versa • Stating: Required Element appears to be met – Instead, please report on what they have and the quality of what they have The team should also avoid statements that conflict with other statements in the report. This is most often seen when the Overview is not consistent with comments found elsewhere in the report. In addition, the team should avoid stating that the required element appears to be met or is met. In part this is because only CAPTE can make that determination, but also because it provides no details that assists CAPTE in determining compliance. 18

Overview Section of the Visit Report • Includes – Brief history of program – Brief description of program » » » Type of institution Number of students Degree offered Number of faculty Basic curricular model The Overview section of the report is used for the Oral Report, which is given on the last day to core faculty and institutional administrators. The first part of the overview is intended to provide information that puts the program in context with its environment. 19

Overview • Summary of each major section of the Standards and Required Elements – If problem identified in related required element, needs to be in Overview/Oral Report The Overview also includes a summary statement for each major section of the Standards. As has been indicated previously, any major issue identified in the body of the report must also be commented on in the Overview. Likewise, when a problem is identified in the Overview, it must also be included under the appropriate required element. This is necessary to ensure that the Oral Report provides core faculty and administrators with an accurate understanding of the team’s findings. The last part of the Overview is the team’s assessment of the program meeting its mission. The next slide provides an example of this. Please note that while there is no statement regarding program strengths and weaknesses; such assessments can be included under the appropriate standard or Overview section. 20

Example of Assessment of Meeting Mission The following faculty, student and program outcomes support that the program is meeting its mission of graduates being prepared for practice in rural health care: » 90% graduation rate; » 98% licensure pass rate (3 year average; all takers); » 100% employment rate; 72% employment in rural areas; » Curricular content and clinical experiences that prepare graduates for practice in rural health environments; and » Faculty practice and scholarship related to rural health care. This slide provides an example of a team’s assessment of how the program meets its mission. Please note that this is only intended as an example based on what a program might indicate is its mission. CAPTE has no requirement for preparing graduates for rural health care. 21

Confidentiality CAPTE Considers the Visit Report to be confidential so Team cannot release information If programs release information, it must be complete and accurate Programs that release incorrect information will be publicly corrected by CAPTE considers the Visit Report to be confidential. Therefore, team members cannot release any information contained in the Report. Should the program or institution choose to release any information about the Visit Report, that information must be complete and accurate. If CAPTE determines that the institution or program has released any incorrect or misleading information about the Report, CAPTE reserves the right to publicly correct the incorrect or misleading information and to release the Report in its entirety. Notice of this policy is included in the report. 22

Summary • Purpose • Process • Path The Visit Report is an essential data source used by the Commission in its deliberations. Information has been provided to assist writing a report that will be useful to CAPTE in its review of a program. Details regarding responses to specific required elements will be addressed in a later module. You are now ready to proceed to the next module. 23