- Количество слайдов: 42
Certification and Conformance Approaches for Canada’s EHR Elaine Sawatsky, Senior Management Consultant, Privacy & Security, Canada
Conformance: Canada’s EHR • Infoway Conformance Services in Canada • Supporting the Implementation of pan-Canadian Standards • Provincial/Territorial work • On the ground experiences
Conformance: Canada’s EHR • Sharing information requires a quality assessment of products to ensure interoperability, to reduce risk to patients, purchasers and the ‘system’. • Standards support sharing data and bringing it together • Improves comparability and harmonisation between products. • Certification: “a process of validation of products in a given context against specific validation criteria. ” • Certification criteria are – even regarding the same functionality – up to a certain degree similar but also different between jurisdictions in: » The way they are formulated & validated » The importance given to each of them • Certification should be unambiguous, fair and trustworthy
Conformance Services Vision • To support the successful implementation of pan-Canadian Standards within projects through the provision of conformance testing artefacts and tooling. – To improve the consistent implementation of pan. Canadian Standards – To reduce the effort required to implement pan. Canadian Standards – To reduce the cost of implementing pan-Canadian Standards • Strategic Intent of Conformance Services: – Ensure that there is a national service to support the vendor conformance and certification of i. EHR, public health and EMR solutions.
SK EHR Blueprint and Concepts
The need: Bridging The Gap The challenge is to bridge the gap that exists between the health informatician (modeler) and the clinical integration specialist (solution developer). Health Informatics Modeler Integration Specialist Standards Implementation, Conformance & Maintenance Standards Development Need Options Develop SFU in HR ation E r e Op Authoring Profiles Release Modeling Culture • Delivering Specifications Vocabulary Tooling Assessment Certification Change Helpdesk Project and Product Culture • Delivers Production Solutions 6
Conformance: Canada’s EHR
Conformance: Canada’s EHR Approach • There is recognition that there is a need for conformance testing to ensure alignment • The approach is founded on international standards • Prototype & Learn – conformance testing is not well understood yet • Build Methods & Processes with stakeholder input • Build tooling & leverage : • Standards – HL 7 v 3, LOINC, SNOMED, XDSI • Tooling • Component model – integration is key! • Sharing of testing artefacts – huge value to stakeholders
Conformance: Canada’s EHR Key Messages • There is recognition that we all need to do end to end and integrated testing • We are committed to leveraging work of others • We are making good progress in developing processes and tooling to assist our jurisdictions in implementing “Conformant” solutions
Conformance Continuum Conformance includes: – Message content and structure – Vocabulary and terminology – Interoperability profiles and message sequencing – Sending and receiving applications and intermediaries Conformance Testing includes: – Functionality of application – Basic privacy and security services related to the functionality – Workflow (to an optimal level determined by conformance continuum) 10
Our Approach • First, develop Profiles for all pan-Canadian standards • Develop a process & mechanisms linking the pan-Canadian Profiles through to conformance test • Develop, deploy, continuously improve “quick-hit” tools • Provide business and technical support for Investment Projects to set up and execute conformance testing • Document & refine requirements for long-term solution • Harvest artefacts for sharing with other jurisdictions • Rationalize jurisdiction artefacts to create pan-Canadian versions • Establish a pan-Canadian Working Group of early participant jurisdictions and their vendor partners to provide guidance and assistance in the development of Conformance Testing artefacts
Our Approach Change Management Scope of Part 4 Licenses Needs Identification & Business Definition Tools and Templates Options Research and Analysis Standards Selection Strategy Standards Development Maintenance Pan-Canadian Standards Knowledge Management Stable for Use Conformance Test Pilot Evaluation Liaison Final Approval Implementation Training Education Promotion & Communication Adoption Strategies
Conformance: Canada’s EHR Benefits: • Financial savings through reduced duplication of efforts in test authoring • Provinces leverage other provinces’ Business Rules • Financial savings though tools based test message development • Provinces use Infoway tools and other provinces’ messages • Breadth of testing improved through increased quantity and quality of test cases • Provinces are committed to sharing test artefacts.
Conformance: Canada’s EHR Benefits: • Depth of testing improved through consistent test rules and tooling • Provinces’ use of tooling has improved UAT testing Too early to accrue benefits • Quality and consistency of conformance testing improved between Investment Projects • Too early to accrue benefits • Improved consistency in Investment Project implementations of pan-Canadian standards • Too early to accrue benefits
Benefits Providing Implementation support for the development of Test Authoring and Test Execution will enable jurisdictions to leverage and share Conformance Profiles and testing artefacts, leading to: – Improved quality of testing – Financial savings through reduced duplication of efforts – Breadth of testing improved through increased quantity and quality of test cases – Improved quality and consistency of conformance testing – Financial savings though tools based test message development – Depth of testing improved through consistent test rules and tooling – Improved consistency in implementations of pan-Canadian standards
Conformance: Canada’s EHR Conformance Services Overview • Main elements influencing content of test criteria: Legal, national, cultural (business and social) • Sophistication of the industry area • Environment • Certification is the process • Conformance is the result • Must be good enough, not best or everyone will fail • Area subject matter experts – what will the medical professional need?
Overview of Conformance Services Test Authoring Services • Business Rule Management • Conformance Profile Management • Vocabulary Maintenance • Test Case Authoring • Test Data Management • Message Instance Authoring Test Execution Services • Conformance self assessment & evaluation • Conformance independent verification & validation • Conformance certification Technology • Test Authoring Tools ØRepository, Ø remote user access Øshared authentication • Test Execution Tools ØConformance Validation ØTest Case Validation ØSelf-test, monitoring • Web presence for jurisdictions and vendors ØRegistration, Øartifacts library, Øtest results Stakeholder Engagement • Governance • Collaboration on Test Authoring components • Collaboration on supporting tools development and implementation • Registry of vendor systems & certification status • Registry of implementation projects • Ongoing liaison/communication with jurisdictions and vendors • Promotion and Education around Conformance Services 17
Strategic Direction Analysis Project • Project Objective: – Develop a preferred option for the future delivery of Conformance Services. • Define and identify existing conformance services models; • Develop a shared vision for the delivery of Conformance Services for the Canadian healthcare sector • Develop conformance services delivery options for the Canadian healthcare sector; and detail the preferred option. • Defined Strategic Intent of Conformance Services: – “Ensure that there is a national service to support the vendor conformance and certification of i. EHR, public health and EMR solutions. ” RFP Release February 15 ‘ 08 Project Start March 15 ‘ 08 Draft recommendations June 30 ‘ 08 Final Recommendations Available August 30 ‘ 08 18
Investment Project Support Overview • Project Objective: – Provide immediate Standards Implementation Support for Test Authoring & Test Execution to key Infoway Investment projects • Project Approach: – Develop a process & mechanisms linking the pan-Canadian Profiles through to conformance test – Develop, deploy, continuously improve “quick-hit” tools – Rollout to qualified projects (if proof of concept successful) – Harvest artefacts for sharing with other jurisdictions – Provide lessons learned and input into CS Strategic Direction Project – Provide transition and support to recommended Strategic Direction Project Kick-off November 1 ‘ 07 Proof of concept (2 -3 projects) March 31 ’ 08 Proposed Project Extension: Limited rollout (4 -6 projects) September ‘ 08 19
Investment Project Support Activities • The specific services that Infoway is offering to assist projects • Assistance in developing test strategies – Sort out overlap between conformance testing vs. UAT • Education on development of business rules & test cases • Technical education on development of message instances and test data • Sharing of testing artefacts • Quality review of artefacts developed by investment projects • Full set of tools to assist getting to Conformance Testing • Assistance to jurisdiction in rationalizing and integrating their testing tools with Infoway conformance tools • Harvesting of testing artefacts for sharing with jurisdictions • Sharing of Lessons Learned • Facilitating inter-jurisdiction collaboration on conformance testing 20
Engagement Process to Conformance 00 Introduction Workshop 1 Test case Examples End to End Conformance Process Workshop (2 -3 days) Harvested artefacts End to End Test Authoring Examples Tools 2 Jurisdiction End-to-End Test Case Development (10%) (15 -20 days) Jurisdiction Test Artefacts 33 Quality Review Harvest artefacts (2 -3 days) End to End Test Execution Examples 4 Jurisdiction End-to-End Test Case Development (20 -40 days) Jurisdiction Test Authoring Artefacts Jurisdiction 5 Conformance Pilot Testing, Quality Review Harvesting artefacts, Lessons learned (3 -5 days) Jurisdiction still need to complete, but ready to perform Conformance Testing Ongoing Harvesting 21
Conformance Constructs – End-to-End Process Standards Artefacts - HL 7 V 3 - “Quick-hit” tools Jurisdiction Input Best Practice - “Existing” tools Conformance Profiles Business & Technical Rules Implementation Guides Test Data Test Case Authoring Instance Editor (Messages) Test Cases Vocabulary Data HL 7 Test Harness External Application 22
Conformance: Canada’s EHR Major Tooling Components • Business Rules Manager- Maintains Business Rules • Conformance Profile Manager - browse pan-Canadian Conformance Profiles • Implementation Conformance Profile Manager - browse profiles and create/maintain ICPs • Static Model Constraint Tool - captures changes to models, and traceability information • Instance Editor - creates Conformance Test Scenario Files, Message Fragments and Parameter Files as XML output. • Test Case Author - Maintains Test Cases • Test Authoring Manager - The main user interface • Test Execution Manager - links the Test Authoring Manager to control the test execution • Test Authoring Reports Manager - run reports
Conformance: Canada’s EHR Conformance Test Execution Scenarios There are different ways to perform Conformance Test Execution. This process uses the p. CCT Test Execution manager and / or the HL 7 Test Harness to allow the user to test both POS and i. EHR systems as “real” or “simulated” applications under test. 1. 2. 3. 4. Scenario 001 - Real POS with Simulated i. EHR Application Scenario 002 - Simulated POS with Simulated i. EHR Application Scenario 003 - Simulated POS with Real i. EHR Application Scenario 004 - Real POS with Real i. EHR Application
Lessons Learned – Considerations for AB • Testing artefact reuse is seen to be a big win • One province has 100 business rules & 300 test cases – target is 500 -700 test cases • Another target will be significantly higher (~2, 500 test cases) • Others will leverage oand likely add more • Toolset are not all or nothing • One province using blend of Infoway tools and home grown • Uses Infoway tools and processes first to perform testing then conformance testing of Po. S • One province using Infoway tools for go forward & Po. S testing • Specific initiatives will likely use blend • Infoway is investing in tools so jurisdictions don’t have to!
Conformance: Canada’s EHR Tooling Vision • Create a suite of integrated Conformance Test tools based on Open Source principles and accepted pan-Canadian standards that allows jurisdictions, and vendors to perform conformance testing • Includes: – Well documented processes and procedures – A repository of structured conformance profiles, test cases, test data and business rules that can be reused – A suite of re-usable conformance testing solution components able to easily exchange structured data between each component and projects – An Enhanced Web Based Presence to allow remote administration, self registration, shared resources management, self testing / monitoring and publication of results
Lessons Learned – Considerations for AB • Many jurisdiction projects have left UAT & Conformance Testing until very late in their project lifecycle • There is not a clear understanding of the continuum of conformance testing • Profile Business rules Test cases Messaging Conformance test • Traceability is key to conformance test • Profile Business rules Test cases Messaging Conformance test • Using Profiles to help them define user roles • What does a physician do vs. project focus • Tools can and are providing immediate benefit to jurisdiction projects for publishing • Consolidated view of all business rules • Publication of specification for vendor 27
Conformance Profiles Pan-Canadian Conformance Profile (p. CCP) Sask #47 + BC Nfld Extension p. CCP Constraint Implementation Conformance Profile (ICP) Sask #84 + BC Nfld BC Extensions #84 BC Constraint Likely Initial State Example: BC ICP #84 Library of Conformance Profiles Desired State
Conformance: Canada’s EHR Tooling for Test Authoring Components (TAC) Use prototyping methodology to conduct staged development Test with jurisdictional review and input Document processes, requirements and changes Garner feedback and experience to inform the refinement of detailed specifications for long-term open source tooling development – Address immediate jurisdictional needs & provide tangible benefits • All test artefacts exportable • Work with jurisdictions to rationalize and integrate tooling – Align test cases, business rules across jurisdictions – Promote early sharing and leveraging of work by jurisdictions – –
Conformance: Canada’s EHR Tooling for Test Execution – Leverage an existing open source test harness (previous Infoway investment), adding ability for jurisdictions to easily configure to reflect local technology solutions such as transport and security – Leverage Eclipse Instance Editor for message and vocabulary creation – Use data exported from test authoring tools • Key is to ensure tools export into a format that can be consumed by HL 7 Test Harness • Invest in documenting processes, requirements, integration – Rationalize and integrate tools – Develop guidelines for local Test Execution prototyping environments including a reference implementation – Develop processes to ensure data integrity between different groups of users during prototyping phase
Conformance: Canada’s EHR It’s not easy: • In order to get to a conformance test, you need to start with the Profile, know what the business rules are and create test data and message instances that will then test a scenario • Once you have done this for every business and technical scenario, only then can you think about performing a conformance test • By our simple calculations, this will be thousands and thousands of test cases!
Conformance Services Activity in Canada Panorama 0 0, 1 0 – Step 0 Engagement 1 – Step 1 Workshop 2 – Step 2 Activities 3 – Step 3 Workshop 4 – Step 4 Activities 5 – Step 5 Workshop 0, 1 0 0, 1, 2, 3 4, 5 0 0, 1 0 0 Planned Scheduled/Underway Completed 13 -Mar-08 32
Conformance: Canada’s EHR • Provinces have done functional Certification for EMRs specifically. • Other certification means defining differences in function, scope, perspective (e. g. Social interaction between users, staff and patients), • Legal requirements are different between provinces, policies are different: Infoway cannot set policy but provinces must • Provinces must work in collaboration with Health Authorities and professional bodies. • Conformance criteria will be different from one business environment to another.
Conformance: Canada’s EHR • Work is based on existing provincial models – Establish business rules to support policy set by Ministries of Health, College of Physicians and Surgeons and College of Pharmacists – Publish compliance specifications that include all the transactions needed at the point-of-service (e. g. pharmacy claims, pharmacy prescriptions, Client Registry, Provider Registry, Common, Lab, Immunization, etc. ) – Provide support to vendors during their development – Conduct compliance testing of vendor software once they pass a pre-test – Support change agents (chain stores, vendors, etc. ) during their training and deployment activities
Conformance: Canada’s EHR POS Software Compliance Principles • POS systems – – Must enhance patient safety & Support clinician workflow Must be secure and ensure privacy of patient medical records Must support interoperability standards Are integral in creating and maintaining quality data for use throughout the health care system • POS vendors must be able to differentiate their products • Compliance documents – Are referred to in some provinces legislation – Communicate what rules need to be followed, not how the rules are implemented in vendor software – Provide one place to house Ministry policy, College rules and application capabilities – Provide a level playing field for all vendors regardless of size
Conformance: Canada’s EHR POS Conformance Specifications • Define Business rules – Based on existing business rules from compliance documents and input from Clinical Working Group – Identify items that need further clarification • Develop Compliance specifications – Based on existing compliance documents and published message specifications from Infoway – Builds on Infoway Conformance Profile definitions – Leverage, where possible, work from other jurisdictions • Materials include: – – POS Software Vendor Compliance specifications; Vendor registration and development support processes; Compliance testing process; Process for approving vendors with compliant software • Stakeholders included in joint review of material prior to publishing
Conformance: Canada’s EHR Compliance – Lessons Learned • It takes time for the vendors to develop code to the level that warrants compliance testing • Vendors may present software that was not stable enough to warrant compliance testing • Vendors can require multiple attempts at compliance testing • Thorough compliance testing takes time, especially on the very first test of new software or a major release change • Good rules are needed on what constitutes an initial test and a follow-up test for each version of the vendor software • Good rules are needed on when penalty and charges are applied to the test
Conformance: Canada’s EHR • Sample tests were provided for vendors • Focus needed to ensure that the vendors could prove they ran the tests and the tests worked before scheduling a compliance testing • Need mechanisms to ensure compliance team provided consistent responses to all vendors throughout the compliance support process • Good rules are needed around mandatory training that a vendor must provide e. g. training plans, training outcomes, etc. • Good test facilities are needed for vendors and for compliance testing • Multiple training environments are needed for each vendor • Good tools are needed for the compliance testers that assist them in developing the test cases and support them during the testing
Conformance: Canada’s EHR • Clinical guidance was critical during vendor development and compliance testing to clarify options and workflow – Requires ongoing clinical involvement • Specifications, clinical guidance and support needs to include details • Compliance team requires good communication and negotiation skills and need to be able to stand their ground under pressure • Compliance documentation needs to be targeted to specific audiences and also clearly identify elements common across audiences • A tool that enables easy linking and/or packaging of modules is highly desirable. Documents that are difficult to maintain don’t get maintained • Business context for the compliance material is just as important as the rules themselves
Conformance: Canada’s EHR Conformance includes: – – Message content and structure Vocabulary and terminology Interoperability profiles and message sequencing Sending and receiving applications and intermediaries Conformance Testing includes: – Functionality of application – Basic privacy and security services related to the functionality – Workflow (to an optimal level determined by conformance continuum)
Contact Standards Collaborative Information Desk Bureau d'information de l'Unité collaborative de normalisation d’Inforoute 1 -877 -595 -3417 or 416 -595 -3417 [email protected] ca or [email protected] ca Elaine. [email protected] net Privacy and Security, SME