Euro Rec Certification Guidelines and Procedures 1

Euro. Rec Certification Guidelines and Procedures 1 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Generic Version Model Certification Guidelines & Procedures u Introduction u What is Certification? u Market Assessment u The Management of the Certification Process in Europe u Generic Certification Process u Generic Certification Documentation/Manual u Issues/Options in introducing Certification u Legal & Management u Organisation & Process Next steps u 2 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Introduction Generic Version, Model Certification Guidelines & Procedures u u Describes “what” to certify and “how” to perform the certification Documents preliminary model - generic guidelines and procedures Generic documentation - Benchmarking Manual for Certification and quality labelling of EHRs In the context of serving as a “template” for countries and/or organisations in Europe planning the certification and quality labelling : u 3 Outlines the process, organisational, management & legal issues to be addressed in order to introduce certification and quality Labelling of EHRs Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

What is EHR Certification? Critical component in ensuring the quality of EHR systems is the introduction of a Certification and Quality Labelling process Certification of EHRs can be defined as the procedure and action by which a body duly authorised and recognised as a legitimate provider of this service evaluates and certifies an Electronic Health Record system as meeting predetermined quality standards and certification criteria w Certification addresses functional quality of EHR systems considered as end products. w Certification or quality labelling processes can only evaluate the functionality of a software application. w The technical design (the way the function was implemented) cannot be subject of a certification or labelling exercise. The latter one remains the full responsibility and the exclusive authority of the software vendor. 4 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Market Assessment - Certification of EHRs Questionnaire administered through national Pro. Rec centres to 4 categories of Respondent Developers/Suppliers Policy Makers Purchasers System End-Users Country Respondent Belgium 9 Bulgaria 62 Denmark 2 Germany 6 Ireland 22 Romania 13 Slovenia 11 Developers/Suppliers 33 Policy Makers 31 Purchasers 28 System End-Users 33 Total 125 Total 125 5 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Market Assessment - Certification of EHRs 91% - 97% of all 4 respondent categories agree that the certification process is important to the development of EHRs Developers/ Policy Makers Purchasers System Suppliers End-users 88%-91% 94%-100% 54%-64% 73%-82% believe that EHR certification will result in one or more of these benefits: w w improve prospects of competing favourably in EHR market improve quality, safety & cost effectiveness of healthcare improve business process, including security, interoperability and data reliability allow the provision of a better service to patients w. Above suggests need for special focus on purchasers 6 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Market Assessment - Certification of EHRs Developers/Suppliers w w w 88% do not currently have certified EHR systems 66% plan to submit EHR system for certification 66% say their organisation prepared to pay for certification from a nationally/internationally recognised certification body Purchasers w 61% interested in buying a certified EHR system knowing the financial commitment this involves but w 7 only 36% say their organisation is prepared to pay extra for a certified system Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Market Assessment - Certification of EHRs All 4 respondent categories give a High rating to certification and other QREC products that will: w w w Define what an EHR product is expected to provide and how it will meet specifications (Range 81% - 93%) Significantly reduce investment and associated risks of implementation of EHR systems by specifying interoperability requirements (Range 70%-89%) High ratings in the main also given to certification and other QREC products that will: w w w Provide an independent, unbiased, professional and trustworthy quality assurance mechanism for EHR products Support industry co-operation by convergence towards known and new national standards and criteria, improving interoperability of EHR systems Improve the quality and safety of healthcare: w by assessing the quality and suitability of EHR product w by providing an inventory of resource w by educating vendors/users on requirements of successful EHRs 8 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Market Assessment - Certification of EHRs A significant majority of all 4 respondent categories say they/their organisation would be interested in: w Using the certification process service - (82%-90%) w Utilising a register of certified EHR systems - (90%-93%) w Utilising a register of criteria that EHRs need to fulfil in order to gain certification – (82%-93%) w Receiving advice on benchmarking processes formal testing of EHR w Products for certification and quality labelling – (86%-91%) w Having access to libraries of terminology underpinning certification and quality labelling – (81%-97%) 100% of Purchasers say their organisation would appreciate if: w suppliers/sellers of their EHR system had access to current up-to-date information regarding terminology underpinning certification w if seller had received advice on the testing of that system on the way to achieving certification w. A number of key issues now to be finalised for certification process 9 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

How Certification will be Managed Euro. Rec will act as a central repository of validated quality criteria and other relevant materials that can be used to harmonise European testing, certification & quality labeling and procurement specification for EHR systems Rather than impose particular certification models or specific criteria on any member country Euro. Rec will foster, via Pro. Rec centres and other channels, the progressive adoption of consistent and comparable approaches to EHR system certification and quality labelling The management of the certification process in Europe will operate through the Euro. Rec Institute and the National Certification Authorities and/or the National Pro. Rec Centres The certification process may be carried out by the National Certification Authority itself, or a mandated subcontractor, - National Pro. Rec centre or other agency 10 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

How Certification will be Managed The role of Euro. Rec - initially through the Q-Rec project w Build and develop the Certification process centrally w Distribute the Certification process to National Certification Authorities/Pro. Rec Centres w Assist and support National Certification Authorities/ Pro. Rec Centres in introduction and adoption w Providing guidance and education to all stakeholders in the market w Auditors of the process w Establish and Maintain Registers w Maintaining the standards and certification criteria The role of the National Certification Authorities/ Pro. Rec Centres w Taking the Certification process and adapting for local needs. Be the centre of certification for each country w Establish and Maintain Registers (based on Euro. Rec registers) w Maintaining the standards and certification criteria 11 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

The Generic Certification Process Generic certification process stages are: w Initial contact by supplier/developer/purchaser w Formal application w Application process Contract between Certification Authority and Applicant Certification Manual describing: Certification process Statement of criteria for Certification Documentation to be supplied by vendor Testing Schedule w Test/Retest – 2 levels Pre-test self-assessment External evaluation by C. A. w Pass/Fail w Certification Register Details certification status of each product Schedule of dates for retesting/recertification w Issue of Certificate/Seal - issued on successful completion of test w Publication 12 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Generic Certification Documentation/Manual includes specific forms and statement of requirements in regard to: w w w 13 Application form Contract Instruction /information /guidelines for the Certification process Application Test Pack Vendor Documentation Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Application Form Application form, details: w Supplier’s name, address, contact details w Product name, version, status (live. pre-live, development - if product is live, details of at least one site where it is in use – location and contact) w Requested test date/alternative date w Signature of director/company representative with appropriate authority 14 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Contract is completed between Certification Authority/Agency and Applicant Contract to cover the certification process, specifying Certification Authority/Agency and Vendors rights, accountability, responsibilities, limitations, confidentiality, freedom of information and waivers. 15 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Certification Manual Instruction /information /guidelines for the Certification process Details: w w w w w 16 Overview of certification process How to apply Documentation required Criteria to attain certification Overview of range of functional, operational, implementation, standards and user criteria used for tests Overview of tests and testing methodologies Approval process Retest and appeal process Use of Seals and Certification Fees Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Application/Test Pack Application /Test Pack contains: w w w Detailed description of certification process Detailed description of test process/methodology Documentation to be completed Supporting documentation required Detailed set of criteria to be utilised for testing Statement of Criteria Requirements for Certification regarding w w w 17 Functionality Interoperability Security Data Structures, Codes and Standards Test Scripts/Scenarios to test each of the criteria Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Vendor Documentation to be supplied by vendor: w w w w w Software/Hardware Maintenance. Agreement(s) Employment contracts and agreements Procedures document System documentation (including operating systems and database) Hardware/communications specifications User Manual Conformance statement Training manual and arrangements Software/Product Licence Escrow agreement w Licences from relevant Third party suppliers. 18 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Implementing the Certification Process A number of issues and have been identified. Options are proposed for consideration and decision by each national certification authority. These are categorised into: Legal & Management Issues/Options u u Organisation & Process Issues/Options 19 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Legal & Management Issues/Options in the Certification Process The major issues/options to be decided are: 1. Balance between Pan European vs. National based certification 2. Status of Certification/Accreditation Authority (C. A. ) - Government vs. Non-Government 3. Mandatory vs. Voluntary Approach 4. Supplier Participation in Evaluation 20 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

1. Pan European vs. National based certification Process issues/considerations comprise primarily the following: Considerations here include the following: w w National based approach likely to: w 21 Pan European based approach likely to: Joint Pan European/ National w w w w w Be more efficient Assure consistency Prove more acceptable to suppliers Raise National difficulties Be better supported at Country level Diminish consistency/efficiency Increase cost both to C. A. and suppliers Be opposed by suppliers European certification - generic criteria/functionality National Certification – specific national requirements Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Market Assessment Should certification be organised/provided on: A – a Pan-European basis B - a national basis C – a combination of A and B Suppliers/Developers Policy Makers Purchasers End-Users A 33% 13% 25% 42% B 12% 6% 11% 9% C 54% 74% 64% 46% 22 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

2. Certification Authority (C. A. ) - Government vs. Non-Government Considerations here include the following: w Government Body likely to: w w Be less acceptable to suppliers Be more independent w Non-Government Body likely to: w w w 23 Be more acceptable to suppliers Hold more monopolistic status Be dependent on suppliers Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Market Assessment Should certification be organised/provided by: A – a Government organisation B – a Non-Government organisation Suppliers/Developers Policy Makers Purchasers End-Users A 9% 52% 39% 33% B 91% 45% 61% 58% 24 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

3. Mandatory vs. Voluntary Approach Important to determine the approach Considerations include the following: w Mandatory approach likely to: w Give greater assurance to purchasers/end-users w Be welcomed by higher performing suppliers w Drive product improvements in lesser performing suppliers w Create potential difficulty for new entrants w 25 Voluntary approach likely to: w Be generally more welcomed by suppliers w Effectively produce the same participation outcome as mandatory approach w Create more amenable relational arrangement with suppliers Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Market Assessment Should certification be A Mandatory or B Optional? Suppliers/Developers Policy Makers Purchasers End-Users A 15% 52% 85% B 85% 45% 11% 12% 26 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

4. Supplier Participation in Evaluation Considerations here include the following: w Supplier participation likely to: w w w w Enhance supplier confidence Assure peer status Enhance supplier acceptance Raise questions on objectivity Potentially undermine business confidentiality Advantage larger/more established suppliers Supplier non-participation likely to: w w Assure greater objectivity Enhance procurer acceptance Improve confidence of and acceptance by funding authorities To raise arguments on feasibility of the requirements, correctness of the tests and professionalism of the process 27 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Market Assessment Should developer/supplier participate in certification process? Suppliers/Developers Policy Makers Purchasers End-User Agree 100% 93% 86% 73% 28 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Organisation & Process Issues/Options in the Certification Process The organisation and process issues/options to be decided are: 1. Audit vs. Self-Assessment / Surveyor Review Evaluation Approach 2 Audit/Surveyor Workforce 3. Quality Assurance vs. Quality Improvement Focus 4. Scoring / Rating Scheme 5. Certification Award 6. Evaluation Event For Certification 7. Scheme Review and Evaluation 29 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

1. Audit vs. Self-Assessment / Surveyor Review Evaluation Approach Considerations here include the following: w Audit Approach likely to: w w Be more rigid and extreme Lack nuance Potentially unfairly advantage larger/more established suppliers Be more objective w Self-Assessment /Surveyor Review Approach likely to: w w w 30 Be more rounded in outcome Avoid extremes Be better able to reflect potential of suppliers Be generally more acceptable to suppliers Potentially be more open to surveyor persuasion Potentially be less objective Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

2. Audit/Surveyor Workforce Important considerations here include: w w w w w 31 Competence / personal profile considerations Job specification Part-time (temporary) vs. full time (permanent) appointees Peer Status Credibility Calibre Selection Training Knowledge / skills maintenance Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

3. Quality Assurance vs. Quality Improvement Focus Considerations here include the following: w QA approach likely to: w w w Be more rigid Distance C. A. from procurers and suppliers Minimise value return of scheme in the context of its potential to develop procurer / supplier synergies w QI approach likely to: w w w 32 Add significant value for procurers/suppliers Encourage and benefit smaller / start up suppliers Optimise value return of scheme in the context of its potential to develop procurer / supplier synergies Enhance engagement between C. A. and procurers/ suppliers Be more resource demanding Further Euro. Rec’s mission objectives Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

4. Scoring / Rating Scheme Considerations here include the following: w Pure numeric scoring / rating vs. combined numerate / judgement scoring / rating w Standards / criteria weightings w Mandatory standard / criteria w Minimum score / rate requirements Decision on scoring/rating scheme will impact on final criteria definition 33 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

5. Certification Award: Pass / Fail vs. Graded Considerations here include the following: w Pass / fail approach likely to: w w Benefit established / large scale suppliers Militate against small scale suppliers Potentially inhibit market entry Improve certainty for procurers w Graded approach likely to: w w w 34 Benefit small scale suppliers Encourage / assist new entrant suppliers Create less certainty for procurers Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

6. Evaluation Event For Certification Considerations here include: w Necessity to develop a clear and unambiguous template setting out the process to be followed for the evaluation event w Imparting details to / training suppliers with respect to the process w Articulation of the process and mechanism by which the evaluation outcome is converted into a certification award w Need to provide for a formal and objective appeals mechanism with respect to a contested evaluation / certification outcome 35 Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

7. Scheme Review and Evaluation There is a need to incorporate provision in the scheme process for: w w 36 Scheme review (3/5 year cycle) Continuous scheme evaluation (i. e. to establish extent to which scheme is meeting its stated objectives / expectations) Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008

Next Steps Utilise the pilot implementations in Europe of a quality labelling and certification process for EHR systems in compliance with the “good practice requirements” elaborated by Euro. Rec Validate u u Model - generic guidelines and procedures Generic documentation - Benchmarking Manual for Certification and quality labelling of EHRs Determine and agree and – on a Country/European level u Options are that are proposed for consideration and decision by each national certification authority Implement on a Country and European level u 37 Quality labelling and certification process for EHR systems Euro. Rec-SDMI/Prorec. SI workshop "Quality Labelling and Certification of Electronic Health Record Systems in Europe” Portoroz, Slovenia, May 5 th 2008