Training in Portugal 1 EUROSAI IT Working Group

Training in Portugal (1) EUROSAI IT Working Group - Michel Huissoud 1

Project: « Design a self-assessment tool for SAI’s based on » EUROSAI IT Working Group - Michel Huissoud 2

What we want ü Improve IT audit (methodology and ü practical approach with Cobi. T) IT Governance (with self-assessment) by the SAIs EUROSAI IT Working Group - Michel Huissoud 3

1. Genesis of a success story the Hague, 1 October 2002 EUROSAI IT Working Group - Michel Huissoud 4

Our mandate The objective of this project is to design and pilot a selfassessment tool for all SAIs. It is based on 'Cobi. T', which is a governance (and audit) framework for the domain of information technology. The self-assessment tool we are developing should enable us to measure the maturity of the IT control of our own offices. EUROSAI IT Working Group - Michel Huissoud 5

2. Why. . . a self-assessment ? . . . of Information Technologies ? . . . based on Cobi. T ? EUROSAI IT Working Group - Michel Huissoud 6

Why a self-assessment? l It allows « proximity » . The evaluation is carried out by the people : – – l l who know the subject who are interested in solving the problems It is confidential. The organization is in control of the results of the evaluation and their distribution. Self-assessment is not an audit. The extern moderation encourages the people to speak freely. EUROSAI IT Working Group - Michel Huissoud 7

Why IT? l l As in every organisation or company, it is in the interest of the SAI to maintain control of its IT system. The latter is of fundamental importance, whether this has to do with managing dossiers, planning auditor tasks, communication or knowledge management. Issues concerning communication and defining the roles between the different partners represent one of the main challenges in IT governance. The SAIs, together with other enterprises, need better communication between the sponsors and the IT specialists. EUROSAI IT Working Group - Michel Huissoud 8

like the other organisations. . . v we lose time because of system shutdowns. . . v we type the same information in different systems two or three times. . . v we develop projects which don’t meet expectations. . . v we manage expensive service providers. . . v we use IT without enough training. . . EUROSAI IT Working Group - Michel Huissoud 9

Why “based on Cobi. T”? Cobi. T is a well accepted standard ü Cobit can be downloaded free from www. isaca. org ü Cobi. T is also available in French www. afai. asso. fr , German www. isaca. ch and Spanish www. isaca. org ü l but our group wanted to be sure that Cobi. T is the best choice. . . EUROSAI IT Working Group - Michel Huissoud 10

What have we done? v Studies of other tools: v ISO 9001 v European Foundation for Quality Management (EFQM) Excellence Model v ITIL / Process Maturity Self-Assessment & Action Plan v CMM Capability Maturity Model v Common Assessment Framework (CAF), result of the cooperation among the EU Ministers responsible for Public Administration v Contact with specialists: v Philips, The Netherlands v Swisslife, Switzerland v Prof. W. van Grembergen (University of Antwerp, Belgium) . . our research confirmed the legitimacy of choosing Cobi. T EUROSAI IT Working Group - Michel Huissoud 11

3. Looking for the gaps and use Cobi. T as a bridge!. . . the problem is always by the interface l Management IT l IT Audit l IT audit Financial audit EUROSAI IT Working Group - Michel Huissoud 12

COBIT includes 36 national and international standards Codes of conduct issued by Council of Europe, OECD, ISACA, etc. l Qualification criteria for IT systems and processes: ITSEC, TCSEC, ISO 9000, SPICE, Tick. IT, Common Criteria, etc. l Professional standards in internal control and auditing: COSO Report, IFAC, AICPA, IIA, ISACA, PCIE, GAO standards, etc. l EUROSAI IT Working Group - Michel Huissoud Industry practices and requirements from industry forums (ESF, I 4) and government-sponsored platforms (IBAG, NIST, DTI), etc. l Technical standards from ISO, EDIFACT, etc. l Emerging industry-specific requirements such as from banking, electronic commerce and IT manufacturing l 13

the three most important sources: qualification standards (ISO, SPICE, ITIL, . . . ) IT security standards (ITSEC, BS 7799, etc. . . ) EUROSAI IT Working Group - Michel Huissoud audit standards (IFAC, IIA, COSO, GAO, . . . ) 14

with Cobi. T, they can Control OBjectives for Information communicate together!. . . and Related Technology EUROSAI IT Working Group - Michel Huissoud 15

Service level for example Control Objectives • The service level agreement should cover at least the following aspects: availability, reliability, performance, capacity for growth, levels of support provided to users, continuity planning, security, minimum acceptable level of satisfactorily delivered system functionality, restrictions (limits on the amount of work), service charges, central print facilities (availability), central print distribution and change procedures. (. . . ) EUROSAI IT Working Group - Michel Huissoud Management Guideline Key Performance Indicators • Time lag of resolution of a service level change request • Time lag to resolve a service level issue • Number of times that root cause analysis of service level procedure and subsequent resolution is completed within required period • Significance of amount of additional funding needed to deliver the defined service level (. . . ) Audit Guideline • Considering whether recourse process is identified for non-performance • Testing that historical performance against prior service improvement commitments is tracked (. . . ) 16

or Information Architecture Management Guideline Key Goal Indicators • (. . . ) • Reduction of data redundancy • Increased interoperability between systems and applications (. . . ) Control Objectives • Data Classification Scheme A general classification framework should be established with regard to placement of data in information classes (i. e. , security categories) as well as allocation of ownership. The access rules for the classes should be appropriately defined. (. . . ) EUROSAI IT Working Group - Michel Huissoud Audit Guideline • Considering whether a medium is used to distribute the data dictionary to ensure that it is accessible to development areas and that changes are reflected immediately • Identifying data items where ownership is not clearly and/or appropriately defined. (. . . ) 17

or manage the operations Control Objectives • Job Scheduling IT management should ensure that the continuous scheduling of jobs, processes and tasks is organised into the most efficient sequence, (. . . ). The initial schedules as well as changes to these schedules should be appropriately authorised. • Remote Operations For remote operations, specific procedures should ensure that the connection and disconnection of the links to the remote site(s) are defined and implemented. . (. . . ) EUROSAI IT Working Group - Michel Huissoud Management Guideline Critical Success Factors • Changes to job schedules are strictly controlled • There are strict acceptance procedures for new job schedules, including documentation delivered • Clear and concise detection, inspection and escalation procedures are established(. . . ) Audit Guideline • Review of a sample of limited IT operations and determining whether they meet policy and procedures requirements. • Identifying a sample of abnormal ends (ABENDS) for jobs and determining resolution of problems which occurred. (. . . ) 18

Cobi. T is special ü ü ü Quality Cost Delivery ü ü ü Confidentiality Integrity Availability ü Effectiveness and Efficiency of operations or projects Reliability of Information Compliance with laws and regulations ü ü . . this framework goes further than the other ones! EUROSAI IT Working Group - Michel Huissoud 19

Navigation in Cobi. T: How can you select the right process? « availability » for example EUROSAI IT Working Group - Michel Huissoud 20

or « human ressources » ? EUROSAI IT Working Group - Michel Huissoud 21

Warm up… ü Who doesn't know what the EUROSAI IT Working Group is? ü Who doesn't know what Cobi. T is? ü Who doesn't know what self-assessment is? ü Is self-assessment a questionnaire or an interview method? ü Are we looking for problems in efficiency or in security? EUROSAI IT Working Group - Michel Huissoud 22

4. Our method EUROSAI IT Working Group - Michel Huissoud 23

How do we proceed? Documentation to study will be provided on Cobi. T, selfassessment, etc. . ) The instructor will provide more information, the structure of your business will be discussed and then forms will be filled in 2 weeks before EUROSAI IT Working Group - Michel Huissoud The instructor will consolidate the results and a discussion of the results will follow Workshop An action plan for the future will be prepared together and the exercise will then be evaluated The results of the workshop are then presented to the top management of the SAI Post ws 24

The problem has 2 dimensions Planing ans organisation acquisition and implementation Etc… business process 1 business process 2 Etc… Etc. . . AI 2 AI 1 PO 2 business process 3 business process 4 PO 1 first dimension = business second dimension = IT business process 5 business process 6 business process 7 EUROSAI IT Working Group - Michel Huissoud 25

the first form identify the business process EUROSAI IT Working Group - Michel Huissoud 26

What do we understand by “business process”? examples: l. Audit Risk Management l. Organise the missions l. Analyse the data l. Test the IT by the IT- Audit l. Report the results to the auditee l. Track the implementation of the recommendations l. Manage the knowledge EUROSAI IT Working Group - Michel Huissoud l Manage finances and l l l human resources Administer and archive the dossiers Publish the results of the audits Communicate Automated data inputs Automated relations between different audits 27

EUROSAI IT Working Group - Michel Huissoud 28

then, we evaluate the importance and the quality of the current IT systems Importance of the IT systems? EUROSAI IT Working Group - Michel Huissoud Quality of the IT systems? 29

the second form COBIT's Domains and Processes Planning and Organisation PO 1 PO 2 PO 3 PO 4 Which business processes (see in Form 1) are affected by this problem (especially if level = 0 o 1)? Importance Define a Strategic IT Plan Quality of of Define the information architecture. IT the Determine the technological direction systems? . . . Define the IT Organisation and Relationships EUROSAI IT Working Group - Michel Huissoud PO 5 non-existent (0) initial / ad hoc (1) repeatable but intuitive (2) defined process (3) managed and measurable (4) optimised (5) Cobi. T Form 2: What is the maturity level of the Maturity level of IT-processes? the process very important (2) important (1) not important (0) not sure (0) Importance of the process Manage the IT investment 30

6 maturity levels EUROSAI IT Working Group - Michel Huissoud 31

Maturity model? Example: “DS 04 Ensure continuous service” 0 Non-existent. There is no understanding of the risks, vulnerabilities and threats to IT operations or the impact of loss of IT services to the business. Service continuity is not considered as needing management attention. 5 Optimised Integrated continuous service processes are proactive, self-adjusting, automated and self-analytical and take into account benchmarking and best external practices. Continuous service plans and business continuity plans are integrated, aligned and routinely maintained. Buy-in for continuous service needs is secured from vendors and major suppliers. Global testing occurs and test results are feed back as part of the maintenance process. Continuous service cost effectiveness is optimized through innovation and integration. Gathering and analysis of data is used to identify opportunities for improvement. Redundancy practices and continuous service planning are fully aligned. Management does not allow single points of failure and provides support EUROSAI IT Working Group - Michel Huissoud 32 for their remedy. Escalation practices are understood and thoroughly

Example 2: “PO 10 Manage projects” 0 Non-existent. Project management techniques are not used and the organization does not consider business impacts associated with project mismanagement and development project failures. 5 Optimised A proven, full life-cycle project methodology is implemented and enforced, and is integrated into the culture of the entire organization. An on-going program to identify and institutionalize best practices has been implemented. There is strong and active project support from senior management sponsors as well as stakeholders. IT management has implemented a project organization structure with documented roles, responsibilities and staff performance criteria. A long term IT resources strategy is defined to support development and operational outsourcing decisions. An integrated program management office is responsible for projects from inception to post implementation. The program management office is under the management of the business units and requisitions and directs IT resources to complete projects. Organizationwide planning of projects ensures that user and IT resources are best EUROSAI IT Working Group - Michel Huissoud 33 utilized to support strategic initiatives.

matching the results. . . Where are the reasons for the dissatisfaction? EUROSAI IT Working Group - Michel Huissoud What impacts do the IT problems have? 34

5. what you get ü gaps analysis ü a good discussion ! ü action plan EUROSAI IT Working Group - Michel Huissoud 35

For example: satisfaction with the IT support of the business processes B 10 confidential. . B 12 confidential. . B 6 confidential. . . B 5 confidential. . . B 3 confidential. . . B 4 confidential. . . B 1 confidential. . . B 9 confidential. . . B 7 confidential. . . B 2 confidential. . . B 11 confidential. . . B 8 confidential. . . EUROSAI IT Working Group - Michel Huissoud 2. 29 2. 00 1. 75 1. 38 1. 33 1. 29 1. 00 0. 86 0. 83 0. 60 0. 00 36

identification of the problems (business point of view) 6 5 4 What is the quality of the current IT systems ? 3 What is the importance of the future IT systems ? 2 1 B 13 B 12 B 11 B 10 B 9 B 8 B 7 B 6 B 5 B 4 B 3 B 2 B 1 0 process EUROSAI IT Working Group - Michel Huissoud 37

identifying the problems (IT point of view) Importance of the process ? Maturity level of the process ? 6 5 4 3 2 1 process EUROSAI IT Working Group - Michel Huissoud 38 M 4 M 3 M 2 M 1 DS 13 DS 12 DS 11 DS 10 DS 9 DS 8 DS 7 DS 6 DS 5 DS 4 DS 3 DS 2 DS 1 AI 6 AI 5 AI 4 AI 3 AI 2 AI 1 PO 10 PO 9 PO 8 PO 7 PO 6 PO 5 PO 4 PO 3 PO 2 PO 1 0

An action plan EUROSAI IT Working Group - Michel Huissoud 39

and perhaps in the future: a benchmarking Big SAIs Middle SAIs Small SAIs EUROSAI IT Working Group - Michel Huissoud 40

A reasonable time management (first day) 14. 00 Start of the workshop 15. 00 Identify the business processes 15. 30 Coffee break (moderator) Adaptation of the form 1 and print them 16. 00 Fill form 1 16. 15 Presentation Cobi. T 17. 15 Select the most important IT processes 18. 00 Fill form 2 18. 30 End of the first day Then, put the results in your EXCEL sheet, prepare the presentation of the results and the discussion of tomorrow… EUROSAI IT Working Group - Michel Huissoud 41

A reasonable time management (second day) 09. 00 09. 30 10. 00 10. 15 10. 45 11. 30 12. 30 15. 00 Presentation of the results Discussion (validation of the results, looking for consensus) Listing the most important problems and strengths Coffee break Prepare an action plan Fill the evaluation forms Finalization of the action plan Discussion and end of the workshop Preparation of the final presentation Presentation and discussion with the head of the SAI Write the evaluation report! EUROSAI IT Working Group - Michel Huissoud 42

We will now focus on the following points Get the right Documentation to persons! study will be provided on Cobi. T, selfassessment, etc. . ) Identify the The instructor will provide processes! more information, the structure of your business will be discussed and then forms will be filled in Get a good The instructor will action consolidate An action plan the results for the future plan! Use the will be and a The results discussion of of the prepared the results workshop EXCEL sheet together and will follow are then the exercise presented to will then be correctly! the top evaluated managemen t of the SAI 2 weeks before EUROSAI IT Working Group - Michel Huissoud Ask the right questions! Workshop Post ws 43