NASA are owners of this picture produced by

NASA are owners of this picture produced by The Visible Earth atsec information security © 2007 CC in the Real World Fiona Pattinson When technologies based on the state of the art in a rapidly maturing discipline are put into practice some “real world” issues surface for the practitioners to cope with. 8 th ICCC: CC in the Real World 1

In the U. S. GAO report: (CC consumers) © 2007 atsec information security • March 2006 – difficulty in matching agencies’ needs with the availability of NIAP-evaluated products – vendors’ lack of awareness regarding the evaluation process – a reduction in the number of validators to certify products – a lack of performance measures and difficulty in documenting the effectiveness of the NIAP process http: //www. gao. gov/new. items/d 06392. pdf 8 th ICCC: CC in the Real World 2

Vendors Issues © 2007 atsec information security • May 2007 GCN: “Symantec: Common Criteria is bad for you” – CC is a closed standard development – Does not address areas where vulnerabilities occur – Starts late in the development process – It is difficult to produce a complete ST at the beginning of a project – Assurance vs Convenience balance http: //www. gcn. com/online/vol 1_no 1/44205 -1. html 8 th ICCC: CC in the Real World 3

General Issues with CC… • Timeliness – Takes too long • Cost – Costs too much • Validity/Scope of certification © 2007 atsec information security – Too narrow • Mutual Recognition – limited • Composition – Was not addressed • Comprehensibility – Too hard to cope with David Brewer: Is the Common Criteria the only way? ”: https: //www. ipa. go. jp/event/iccc 2005/pdf/B 2 -10. pdf 8 th ICCC: CC in the Real World 4

“The product I want to buy is not on the list. ” © 2007 atsec information security • Over the last few years: – greater emphasis in understanding the risks presented by a greater number and variety of IT products – Emphasis on certification as a solution to help assist risk based decisions • Scalability – Can the CC schemes handle the needed volume? 8 th ICCC: CC in the Real World 5

Rise in number of evaluations ITSEC Evaluations* (*From “Is the Common Criteria the only

“I don’t know where to start with CC. ” © 2007 atsec information security • This *is* a FAQ from new vendors & teams – Most national schemes publish good beginner info – So does the CC Portal – Many labs provide basic information (and consultancy!) – “I’ve heard that CC is expensive, takes forever, needs lots of specialised documents, is hard work. . etc” 'Does Common Criteria need marketing? ', ICCC 2006, Tokyo. 8 th ICCC: CC in the Real World 7

“It costs HOW much? ! ” • Effects © 2007 atsec information security – Some claim it is used as a trade control measure – Prices smaller companies out of the market • Compare – FIPS 140 -2: Smaller scope, smaller cost • Consultation and testing components – ISMS: • Consultation and auditing costs 8 th ICCC: CC in the Real World 8

© 2007 atsec information security “Our products are released every 6 months, but we were told that our CC evaluation would take 9 -18 months. ” • This may be true for a first time evaluation • There’s a learning curve for product teams using CC • As a product and dev team’s “evaluation career” progress it is possible to have certification coincide to within a few weeks of the end of the development cycle 8 th ICCC: CC in the Real World 9

© 2007 atsec information security Example: Linux 8 th ICCC: CC in the Real

How can I make CC easier next time? Improving development process with assurance provision in mind © 2007 atsec information security • CC is not a direct process or product improvement standard • If an organization has assurance provision as part of a product-line strategy then security process and product improvement are vital – Informal, or formal (SSE: CMM ISO/IEC 21827, CMMI, Tick. IT, even ISO/IEC 9001 and 90003) – Processes may even address the needs of particular standards • Secure System Design – Design to provide assurance 8 th ICCC: CC in the Real World 11

you didn’t evaluate anything I use or that’s remotely useful. ” • Unrealistic boundaries are specified in order to: – Reduce evaluation effort (time/cost) – Avoid problem areas • CC is deliberately flexible, and does not prevent ST writers defining unrealistic boundaries • The specification of inappropriate boundaries bring CC into disrepute as consumers are left with claims that are meaningless in their objective of assessing their risk © 2007 atsec information security “But 8 th ICCC: CC in the Real World 12

• In practice this can be alleviated by: – Scheme policies – Specification and adoption of appropriate PPs – Ethical laboratories and consultants advise their customers appropriately – Specifications e. g. FIPS 140 -2 • Boundary is more rigorous but still mis-used sometimes – ISMS(& QMS) • (At least for QMS) the specifications for scope have been mis -used and caused disrepute. © 2007 atsec information security • Compare with : 8 th ICCC: CC in the Real World 13

“It doesn’t evaluate a whole product” • Security functionality only – No assurance given for non-security functionality © 2007 atsec information security • Some product vulnerabilities appear from the TOE environment and are “dismissed” by broad assumptions – This is technically correct – Hard for real-world users to understand – Does it help assess risk more accurately in the realworld? • Somewhat true also for FIPS 140 -2, although CMVP certificates and security policies tend to reference the whole product 8 th ICCC: CC in the Real World 14

“CC is inflexible; it can’t be used for my product!” • CC is very flexible © 2007 atsec information security – – SFR extensions PP Interpretations Ongoing standards development • It is difficult to produce a complete ST at the beginning of a project – “Before and during the evaluation, the ST specifies “what is to be evaluated”. In this role, the ST serves as a basis for agreement between the developer and the evaluator on the exact security properties of the TOE and the exact scope of the evaluation…. After the evaluation, the ST specifies “what was evaluated…” (CC part 1 A. 3. 1) • Compare – FIPS 140 -2 which is more restrictive, and actually means that there are several products that it can’t be used for or that it is very difficult to be used for. – ISMS can be used for any organization 8 th ICCC: CC in the Real World 15

“It’s hard work” • The more assurance end users need, the harder work it is • Internal effort is related to evaluation strategy, and can be reduced – Process improvements © 2007 atsec information security Yes it is! 8 th ICCC: CC in the Real World 16

© 2007 atsec information security “It assumes a waterfall model for development” • No it doesn’t! CC paradigm is flexible and it fits all development models. • Spiral methods, pair programming, iterative and incremental, agile, open source development, rapid prototyping etc can and have been used. • The evaluation too can be built in to the processes of these models and use similar paradigms • Examples: – The successful evaluation of Open Source development paradigm for various Linux evals, the provision of evaluation artifacts and evidence back to the community. – An iterative evaluation methodology to track and pace iterative product development. 8 th ICCC: CC in the Real World 17

© 2007 atsec information security “It requires too much documentation” 8 th ICCC: CC

It is a formalized methodology, and documentation that would not otherwise be produced is required in order to provide high assurance © 2007 atsec information security – E. g. ST, Correspondence analyses, • Other documentation is often produced “just for CC” that the lab should be able to find in a regular mature product development – E. g. Design documentation, Process documentation, Vulnerability assessment 8 th ICCC: CC in the Real World 19

“Documentation is never the cause of a security failure so why emphasise that!” • Documentation is an evaluation tool. It shows © 2007 atsec information security – The design is sound: A product can (and often is) designed poorly. Design flaws => vulnerabilities – The design is implemented to the level of abstraction specified – It shows the system was designed to implement the security requirements • Specialised documentation is an important part of providing assurance: Especially boundaries – PPs and ST for CC – Security Policy, Finite State Model (FSM) for FIPS 140 -2 – Scope and Statement of Applicability (SOA) for ISMS • Labs must be prepared to read and use existing documentation in any format. 8 th ICCC: CC in the Real World 20

“We want ONE evaluation that’s accepted everywhere” © 2007 atsec information security • Mutual Recognition • CCRA /SOGIS are examples • ISO/IEC 15408 • Failed for Smartcards • Failing because of need for higher levels of assurance (medium level robustness PPs) which are not covered under current MRAs 8 th ICCC: CC in the Real World 21

© 2007 atsec information security Addressing these issues using the CC paradigm • Developing useful Protection Profiles • Using a “staged” evaluation strategy, so that initial evaluation of a product at a lower evaluation level builds a good platform for later evaluation at a higher level • Pursuing ongoing communication between consumers, vendors, evaluation labs, and schemes, so that emerging consumer requirements on mature product lines are clearly understood by all parties in the evaluation process • Conducting development, Common Criteria consulting, and Common Criteria evaluation efforts in parallel with development efforts, so that certification of new products is timely (atsec example: Red Hat Linux evaluation project) http: //www. atsec. com/downloads/pdf/efficient_cc_evaluations. pdf 8 th ICCC: CC in the Real World 22

Conclusion © 2007 atsec information security • Real World Issues occur with every certification framework Ø Both standard and the certification scheme have to be sound. Ø Some specialised documentation is necessary, but this should be minimal. Ø The definition of clear boundaries are vital. Ø Balance “certification as a tax” vs “real-life improvement and benefits of certification” Ø Communication is vital ØMutual Recognition arrangements ØIncluding training for all stakeholders (Overseers, Labs, Vendors, Consumers, Sponsors) 8 th ICCC: CC in the Real World 23

Thank You atsec Information Security Corporation Austin, Texas, USA Phone: +1 512 615 7382 Email: fiona@atsec. com www. atsec. com © 2007 atsec information security Fiona Pattinson 8 th ICCC: CC in the Real World 24

References & Bibliography • • • © 2007 atsec information security • • • Olthoff, K. G. 1999, 'A cursory examination of market forces driving the Common Criteria', in New Security Paradigms Workshop, ACM Press, pp. 61 -66. Gordon, L. A. & Loeb, M. P. 2001, Economic Aspects of Information Security, Rainbow Technologies. Mahon, G. 2002, 'Developers backward engineering their products to meet the Common Criteria', in International Common Criteria Conference (ICCC), Ottawa. Merkow, M. 2003, Security assurance through the Common Criteria, New Riders Pub. , Indianapolis, IN. Apted, A. J. , Carthigaser, M. & Lowe, C. 2002, 'Common problems with the Common Criteria', in International Common Criteria Conference, Ottawa. Aizuddin, A. , The Common Criteria ISO/IEC 15408– The insight, some thoughts, questions and issues. Smith, R. E. , Trends in Government Endorsed Security Product Evaluations, Secure Computing Corporation. The common criteria -- Good, bad or indifferent? , Information Security Technical Report, vol. 2, no. 1, pp. 5 -6 PY - 1997 AU - L. Security evaluation and certification: The future of a national scheme. , Information Security Technical Report, vol. 2, no. 1, pp. 6 -6 PY - 1997 AU - H. Brewer, D. 2005, 'Is the Common Criteria the only way? ', International Common Criteria Conference, Tokyo. Oldegård, M. & Farnes, M. -L. 2005, 'Does Common Criteria need marketing? ', International Common Criteria Conference, Tokyo. Mueller, S. , 4/10/2006, ”Efficient Common Criteria Evaluations” : http: //www. atsec. com/downloads/pdf/efficient_cc_evaluations. pdf Katzke, S. 2004, The Common Criteria (CC) Years (1993 -2008): Looking Back and Ahead. GAO. 2006, Information Assurance: National Partnership offers benefits, but Faces Considerable Challenges, United States Government Accountability Office. Government, Computer News, 2007/05/04, “Symantec: Common Criteria is bad for you” Jackson, J: http: //www. gcn. com/online/vol 1_no 1/44205 -1. html CCMB-2006 -09 -01 "Common Criteria for Information Technology Security Evaluation Version 3. 1: Part 1: Introduction and general model" The Common Criteria Management Board. 8 th ICCC: CC in the Real World 25