Скачать презентацию Software Testing Verification Group Moonzoo Kim http Скачать презентацию Software Testing Verification Group Moonzoo Kim http

2447df2b23c9ee761f458dde1b8e0384.ppt

  • Количество слайдов: 18

Software Testing & Verification Group Moonzoo Kim http: //swtv. kaist. ac. kr Software Testing & Verification Group Moonzoo Kim http: //swtv. kaist. ac. kr

Main Research Theme • To Improve SW reliability through automated testing – Quality attribute Main Research Theme • To Improve SW reliability through automated testing – Quality attribute for minimizing malfunctions of systems to reduces damage to human life or valuable properties (i. e. , to fight software bug) • Highly reliable SW technology is a key to the success of industrial products – The portion of SW in embedded devices increases continuously Home Service Robots Intelligent Medical Devices Home Network Intelligent Mobile Systems Highly Reliable Software-intensive Systems Introduction of SWTV group 2/14 SWTV group. CS Dept. KAIST

How to Assure Quality of SW? Introduction of SWTV group 3 SWTV group. CS How to Assure Quality of SW? Introduction of SWTV group 3 SWTV group. CS Dept. KAIST

Keynote by Bill Gates @ OOPSLA, 2002 • “… When you look at a Keynote by Bill Gates @ OOPSLA, 2002 • “… When you look at a big commercial software company like Microsoft, … We have as many testers as we have developers. Testers basically test all the time, and developers basically are involved in the testing process about half the time…” • “… We're not in the software industry; we're in the testing industry, and writing the software is the thing that keeps us busy doing all that testing. ” • “…The test cases are unbelievably expensive; in fact, there's more lines of code in the test harness than there is in the program itself. Often that's a ratio of about three to one. ” Introduction of SWTV group 4 SWTV group. CS Dept. KAIST

Significance of Automated SW Analysis • Software has become more ubiquitous and more complex Significance of Automated SW Analysis • Software has become more ubiquitous and more complex at the same time • Human resources are becoming less reliable and more expensive for highly complex software systems • Computing resources are becoming ubiquitous and free – Tencent @ China provides 10 TB storage free – Amazon EC 2 price: you can use thousands of CPUs @ 0. 057$/hr for 3. 2 Ghz Quad-core CPU • Remaining task? – To develop automated and scientific software analysis tools to utilize computing resource effectively and efficiently Introduction of SWTV group 5 SWTV group. CS Dept. KAIST

Research Methodology Science Engineering • Discrete math SW TV • Algorithm group • PL Research Methodology Science Engineering • Discrete math SW TV • Algorithm group • PL Embedded Systems Software Engineering System modeling Requirement properties Introduction of SWTV group • System programming • SW Engineering • Security Programming Languages System spec. Req. spec. Algorithms Automated Testing assert(φ ) 6/14 Counter example(s) SWTV group. CS Dept. KAIST OK or

Ex. Testing a Triangle Decision Program Input : Read three integer values from the Ex. Testing a Triangle Decision Program Input : Read three integer values from the command line. The three values represent the length of the sides of a triangle. Output : Tell whether the triangle is • 부등변삼각형, 이등변삼각형, 정삼각형 Create a Set of Test Cases for this program: (3, 4, 5), (2, 2, 1), (1, 1, 1) ? • Condition 1: a > 0, b > 0, c > 0 • Condition 2: a < b + c – Ex. (4, 2, 1) is an invalid triangle – Permutation of the above condition : a < b +c, b < a + c, c < a + b • What if b + c exceeds 2 32 (i. e. overflow)? – long v. s. int v. s. short v. s. char • Developers often fail to consider implicit preconditions – Cause of many hard-to-find bugs Introduction of SWTV group. CS Dept. KAIST

 • # of test cases required? ① 4 ② 10 ③ 50 ④ • # of test cases required? ① 4 ② 10 ③ 50 ④ 100 • # of feasible unique execution paths? • 11 paths • guess what test cases needed Introduction of SWTV group “Software Testing a craftsman’s approach” 2 nd ed by P. C. Jorgensen (no check for positive inputs) 8/60 SWTV group. CS Dept. KAIST

Research Trends toward Quality Systems • Academic research on embedded systems has reached stable Research Trends toward Quality Systems • Academic research on embedded systems has reached stable stage – just adding new functionalities to a target system is not considered as an academic contribution anymore • Research focus has moved to the quality of the systems from the mere functionalities of the systems – Energy efficient design, ez-maintenance, dynamic re-configuration, etc • Software reliability is one of the highly pursued qualities – ASPLOS 2011 Best paper • “S 2 E: a platform for in-vivo multi-path analysis for software systems” @ EPFL – OSDI 2008 Best paper • “Klee: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs” @ Stanford – NSDI 2007 Best paper • “Life, Death, and the Critical Transition: Finding Liveness Bugs in Systems Code” @ U. C. San Diego Introduction of SWTV group 9/14 SWTV group. CS Dept. KAIST

Collaborators of Our Group Introduction of SWTV group 10/14 SWTV group. CS Dept. KAIST Collaborators of Our Group Introduction of SWTV group 10/14 SWTV group. CS Dept. KAIST

Research Directions @ SWTV Sequential program testing Concurrent program testing Test cases Thread sched. Research Directions @ SWTV Sequential program testing Concurrent program testing Test cases Thread sched. scenarios SWTV group Automated SW Quality Assurence Fault localization 2018 -03 -18 Suspicious stmts. 뮤즈 (MUSE): 프로그램의 수많은 돌연변이들을 활용한버그 위치 추정 기법 SWTV group 11

Introduction of SWTV group 12 SWTV group. CS Dept. KAIST Introduction of SWTV group 12 SWTV group. CS Dept. KAIST

Concolic 테스팅 과제의 산업적 성취 산학 과제 수행 연도 2010년 Phase 1(2010) Concolic testing Concolic 테스팅 과제의 산업적 성취 산학 과제 수행 연도 2010년 Phase 1(2010) Concolic testing 타당성 조사 • • Pilot 과제 적용을 통한 concolic testing 타당성 조 사 오픈소스 소프트웨어 (busybox) 및 S사 소프트웨 어(SLP(Tizen 전신) file manager, Samsung security library) 에 적용하 여 새로운 버그 발견 Introduction of SWTV group 2011년-2012년 Phase 2(2011 -2012) Concolic testing 도구 개발 • • • S사 개발 코드를 테스트하기 위한 자동화된 concolic unit testing 도구 CONBOL 개발 4 MLOC 규모의 삼성 내장형 프로그램에 적용하여 결함 발 견 및 개발자 보고 후 수정 기존 concolic testing 도구를 확장하여 bit 수준 정확도, BOF 오류 감지, 실행 속도 개 선 수행 2018 -03 -18 2013년-2014년 Phase 3(2013 -2014) Concolic testing 도구 속도 향상 및 오탐 감소 • • • CONBOL 오탐 감소를 위 한 unit-testing 전략, pre -condition 생성 기술 개 발 CONBOL 실행 속도 및 사 용성 개선: 64 bit 지원, 최 신 SMT solver 지원 등 주 단위 적용을 통해 삼성 내장형 프로그램 및 open source project의 실제 결 함 검출 13 SWTV group. CS Dept. KAIST

Concolic 테스팅 과제의 학문적 성취 산학 과제 수행 연도 • • FSE 2011 held Concolic 테스팅 과제의 학문적 성취 산학 과제 수행 연도 • • FSE 2011 held in Szeged, Hungary 논문 발표 FSE: SE 분야 TOP 2 학 회 Introduction of SWTV group 2010년 • • 2011년 2012년 ICSE 2012 held in Zurich, Switzerland, ICST 2012 held in Montreal Canada 논 문 2건 발표 ICSE: SE 분야 TOP 1 학회 2018 -03 -18 2013년 • • • 2014년 ASE 2013 held in Palo Alto, CA, USA 논문 발표 삼성 논문상 동상 수상 ASE: SE 분야 TOP 3 학회 14 SWTV group. CS Dept. KAIST

Recognition of Success of CONBOL at Samsung Electronics • Bronze Award at Samsung Best Recognition of Success of CONBOL at Samsung Electronics • Bronze Award at Samsung Best Paper Award • Oct’s Best Practice Award • Team leader Dr. Yoonkyu Jang received Samsung Award of Honor 15/23

Ph. D Students • • • Introduction of SWTV group Topic : Automated testing Ph. D Students • • • Introduction of SWTV group Topic : Automated testing 2 top intl. journals such as IEEE TSE and FACJ 16 top intl. conf. Including ICSE/FSE/ASE Software system: – Scalable COncolic testing for Relibaility (SCORE) Topic: Concurrency testing J. of Systems and SW, JSTVR, ISSTA’ 12, ICST’ 13, ICST’ 14, etc. Samsung Humantech Bronze award Qualcomm fellowship award Software system: – COncurrent Bug d. ETection framework (COBET) SWTV group. 16 CS Dept. KAIST

MS Students • Recent Alumni • • • Topic : automated fault localization Attended MS Students • Recent Alumni • • • Topic : automated fault localization Attended ICSE 2014 @ India Preparing OOPSLA 2015 Introduction of SWTV group • • • Topic : concurrent program testing and symbolic execution Best MS thesis award ICSE 2015 SEIP (top S rank), ICST ‘ 14 Best paper award at KCSE 2014 Best paper award at KIISE paper competition Qualcomm fellowship award • Topic : concolic testing SWTV group. CS Dept. KAIST

Questions? Comments? Introduction of SWTV group 18/14 SWTV group. CS Dept. KAIST Questions? Comments? Introduction of SWTV group 18/14 SWTV group. CS Dept. KAIST