Sun s 2002 Taipei Java Developer Conference JUnit 軟體測試架構

Sun’s 2002 Taipei Java Developer Conference JUnit 軟體測試架構 JUnit Software Testing Framework 葉秉哲 2002/7/4. http: //william. cswiz. org/present/20020704

JUnit 的定位 JUnit is… scope JUnit is not… 單元測試骨幹、架構整合測試完整系統具方法論 test case 手動產生自動產生 test driver / script 部份手寫全自動 testware level 2

大綱 • 引子 • JUnit 基礎篇 • JUnit 進階篇 • 軟體測試基礎 • 推薦讀物 3

一、引子一流的科學家可以做出很有價值的實驗，產生新知識；二流科學家只是忙於各種實驗，蒐集大量數據，但對知識的累積沒什麼用處。 --- David Salsburg

JUnit 簡介 • Regression testing framework written by Erich Gamma and Kent Beck • Open Source – Hosted on Source. Forge • Language support – Smalltalk, Java, C++, Perl, Python, etc. • IDE Support – JBuilder, Visual. Age, etc. 5

What is Software Testing? • The execution of code using combinations of input and state selected to reveal bugs. • The process of devising and executing a test suite that attempts to cause failures, increasing our confidence that failures are unlikely. 6

Why Testing? • 目的：矛與盾 – Reveal bugs: quality control – Confidence: qualitative & quantitative • 重要性 – Paradigm↑ productivity↑ testing & debugging↑ – Software features that can’t be demonstrated by automated tests simply don’t exist. [Beck 1999, p. 45] 7

Testing: CMM Perspective Source: Capability Maturity Model for Software, v 1. 1 http: //www. sei. cmu. edu/publications/documents/93. reports/93. tr. 024. html 8 The Key Process Areas by Maturity Level

Testing: RUP Perspective Source: Philippe Kruchten, The Rational Unified Process: An Introduction, p. 62, Addison Wesley, 1998. 9

Testing: XP Perspective Source: Kent Beck, Extreme Programming Explained: Embrace Change, p. 70, Addison Wesley, 1999. 10 If you’re not validating everything all the time, you’re not doing XP. Period.

Test What? • Every artifact during the whole development process should be tested. 11

軟體測試具分類分類實例 reviews & inspections lint Compuware Dev. Partner test planning Rational Suite Test. Studio test design & development test execution & evaluation capture/playback coverage analysis Compuware Dev. Partner Rational Suite Test. Studio Borland Optimizeit memory testing Compuware Dev. Partner Rational Suite Test. Studio test case management sumulations & performance test support 12 profiler

JUnit 的定位 JUnit is… scope JUnit is not… 單元測試骨幹、架構整合測試完整系統具方法論 test case 手動產生自動產生 test driver / script 部份手寫全自動 testware level 13

JUnit Framework 14

How to Test with JUnit? junit Test. Case Foo 1. . * exercise Foo. Test test 1 test 2 … 15 Test. Runner 1. . * run

Questions… • 如何測試？ • 多少測試才夠？ • 流程？ • 具影響？ 16 strategy coverage process medium is message

二、JUnit 基礎篇 Extreme Programmers test everything that could possibly break, using automated tests that must run perfectly all the time. --- Extreme Programming Installed

Design Goals of JUnit • Easy to use – Requires no more work than absolutely necessary to write a new test • Leverages existing tests to create new ones – Reusable fixtures to run different tests • Retains values of tests over time – Combines tests from various authors and run them together without fear of interference 18

Case Study 1) 自動產生測試框架 • Using Borland JBuilder 6. 0 2) 最簡單的完整測試實例 19

實例一 junit Test. Case Foo 1. . * exercise Foo. Test test 1 test 2 … 20

實例一：簡單的待測物 public class Money { private int f. Amount; private String f. Currency; public Money(int amount, String currency) { f. Amount = amount; f. Currency = currency; } public Money add(Money m) { return new Money(amount() + m. amount(), currency()); } public int amount() public String currency() } 21 { return f. Amount; } { return f. Currency; }

實例一：產生測試框架 Borland JBuilder 6. 0 22 [1/7]

實例一：產生測試框架 Borland JBuilder 6. 0 23 [2/7]

實例一：產生測試框架 Borland JBuilder 6. 0 24 [3/7]

實例一：產生測試框架 Borland JBuilder 6. 0 25 [4/7]

實例一：產生測試框架 Borland JBuilder 6. 0 26 [5/7]

實例一：產生測試框架 Borland JBuilder 6. 0 27 [6/7]

實例一：產生測試框架 [7/7] import junit. framework. *; public class Money. Test extends Test. Case { public Money. Test (String s) { super(s); } protected void set. Up() protected void tear. Down() { { } } public void test. Add() { int val 1 = 0; String val 2 = "STRING 0"; Money money = new Money(val 1, val 2); Money val 1 = null /** @todo fill in non-null value */ ; Money money. Ret = money. add(val 1); /** @todo: Insert test code here. Use assert. Equals(), for example. */ } } 28

Lesson Learned • Code a little, test a little 29

實例二 junit Test. Case Foo 1. . * exercise Foo. Test test 1 test 2 … 30 Test. Runner run

實例二：待測物，修正版 public class Money implements Cloneable { private int f. Amount; private String f. Currency; public Money(int amount, String currency) { /*. . . */ } public Money add(Money m) { if (m == null) return (Money) clone(); return new Money(amount() + m. amount(), currency()); } public } 31 Object clone() boolean equals(Object obj) int amount() String currency() { { /*. . . */ } }

實例二：測試碼 import junit. framework. *; public class Money. Test extends Test. Case { public Money. Test (String s) { super(s); } public void test. Null. Add () { String curr = "NTD"; int val 1 = 2002; Money money 1 = new Money(val 1, curr); Money money 2 = null; // null value Money money 3 = money 1. add(money 2); Assert. assert. Equals(money 1, money 3); } 32 [1/2]

實例二：測試碼 [2/2] public void test. Simple. Add() { String curr = "NTD"; int val 1 = 2002; int val 2 = 345; int val 3 = val 1 + val 2; Money money 1 = new Money(val 1, curr); Money money 2 = new Money(val 2, curr); Money money 3 = money 1. add(money 2); assert. Equals(money 3, new Money(val 3, curr)); } } 33

Success java 34 -cp libjunit. jar; . junit. swingui. Test. Runner Money. Test

Failure 1/2 35

Failure 2/2 36

Error 1/2 37

Error 2/2 38

Lessons Learned • Generate & exercise test case(s) – Write test…() method(s) • Verify – Choose a Assert. assert…() • Choose a Test. Runner – junit. textui. Test. Runner – junit. swingui. Test. Runner • Failure vs. error 39

三、JUnit 進階篇 The tests that you write in XP are isolated and automatic. --- Extreme Programming Explained

Case Study 3) Fixture/context 管理 4) Test suite 與進入點 5) Grouping by tasks • Configuration management • Regression testing 41

實例三 junit Test. Case Foo 1. . * exercise set. Up() tear. Down() Foo. Test test 1 test 2 … 42 Test. Runner 1. . * run

實例三：測試碼 [1/2] import junit. framework. *; public class Money. Test extends Test. Case { private Money money 1, money 2, money 3; public Money. Test(String s) { /*. . . */ } protected void set. Up() { String curr = "NTD"; int val 1 = 2002; int val 2 = 345; money 1 = new Money(val 1, curr); money 2 = new Money(val 2, curr); money 3 = new Money(val 1 + val 2, curr); } 43

實例三：測試碼 protected void tear. Down() [2/2] { } public void test. Null. Add() { Money money 10 = money 1. add(null); // null value assert. Equals(money 1, money 10); } public void test. Simple. Add() { Money money 10 = money 1. add(money 2); assert. Equals(money 3, money 10); } } 44

Lessons Learned • Fixture – Overrides set. Up() – Overrides tear. Down() • Each test…() method is isolated 45

實例四 junit Test. Case Foo 1. . * exercise Foo. Test. Suite 1. . * test 1 test 2 … test case 46 test suite Test. Runner run

實例四：測試碼 [1/3] import junit. framework. *; public class Money. Test extends Test. Case { private Money money 1, money 2, money 3; public Money. Test (String s) protected void set. Up() public void test. Null. Add() public void test. Simple. Add() 47 { { /*. . . */ } }

實例四：測試碼 [2/3] // [interactive mode] // entry point of the whole test suite! /* // Version 1 public static Test suite() { Test. Suite suite = new Test. Suite(); suite. add. Test(new Money. Test("test. Null. Add")); suite. add. Test(new Money. Test("test. Simple. Add")); return suite; } */ // Version 2 public static Test suite() { return new Test. Suite(Money. Test. class); } 48

實例四：測試碼 // [batch mode] // entry point of the whole test suite! public static void main(String args[]) { junit. textui. Test. Runner. run(suite()); } } 49 [3/3]

Lessons Learned • Test case suite – Write a suite() as the interactive mode entry point – Write a main() as the batch mode entry point 50

實例五：Composite Pattern 51

實例五：before grouping test case Foo. Test Foo 52 exercise test 1 test 2 test 3 … test suite

實例五：grouping by tasks test suite basic test 1 test 2 Foo exercise random test 3 monkey test stress test 53 Foo. Test

$實例五：composite test suites public class Money_test extends Test. Case { public Money. Test (String$

實例五：composite test suites public class Money_test extends Test. Case { public Money. Test (String s) { /*. . . */ } public static Test suite() { Test. Suite suite = new Test. Suite(); suite. add. Test(Money_test_basic. suite()); suite. add. Test(Money_test_random. suite()); suite. add. Test(Money_test_monkey. suite()); return suite; } public static void main(String args[]) { junit. textui. Test. Runner. run(suite()); } } 54

55

Lessons Learned • Grouping by configuration policy – By tasks – By persons – By revisions • Configuration management – Naming scheme – Process – Scripts 56

小結：JUnit 使用四部曲 • Fixture – Overrides set. Up() and/or tear. Down() • Exercise – Writes test…() • Verify – Uses Assert. assert…() • Suite management – Writes suite() and/or main() 57

其他主題 • Reports – Test. Result • Extensions for various application domains 58

四、軟體測試基礎 Program testing can be used to show the presence of defects, but never their absence! --- Edward Dijkstra

Why Verification/Testing is Hard? • 理論限制 – Intractable – Undecidable • 實務限制 – 成本 – 流程 – 人因 60

軟體測試方法論：分類 • Scope – – Unit test Integration test System test Acceptance test • Strategy – – 61 Responsibility-based (black box) Implementation-based (white box) Hybrid (gray box) Fault-based

OO Testing Manifesto • 觀察 – – No silver bullet Every advance has a price Iterative & incremental Regression testing • 信條 – Fault models – Test automation – Test-effective process 62

Fault Models • 重要性 – exhausive testing 不可能，合理的 testing strategy 必須根據 fault models 的引導 • 正面: conformance-directed – Feature sufficient • 反面: fault-directed – Fault efficient 63

五、推薦讀物 Tests are both a resource and a responsibility. --- Extreme Programming Explained

軟體測試通論 • Edward Kit, Software Testing in the Real World: Improving the Process, Addison-Wesley/ACM Press, 1995. • Boris Beizer, Black-Box Testing, John Wiley & Sons, 1995 • Larry J. Morell and Lionel E. Deimel, Unit Analysis and Testing, SEI Curriculum Module SEI-CM-9 -2. 0, June 1992. http: //www. sei. cmu. edu/publications/documents/cm. 009. html • Hong Zhu, Patrick A. V. Hall, John H. R. May, “Software Unit Test Coverage and Adequacy, ” ACM Computing Surveys, vol 29, no 4, 1997/12. 65

物件導向軟體測試 • Robert V. Binder, Testing Object. Oriented Systems: Models, Patterns, and Tools, Addison-Wesley, 2000. 66

JUnit • JUnit 網站: http: //www. junit. org – Test Infected: Programmers Love Writing Tests – JUnit Cookbook – JUnit: A Cook’s Tour 67

與 JUnit 有關的軟體開發流程 • [Beck 1999] Kent Beck, Extreme Programming Explained: Embrace Change, Addison-Wesley, 1999. • Ron Jeffries, Ann Anderson, and Chet Hendrickson, Extreme Programming Installed, Addison. Wesley, 2001. 68

69