ETSI Testing Projects ICT-OSA Parlay Workshop Brazil March 2006

ETSI Testing Projects ICT-OSA/Parlay Workshop Brazil, March 2006 Anthony Wiles Manager ETSI Protocol and Testing Competence Centre ETSI Testing Projects - ICT Workshop, Brazil March 2006

The ETSI PTCC q Protocol and Testing Competence Centre q In-house team of experts providing direct support to ETSI Technical Bodies (for over 12 years) q Development of protocols and profiles and the application of modern specification techniques (35%) Ø ‘Standards Engineering’ q Validation of standards (5%) Ø E. g. , testbed activities (IPv 6, SIP/H. 323) q Development of test specifications (60%) Ø Ø Conformance tests Interoperability tests Network Integration tests Others (PHY, performance …) q http: //www. etsi. org/ptcc ETSI Testing Projects - ICT Workshop, Brazil March 2006 3

Typical ETSI Test Specifications q Cellular: GSM and 3 G (UMTS) terminals Ø Now includes IMS q Wi. Fi: Hiper. MAN, Hiper. ACCESS, Wi. Max q Vo. IP: H. 323, SIP, SIGTRAN q Service Creation: OSA/Parlay (API, IDL) q IPv 6: Core, Security, Mobility, v 4 -v 6 q Cordless phones: DECT q Radio communications: TETRA, DMR q Access terminals: FSK, SMS q Broadband: ISDN, DSL q Smartcards: Readers, cards, security modules q Intelligent Transport Systems (ITS): DSRC q Future: More Security, more ITS, GRID. . . ETSI Testing Projects - ICT Workshop, Brazil March 2006 5

IPv 6 Test Specifications STF 276 q Joint ETSI – e. Europe project q Supported by the ETSI membership Ø Mainly vendors Ø Require IPv 6 test specifications using ETSI methods and test languages Ø Designed to be part of 3 GPP and TISPAN NGN IPv 6 testing strategy (where applicable) q European Commission: Ø Mandate for IPv 6 device interoperability Ø Amplify Europe’s voice in IPv 6 testing q Liaising with the IPv 6 Logo Program q For more info visit: www. ipt. etsi. org ETSI Testing Projects - ICT Workshop, Brazil March 2006 6

Two Phases (2005 – 2007) q Phase 1: Core IPv 6 Ø Total value approx. 0, 5 M€ Ø Jan – Dec 2005 q Phase 2: IPSec, Mobile IPv 6, IPv 4 -6 Transitioning Ø Total value approx. 1, 0 M€ Ø Jan 2006 – May 2007 q Provided as Ø Ø Funding Provision of experts Hosting costs Travel q All technical work done at ETSI by STF 276 Ø Currently 11 participants in the STF ETSI Testing Projects - ICT Workshop, Brazil March 2006 7

Outputs q IPv 6 Testing Framework Ø Published (update due end 2006) q IPv 6 Requirements Catalogue Ø On line as part of IPT Library q Test specifications Ø Ø Interoperability test specifications Conformance test specifications Library of TTCN-3 test code freely available On-line, extensible and reusable q All tests are validated (probably 30% of the effort) Ø On ETSI IPv 6 Testbed Ø By member companies q For more info visit: www. ipt. etsi. org ETSI Testing Projects - ICT Workshop, Brazil March 2006 8

Results Phase 1 q Over 1100 Core IPv 6 Requirements identified Ø Including 3 GPP q 300 Core IPv 6 conformance tests Ø fully validated q 120 Core IPv 6 Interoperability Tests Ø fully validated q All these test specifications including TTCN-3 code are free and open source ETSI Testing Projects - ICT Workshop, Brazil March 2006 9

STF 276@Work ETSI Testing Projects - ICT Workshop, Brazil March 2006 10

3 GPP UMTS (terminal) Test Specifications (STF 160) q Nobody can make money without interoperable handsets (UE)! q UE Interoperability towards access technologies is the key issue for mobility q Conformant UE = Interoperability (of UE in 3 G networks) Ø Good experience from GSM q Solution: conformance testing Ø Very detailed test specifications Ø Based on the 3 GPP core specs Ø Before commercial products enter the market Ø Voluntary declaration, but certified • One of the most important UE quality benchmarks ETSI Testing Projects - ICT Workshop, Brazil March 2006 11

Methodology for 3 G Testing Mobile Under Test SS testing Executable Test Suite (e. g. , C++) implementation Core specs 25. 331, 25. 321, 2 24. 008, 44. 018, 44. 060 PICS (capability applicability) 34. 123 -2, 51. 010 -2 logging and analysis Test Report compilation Test Purposes, 34. 123 -1 Descript (prose), 34. 108, 51. 010 -1 Test Cases (TTCN) 34. 123 -3, 51. 010 -5 ETSI Testing Projects - ICT Workshop, Brazil March 2006 12

3 G Test Industry Commitment is Vital q The TTCN test cases are implemented in the System Simulators (SS) on different platforms Ø Must secure the commitment of the SS vendors q SS vendors organise verification and regression testing Ø Each and every test case is validated Ø Bug-free tests have a high real value Ø Important to ensure that a TC will execute correctly even after updates q Early identification of clients who deploy the tests Ø Prioritising TCs Ø Scheduling the deployment Ø Certifications ETSI Testing Projects - ICT Workshop, Brazil March 2006 13

Funding q Test specs developed at ETSI by STF 160 Ø Duration of several years q Shared costs to ensure the funding Ø 3 GPP funding from Organisation Partners Ø Main source, approx 1 M€ (61 mm) /year) Ø Voluntary partners contributions from the marketing Ø Range from 65 k€ - 150 k€ (5 – 12 mm) /year Ø Man-power (free-of-charge) Ø Range from 15 - 18 mm /year ETSI Testing Projects - ICT Workshop, Brazil March 2006 14

Typical 3 GPP UMTS UE Tests q Protocol and signalling tests via radio interface Ø Largest part of the testing process Ø Full conformance tests q USIM functional tests Ø Identification, PIN, account charging, network priority, phonebook q UICC tests (SIM electrical test) Ø power on/off, 1. 8 v/3 v detection, presence detection Ø bit/character transmitting duration, command processing and error handling q Acoustic tests Ø Loudness rating, idle noise, echo … q Radio Frequency & Radio Resource Management tests Ø Characteristics and performance tests of transmitter & receiver ETSI Testing Projects - ICT Workshop, Brazil March 2006 15

Users of 3 GPP Tests q Global Certification Forum (GCF) q PCS Type Certification Review Board (PTCRB ) Ø GCF + PTCRB are the major clients for W-CDMA (FDD) & GSM/GPRS test specs q TD-SCDMA Industry Alliance (TDIA) Ø TDIA is the major clients for TD-SCDMA (LCR TDD) test specs q GCF WI-10 selected 400 TCs as high priority Ø Divided in 4 packages Ø Progressing sequentially one by one (milestone 80%) q UMTS mobile certification will be started if 80% of each package are validated ETSI Testing Projects - ICT Workshop, Brazil March 2006 16

Approval Process q Test cases are approved by e-mail Ø No quality compromise Ø Two week consultation period Ø TCs are approvedone by one q Each test must run on at least one SS platform against two independent commercial mobiles Ø Log files and radio conditions as support information for approval q Reason for this Ø TTCN TC, SS and UE are developed at the same time Ø None of them is perfect ETSI Testing Projects - ICT Workshop, Brazil March 2006 17

Results q 400 TCs in 8 ATSs, 25 MB code Ø Platform independent Ø Running on > 4 platforms q We have reached the GCF 80% validated milestone Ø 3 G mobile certification starting at beginning of 2005 Ø The light is green for mass 3 G mobiles entering the markets Ø 4 types of 3 G mobiles are ready to go q Begun the real 3 G test business Ø 4 validated 3 G SS platforms available in the markets Ø 4 test houses in Europe opening 3 G test services Ø 3 G TTCN experts becoming hot in the labour market q We know the price of each 3 G coded TC Ø 10 k€ /TC Ø 3 G investment for 4 years: 4 mil. € for 400 TCs q All TTCN code can be downloaded free of charge q Code is continuously maintained and delivered Ø Every three weeks for 3 G members Ø Every three months for everybody ETSI Testing Projects - ICT Workshop, Brazil March 2006 18

Vo. IP-related Test Specifications STF 269, STF 270 and others q SIP (RFC 3261) Ø Over 400 conformance test cases Ø Fully validated (executables available on commercial tools) Ø STF 270 is doing the maintenance q H. 225 Ø About 300 conformance test cases Ø Fully validated (executables available on commercial tools) Ø No ongoing maintenance q SIGTRAN Ø Test purposes for M 3 UA, M 2 UA, SCTP Ø TTCN-3 available commercially (not ETSI) q IMS SIP (together with SDP) Ø Ø 3 GPP RAN 5 (STF 160) TTCN-3 will be used Work started March 2006 Based on the generic SIP test suite (see above) augmented to concentrate on 3 GPP SIP specific requirements ETSI Testing Projects - ICT Workshop, Brazil March 2006 19

ETSI – Wi. MAX Test Specifications STF 252 q Wi. MAX Forum Board meeting in January 2005 Ø Agreed to harmonize the Wi. MAX/Hiper. MAN base specifications Ø Agreed to harmonize the Wi. MAX/Hiper. MAN test specifications Ø Agreed to exploit ETSI’s testing expertise Ø Agreed to accelerate the work using the ETSI STFs Ø Agreed that validation of the test specifications needed • complex interplay of all parties involved • Wi. MAX Forum, ETSI BRAN, equipment vendors, ETSI STF, test tool developers, test labs, etc. q ETSI Wi. MAX Co-operation agreement signed in 2005 ETSI Testing Projects - ICT Workshop, Brazil March 2006 20

Results q Wave 1: Joint test specifications approved and published Ø Ø Completed Dec. 2005 PICS – Protocol Implementation Conformance Statement TSS&TP – Test Suite Structure and Test Purposes ATS – Approx 60 test cases implemented, validated and used in certification q First Wi. MAX products have been certified q Wave 2: New drafts reaching approval Ø PICS – new updated version with corrections Ø TSS&TP – new draft with close to 1400 Test Purposes Ø ATS – new draft with close to 300 additional test cases developed and entering validation ETSI Testing Projects - ICT Workshop, Brazil March 2006 21

IPT Library STF 295 q IP Testing Library Ø Ø Developed by ETSI TC MTS All MTS IP and IP-related conformance and IOP tests PICS, TPs, Test Suites etc. Due to be published April 2006 q SIP, IPv 6, H. 323, H. 248 q Includes IPv 6 Requirements Catalogue Ø Linked to TPs and TTCN-3 code q Powerful search and selection possibilities Ø E. g. , allows test suite tailored to specific search criteria ETSI Testing Projects - ICT Workshop, Brazil March 2006 22

Screen Shot of IPT Library ETSI Testing Projects - ICT Workshop, Brazil March 2006 23

Plugtests. TM ETSI Interoperability Service q Created in 1999, specialized in the organization of test events q Open to all organizations q Experts from ETSI and other technical communities q Skilled ETSI Staff managing test events In 6 years, 66 Events 3500 engineers 1400 Companies Usefulness & Service Satisfaction ranked at 4. 35 / 5 ETSI Testing Projects - ICT Workshop, Brazil March 2006 24

Why? q Provides useful feedback to the standardization process q Accelerates time to market via quick product debugging q Improves both quality and features of implementations q Helps to ensure end users’ satisfaction q Reduces deployment costs q Allows networking with the market key players COST & TIME SAVING! ETSI Testing Projects - ICT Workshop, Brazil March 2006 25

Upcoming events in 2006 Q 1 – 2006 Q 2 – 2006 SIM/Handset Lawful Interception 3 rd Wi. MAX Plugfest Digital Identity Q 3 – 2006 OSA Parlay UWB SIGTRAN IPv 6 Asia 4 th SQTE EU round NGN Mobile Application NGN@Home Q 4 – 2006 Human Factors (HF) IPv 6 French Summit & Interop (FR) Powerline RFID ITS ecall ETSI Testing Projects - ICT Workshop, Brazil March 2006 26

Thank You! Phone +33 (0)4 92 94 43 26 e-mail ptcchelp@etsi. org Website www. etsi. org/ptcc Anthony Wiles ETSI Protocol and Testing Competence Centre ETSI Testing Projects - ICT Workshop, Brazil March 2006