We Test You Succeed Trust the

We Test. . . You Succeed Trust the Experts Demystifying the Independent Test Authority (ITA) Voting Systems Testing Summit 2005 Sacramento, CA November 28 -29, 2005 Presented by: Brian Phillips 3/17/2004

Who is Sys. Test Labs? Established in 1996 Sole Business Focus is to Provide Test Engineering, Quality Assurance and Certification Services Voting Systems ITA Qualification Testing Independent Verification and Validation (IV&V) Software Test Engineering Consulting and Services Accredited Software Independent Test Authority in 2001 Accredited Hardware Independent Test Authority in 2004 Small and Woman-Owned Business (70+ staff) Sys. Test Labs’ Proprietary and Confidential 3/19/2018

What is an Independent Test Authority? ITA Charter from NASED Qualification Testing of DRE Election Systems to Voting System Standards 3 main components Election Management Polling Place Central Count Not Exhaustive Testing Sys. Test Labs’ Proprietary and Confidential 3/19/2018

Software ITA Test Standards & Resources Sys. Test Labs uses the following to perform ITA Qualification Testing: Federal Election Commission Voting System Standards published April 2002, Released 2003 Sys. Test Labs Quality Manual & Standard Procedures (Advanced Test Operations Management™) Various review/report forms, databases, templates, and tools Sys. Test Labs’ Proprietary and Confidential 3/19/2018

Software ITA Process Pre-Qualification Technical Data Package (TDP) review and report Physical Configuration Audit - documentation, source code, test result review Functional Configuration Audit – hardware and software testing to verify logic, results and accuracy Final Qualification Report Sys. Test Labs’ Proprietary and Confidential 3/19/2018

Process after ITA State Certification County Acceptance Testing Sys. Test Labs’ Proprietary and Confidential 3/19/2018

We Test. . . You Succeed Trust the Experts Sys. Test Labs’ Proprietary and Confidential 3/19/2018