University of Southern California Center for Systems and

University of Southern California Center for Systems and Software Engineering A Tractable Approach to Handling Software Productivity Domains Thomas Tan Brad Clark 24 th International Forum on COCOMO and Systems/Software Cost Modeling November 3, 2009 24 th International Forum on COCOMO and Systems/Software Cost Modeling

University of Southern California Center for Systems and Software Engineering Table of Contents • • Overview Operational Context Software Application Difficulties Productivity Group Matrix – Example Mapping of Systems – Aerospace Project Data Analysis • COCOMO II Implications • Next Steps • Q&A 24 th International Forum on COCOMO and Systems/Software Cost Modeling 2

University of Southern California Center for Systems and Software Engineering Overview • Conventional labels used in Software Productivity Domains use system engineering terms • Systems can be comprised of many different software applications, e. g. Command Control • Productivities are influenced by the difficulty of the application type and the operating environment • Many applications in a system can exhibit different productivities 24 th International Forum on COCOMO and Systems/Software Cost Modeling 3

University of Southern California Center for Systems and Software Engineering Motivation • Our motivation is to create a new productivity classification system that can be – Understood by both cost estimators and engineers – Flexible enough to accommodate a wide range of software applications • This presentation discusses this new approach • As a work-in-progress, we welcome comments and suggestions • Please join us at the workshop to discuss 24 th International Forum on COCOMO and Systems/Software Cost Modeling 4

University of Southern California Center for Systems and Software Engineering Overview • This approach employs general descriptions to characterize the intended “operational context” (environment) and an application’s “degree of difficulty”: – Operational Context describes the intended environment, platform or target host on which the software application will operate – The application difficulty is the degree of difficulty in specifying, developing, and testing a software application 24 th International Forum on COCOMO and Systems/Software Cost Modeling 5

University of Southern California Center for Systems and Software Engineering Operational Context Dimensions of constraints include electrical power, computing capacity, storage capacity, repair capability, platform volatility, physical environment accessibility, etc. 24 th International Forum on COCOMO and Systems/Software Cost Modeling 6

University of Southern California Center for Systems and Software Engineering Operational Context • Very Unconstrained – – – • Unlimited electrical power with backup battery Regular upgrades and maintenance Heavily COTS-hardware and COTS-software dependent Support multiple users Limited loss during system down time Unconstrained – Operating environment may move over land water – Electrical power is limited by mobility (size dependent) – Computing and storage capability depends on ability to carry weight and size – Upgrades and maintenance are not frequent – Components are modular and cannot be broken apart 24 th International Forum on COCOMO and Systems/Software Cost Modeling 7

University of Southern California Center for Systems and Software Engineering • Operational Context Constrained – Similar to unconstrained in term of upgrade, maintenance, and component structure – Operating environment may move through the air – Electrical power, computing and storage capability are limited due to weight constraints • Very Constrained – Battery is the primary source of electrical power. – Components are miniaturized and custom-made with special tool to access – No ability to upgrade and maintain system after deployment • Highly Constrained – – – Hardware is required to survive in adverse conditions Limited software control Real-time sensor inputs and commands Upgrade and maintenance is not feasible after deployment Unique system platform and evolving in maturity as system being developed 24 th International Forum on COCOMO and Systems/Software Cost Modeling 8

University of Southern California Center for Systems and Software Engineering Software Application Difficulties Difficulty would be described in terms of required software reliability, database size, product complexity, integration complexity, information assurance, realtime requirements, different levels of developmental risks, etc. 24 th International Forum on COCOMO and Systems/Software Cost Modeling 9

University of Southern California Center for Systems and Software Engineering Software Application Difficulties • Very Easy – – – – – • Risks are well understood and little loss Business or operational logic is straightforward Limited in interfacing with other software applications Mostly stand-alone functionality Simple tests Not a new type of application Risks are understood and cure exists Business or operational logic is straightforward Requires low reliability due to small or little loss when unavailable Limited external interface and security requirements Nominal – – – Somewhat complicated business logic Risks exists and may need additional study to find mitigation May require distributed environment with additional security requirements Moderate, easily recoverable loss for nominal reliability Not a new type of application 24 th International Forum on COCOMO and Systems/Software Cost Modeling 10

University of Southern California Center for Systems and Software Engineering Software Application Difficulties • Challenging – High reliability due to greater size of loss or high probability of risk – Risks are challenging to resolve – Very complicated business logic, external storage may be necessary due to distributed environment – New type of application – High level real-time response and security requirements – Additional communication interfaces necessary for external components or systems • Very Challenging – – – Extremely complicated business logic Risks are very challenging to resolve and loss is great (disastrous consequences) Many automated controls with limited human control New type of application High level real-time response and security requirements Communication to external components through different interfaces 24 th International Forum on COCOMO and Systems/Software Cost Modeling 11

University of Southern California Center for Systems and Software Engineering Productivity Group Matrix • The intersection of an Operational Context and Application Difficulty represent a Productivity Group – Each group includes number of project data points, data range, simple cost estimating relationship (of the form: Y = a. Xb), standard error, percent bias, and the coefficient of determination, R 2 Productivity Group Matrix Operational Context Application Difficulty Very Unconstrained Constrained Very Constrained Extremely Constrained Very Easy 1, 1 1, 2 1, 3 1, 4 1, 5 Easy 2, 1 2, 2 2, 3 2, 4 2, 5 Nominal 3, 1 3, 2 3, 3 3, 4 3, 5 Challenging 4, 1 4, 2 4, 3 4, 4 4, 5 Very Challenging 5, 1 5, 2 5, 3 5, 4 5, 5 24 th International Forum on COCOMO and Systems/Software Cost Modeling 12

University of Southern California Center for Systems and Software Engineering Example Mapping to Systems • Very Unconstrained and Very Easy may include systems like: – – – Business Analysis Tools Database Data Mining Data Warehousing Diagnostics • Very Unconstrained and Easy may include systems like: – – Logistics Training Support Test • Constrained and Challenging may include systems like: 24 th International Forum on COCOMO and Systems/Software Cost Modeling 13

University of Southern California Center for Systems and Software Engineering • Aerospace Project Data Analysis Data collection from Aerospace – – • Data from Aerospace Corp. report* 188 data records 4 Operating Environments 8 Application Domains Our experiment – Challenging Application Difficulty: Signal Processing – Easy Application Difficulty: Test – Calculate estimating relationship (in Y = a. Xb form), standard error, and R 2 – Our hypothesis is that Easy Application Difficulty will have higher productivity than Challenging Application Difficulty * Gayek, Long, Bell and Larson, “Software Cost and Productivity Model, ” Aerospace Report ATR-2004(8311) -1, The Aerospace Corporation, El Segundo, CA, Feb 04 24 th International Forum on COCOMO and Systems/Software Cost Modeling 14

University of Southern California Center for Systems and Software Engineering Aerospace Project Data Analysis • Challenging Application Difficulty: Signal Processing – 40 records – Estimating relationship for effort • Effort = 11. 82 x (Size) 0. 93 • R 2 = 0. 7 • Standard Error = 0. 57 24 th International Forum on COCOMO and Systems/Software Cost Modeling 15

University of Southern California Center for Systems and Software Engineering Aerospace Project Data Analysis • Easy Application Difficulty: Test – 16 records – Estimating relationship for effort • Effort = 7. 46 x (Size) 0. 97 • R 2 = 0. 68 • Standard Error = 0. 53 24 th International Forum on COCOMO and Systems/Software Cost Modeling 16

University of Southern California Center for Systems and Software Engineering Aerospace Project Data Analysis • Using the estimating relationship to compare Easy and Challenging Application Difficulty – Size in KSLOC – Effort in PM – Productivity in SLOC/PM Very Easy: Test Challenging: Signal Processing Size Effort Productivity Effort Productivity 10 69. 62 143. 64 100. 60 99. 40 20 136. 38 146. 65 191. 68 104. 34 50 331. 70 150. 74 449. 43 111. 25 100 649. 74 153. 91 856. 28 116. 78 200 1272. 73 157. 14 1631. 46 122. 59 500 3095. 56 161. 52 3825. 25 130. 71 24 th International Forum on COCOMO and Systems/Software Cost Modeling 17

University of Southern California Center for Systems and Software Engineering COCOMO II Implications Productivity Groups may be used to to pre-set COCOMO II Parameters. This is also a work-in-progress. Pre-sets driven by Operational Context COCOMO II Parameters Very Unconstrained Constrained Very Constrained Extremely Constrained Unconstrained PRED H N L VL VL RESL H N L VL VL CPLX-Device VL N, L H VH XH PVOL L N H VH VH TIME N H VH XH XH STOR N H VH XH XH 24 th International Forum on COCOMO and Systems/Software Cost Modeling 18

University of Southern California Center for Systems and Software Engineering COCOMO II Implications Pre-sets driven by Application Difficulty Very Easy Nominal Challenging Very Challenging PRED H N L VL VL RESL H N L VL VL RELY VL L N H VH CPLX-Control VL N, L H VH XH CPLXComputations VL N, L H VH XH CPLX-Data VL N, L H VH XH CPLX-UI VL N, L H VH XH L N H VH VH Factors DATA 24 th International Forum on COCOMO and Systems/Software Cost Modeling 19

University of Southern California Center for Systems and Software Engineering COCOMO II Implications • Factors captured by local calibration of COCOMO constants A and B: – Security and Information assurance (given complexity is at least high) • Very Easy would have no security or information assurance requirements • Very Challenging would require the software to authenticate access, handle denial of service attacks, ensure data integrity, etc. – Components or Sub-system Interactions • Heterogeneous components require special interfaces to communicate with each other • Homogeneous components are generally 24 th International Forum on COCOMO and 20 Systems/Software Cost Modeling

University of Southern California Center for Systems and Software Engineering COCOMO II Implications • COCOMO Parameters not addressed: – – – Development Flexibility Team Cohesion Process Maturity • These are usually driven by Developed for Reuse business decisions Analyst Capability • May be indirectly driven by Difficulty or Context Programmer Capability Personnel Continuity Applications Experience Platform Experience Language and Tool Experience Use of Software Tools 24 th International Forum on COCOMO and 21 Systems/Software Cost Modeling

University of Southern California Center for Systems and Software Engineering Next Steps • The proposed approach to classifying software application productivities by Operational Context and Application Difficulties needs to be verified with the government agencies and industrial communities that will use the approach for software project cost estimates. – Hold workshops and working groups to fill in ratings for operational context and application difficulties. – Test approach by soliciting characterizations of applications and determining where their applications fit in the productivity group matrix 22 24 th International Forum on COCOMO and Systems/Software Cost Modeling

University of Southern California Center for Systems and Software Engineering For more information, contact: Thomas Tan thomast@usc. edu 626 -617 -1128 Questions? 24 th International Forum on COCOMO and Systems/Software Cost Modeling 23