Expert COSYSMO Update Raymond Madachy rjmadach nps edu USC-CSSE

Expert COSYSMO Update Raymond Madachy rjmadach@nps. edu USC-CSSE Annual Research Review March 17, 2009

Expert COSYSMO Introduction • An expert system tool for systems engineering risk management based on the Constructive Systems Engineering Cost Model (COSYSMO) [Valerdi 2005] – Automatically identifies project risks in conjunction with cost estimation similar to Expert COCOMO [Madachy 1997] and provides related advice – Supports project planning by identifying, categorizing and quantifying system-level risks – Supports project execution with automated risk mitigation advice for management consideration • Risk situations are characterized by combinations of cost driver values indicating increased effort with a potential for more problems – Includes 98 risk conditions • Simultaneously calculates cost to enable tradeoffs with risk https: //diana. nps. edu/MSAcq/tools/Expert. COSYSMO. php http: //csse. usc. edu/tools/Expert. COSYSMO. php 2

Method • Analyzes patterns of cost driver ratings submitted for a COSYSMO cost estimate against pre-determined risk rules – Identifies individual risks that an experienced systems engineering manager might recognize but often fails to take into account – Helps users determine and rank sources of project risk. With these risks, mitigation plans are created based on the relative risk severities and provided advice 3

Method (cont. ) • COSYSMO cost factor combinations used as abstractions formulating risk heuristics – E. g. if Architecture Understanding = Very Low and Level of Service Requirements = Very High, then there is a high risk • Since systems with high service requirements are more difficult to implement especially when the architecture is not well understood • Elicitation of knowledge from systems engineering domain experts in CSSE-sponsored workshops – Survey used to identify and quantify risks • Devised knowledge representation scheme and risk quantification algorithm • All risk rules are fired when the effort multipliers of both cost factors involved are > 1. 0 – Can be more granular for gradations of risk • Recent extension for risk mitigation advice 4

Risk Conditions 5

Risk Taxonomy and Weighting Project Risk Product risk Process risk Personnel risk Platform risk #categories #category risks Project Risk= risk leveli , j * effort multiplier p roducti , j å j= 1 where risk level = 1 2 4 å i= 1 moderate high very high effort multiplier product= (driver #1 effort multiplier) * (driver #2 effort multiplier). . . * (driver #n effort multiplier). 6

Expert COSYSMO Inputs 7

Expert COSYSMO Outputs 8

New Risk Mitigation Advice • All risks triggered when both related effort multipliers are > 1. 0 • See handout for review 9

Size Risk Elaboration • Need to establish size range thresholds for risk rules • COSYSMO size distribution: • Min = 82, Max = 17, 763 equivalent requirements • Proposed ranges – Small: – Medium: – Large: < 5, 000 equivalent requirements between 5, 000 and 15, 000 equivalent requirements > 15, 000 equivalent requirements • See handout on size related risks 10

Finer Assignment of Risk Levels • Different severity risks would entail contours of different strength • Some risk combinations may require re-interpretation of the nominal boundaries

Current and Future Work • • • Scaling the risk summary outputs for each category and defining ranges for low, medium and high risks Create more granular risk quantification rules Consider 3 -way risk interactions Add rules to detect COSYSMO input anomalies Systems engineering risk data from industrial projects will be analyzed to enhance and refine the technique – Perform statistical testing • Domain experts from industry and government will continue to provide feedback and clarification – Supporting surveys and workshops will be continued • Integrating alternate risk uncertainty approaches into a more complete risk management framework – Recently added Monte-Carlo analysis 12

Tasks So Far • COSYSMO workshop experts have identified and prioritized risks, and provided advice • Researchers devised the risk taxonomy and its weighting scheme Today • We invite your comments on all • Help us review and complete the advice 13

Assignments • • Dan/Beth pp. 1 -2 Jared/Stan p. 3 Mauricio/Garry p. 4 Miles/Ricardo p. 5 and size risks 14

References • R. Madachy, Heuristic Risk Assessment Using Cost Factors, IEEE Software, May 1997 • Valerdi R. , The Constructive Systems Engineering Cost Model (COSYSMO), Ph. D Dissertation, University of Southern California, Los Angeles, CA, May 2005 • http: //csse. usc. edu/tools/Expert. COSYSMO. php 15