Software Decision Analysis Techniques Chapters 10 -20 in

Software Decision Analysis Techniques Chapters 10 -20 in Software Engineering Economics (SEE) Updated to Med. FRS Barry Boehm, USC-CSSE CS 510, Fall 2017 Edited & presented by Jim Alstad

Topics in Software Decision Analysis Techniques • Economics – The study of how people make decisions in resource-limited situations • Macroeconomics (not covered in this course) – Inflation, taxation, balance of payments • Microeconomics – Make-or-buy, pricing, how much to build • Software economics decisions – Make-or-buy: Software Product – How many options to build? – Which DP architecture to use? – How much testing (prototyping, specifying) is enough? – How much software to re-use? – Which new features to add first? Fall 2017 © USC-CSSE 2

Outline of The Four Lectures (Chapters in SEE) • 10 -12. Context; Med. FRS TPS Example; Cost. Effectiveness Analysis – Models, optimization, production functions, economies of scale decision criteria • 13 -15. Multiple-Goal Decision Analysis I – Net value, marginal analysis, present value, figures of merit. • 16 -18. Multiple-Goal Decision Analysis II – Goals as constraints, constrained optimization, system analysis, unquantifiable goals • 19 -20. Dealing with Uncertainties – Expected value, utility functions, statistical decision theory, value of information Fall 2017 © USC-CSSE 3

Master Key to Software Engineering Economics Decision Analysis Techniques All decision criteria (DCs) convertible to present $? Yes Use standard optimization, net value techniques (chapters 10, 13) Is outcome of decision highly sensitive to assumptions? (chapter 17) No All Non-$ DCs expressible as constraints? Yes No All Non- $ DCs expressed as single “benefit” criterion? Yes Use standard constrainedoptimization techniques (chapter 16) Use cost-benefit (CB) decision making techniques (chapters 11, 12) Yes Find, use less sensitive solution End No Yes All Non-$ DCs quantifiable? No Fall 2017 Use figure of-merit techniques, CB techniques (chapter 15) Use techniques for reconciling nonquantifiable DCs (chapter 18) When $ are a mix of present and future cash flows When some DCs involve uncertainties © USC-CSSE Use present value techniques to convert future $ to present $ (chapter 14) Use statistical decision theory techniques (Chapters 19, 20) 4

Chapters 10 -12 Cost-Effectiveness Analysis • • Introduction Example: Med. FRS transaction processing system (TPS) Performance models Cost-performance models Production functions: economies of scale Cost-effectiveness decision criteria Summary Fall 2017 © USC-CSSE 5

Med. FRS TPS Context • Current Ensayo IT processing capacity less than needed in crises – Top performance less than 1000 transactions/second (tr/sec) – Need 2000 tr/sec soon – Need growth to 4000 tr/sec • COTS server capability can provide over 2000 tr/sec, but mostly for business applications – And can’t achieve 4000 tr/sec • Med. FRS considering developing or outsourcing its own server software Fall 2017 © USC-CSSE 6

Med. FRS TPS Architecture variant on SEE architecture (Fig 10 -1) FRS Devices … Med. FRS communications hub vehicles (CHVs) … … … … … Ensayo Regional Server Area Concentrators … Medical DB Fall 2017 © USC-CSSE N DB Server 1 Patient DB 7

Med. FRS TPS Concept of Operation • Med. FRS devices send patient trauma information to CHVs • CHVs validate information and forward it to area concentrators at regional Med. FRS server – Usually about 10 local concentrators per area • N area concentrators use Med. FRS server to determine best near-term treatment – Send back to device operators via CHVs – Multiprocessor overhead due to resource contention, coordination Fall 2017 © USC-CSSE 8

COTS vs. New Development Cost Tradeoff: Med. FRS TPS • Build special version of Med. FRS regional server systems functions – To reduce COTS server software overhead, improve transaction throughput • Server systems software size: 20, 700 SLOC – Transaction prioritization, status monitoring • COTS license tradeoffs vs. number of area concentrators N – Need 10 N licenses for CHVs – $1 K each for acquisition, $1 K each for 5 -year maintenance Fall 2017 © USC-CSSE 9

COTS/New Development Cost Tradeoff Analysis COTS $K • 100 10 N 200 + 20 N 250 + 20 N 450 + 40 N 606 Included Not applicable 151 757 250+20 N 1007 + 20 N Software – – • • New Development $K Cost to acquire Integrate & test Run-time licenses 5 -year maintenance Server Total Fall 2017 © USC-CSSE 10

COTS/New Development Cost Tradeoff COTS 1200 Life Cycle Cost, $K 1000 800 New 600 400 200 10 20 30 40 50 Number of Area Concentrators, N • Now, we need to address the benefit tradeoffs Fall 2017 © USC-CSSE 11

Med. FRS TPS Decision Problem 1: How Many Regional Concentrators in Server? Performance Parameters (for COTS OS) N, number of processors S, processor speed (MOPS/sec) P, processor overhead (MOPS/sec) M, multiprocessor overhead factor N=? S = 1000 P = 200 M = 80 [overhead=M(N-1) MOPS/sec] T, transaction processing load (MOPS/TR) T = 1. 0 Performance (TR/sec) E(N) = MOPS/sec available for processing MOPS/TR E(N) = N[S-P-M(N-1)] T Fall 2017 © USC-CSSE 12

TPS Performance, E(N) = MOPS/sec available for processing MOPS/sec required N E(N) 0 0 1 800 2 1440 3 1920 4 2240 5 2400 6 2400 7 2240 11 0 = N [ 1000 -200 -80(N-1)] 1. 0 = N [1000 -200+80)-80 N 2 = 880 N – 80 N 2 = 80 N(11 -N) 0= d. E(N) d. N = 880 - 160 N* = 880 N* = 5. 5 E(N*) = 2440 Fall 2017 © USC-CSSE 13

TPS throughout: E(N) versus # of processors, N E(N) 1 800 2 1440 3 1920 4 2240 5 2400 6 2400 7 2240 8 E(N) (tr/sec) N 1960 Number of processors N Fall 2017 © USC-CSSE 14

TPS Decision Problem 2: Which Operating System? Option Cost ($k) Multiprocessor overhead factor M O/H = M(N-1) Option B: A B (Accept Available OS) (Build New OS) 450 + 40 N 1007 + 20 N 80 40 E(N) = N(1000 -200 -40(N-1)) = 840 N-40 N 2 1. 0 = 40 N(21 -N) For N = 10, E(N) = 40(10)(11) = 4400 Fall 2017 © USC-CSSE 15

Cost-Effectiveness Comparison: TPS Options A, B Option B 5000 E(N) = 40 N(21 – N) 4000 C(N) = 1007 + 20 N 3000 2000 1000 0 0 450 500 650 1000 1207 1500 Cost C, $K Fall 2017 © USC-CSSE 16

Production Function Achievable output = F (input consumed) -Assuming only technologically efficient pairs: • No higher level of output achievable, using given input Output X Input PF is nonnegative PF is nondecreasing Fall 2017 © USC-CSSE 17

Segments of Typical Production Function Output Investment Fall 2017 High payoff © USC-CSSE Diminishing returns 18

Natural speech input Value of software product to organization Tertairy application functions Animated displays Secondary application functions User amenities Main application functions Operating System Investment Fall 2017 Basic application functions Data management system High-payoff Cost of software product © USC-CSSE Diminishing returns 19

Software “Gold-Plating” (Adding More Than Is Necessary) • Frequently Gold-Plating – Instant response – Pinpoint accuracy – Unbalanced systems – Agents with attitudes – Animated displays – “everything for everybody” • Usually Not Gold-Plating – – Humanized input – Humanized output Modularity, info. hiding – Measurement, diagnostics • Sometimes Gold-Plating – Highly generalized control, data structures – Sophisticated command languages – General-purpose utilities – Automatic trend analysis USC-CSSE Fall 2017 © 20

Modular Transaction Processing System Module 1 E(2 x 3) = 2 x E(3) = 3840 vs. 2400 P 11 P 12 1 P 13 Trans. in Processed Transaction out 2 P 21 P 22 P 23 Fall 2017 © USC-CSSE Module 2 21

Software Project Diseconomies of Scale SLOC Output PM = C (KSLOC) 1+x Person – Months Input • The best way to combat diseconomies of scale is to Reduce the Scale Fall 2017 © USC-CSSE 22

Cost-Effectiveness Decision Criteria 1. 2. 3. 4. 5. 6. Fall 2017 Maximum available budget Minimum performance requirement Maximum effectiveness/cost ratio Maximum effectiveness – cost difference Return on investment (ROI) Composite alternatives © USC-CSSE 23

E (tr/sec) Production Functions for TPS Options A, B B 4400 P Q 2400 A Z Y X 450 Fall 2017 650 1007 © USC-CSSE 1207 24

Maximum Effectiveness/Cost Ratio 2400 R E (tr/sec) 2000 L Eff/Cost = 8 1600 1200 K 800 Eff/Cost = 3. 69 400 100 Fall 2017 200 300 400 © USC-CSSE 500 600 C, $K 700 25

Maximum Effectiveness-Cost Difference 5000 4400 B-Build new OS 4000 3000 Throughput E(TR/sec) 2000 3193 A-Accept available OS 2400 1750 1207 1000 650 0 500 1000 Cost C, $K Fall 2017 © USC-CSSE 1500 26

Return on Investment ROI 5 4 ROI = E-C C 3 2. 67 2. 60 2 A B 1 0 500 Fall 2017 1000 Cost C ($K) © USC-CSSE 1500 27

Production Function for TPS Composite Alternative E E (r/sec) 4400 2400 450 650 1007 1207 C, $K Fall 2017 © USC-CSSE 28

Summary – Cost-Effectiveness Analysis • Microeconomic concepts help structure, resolve software decision problems – – Cost-effectiveness Production functions Economies of scale C-E decision criteria • No single decision criterion dominates others – Each is best for some situations – Need to perform sensitivity analysis: • Slightly altered situation doesn’t yield bad decision Fall 2017 © USC-CSSE 29