Operations Research and the Role of Probability and Statistics in Military Analysis 21 February 2014 Dr. Rafael E. Matos WBB, Inc. (703) 448 -6081 x 108
Terminology: OR and OA • Operations Research (OR) – The academic discipline comprising a range of tools for analyzing operations for purposes of improving or optimizing various business or functional processes • Operations Analysis (OA) – The discipline of applying OR tools to specific subsets of functional processes, in your case military operations 2
How OA Can Help You • Provide analytic tools in helping to assess budget priorities • Oversee Directed Studies – Keeping tabs on outsourced studies • Manage your own study – How to run your own study • Review studies from other sources – Knowing strengths, weaknesses, good studies and bad ones • Synthesize an analytic position based on other studies – Putting the pieces together 3
Operations Analysis “classical” definition: “A scientific method of providing executive departments with a quantitative basis for decisions regarding the operations under their control” • Methods of Operations Research, Morse & Kimball, 1951 Decision Objective Inputs Array of facts: Quantitative comparison of the meaningful elements of the problem Subjective Inputs Largely provided by the decisionmaker: judgment and experience factors “The OA Domain” Accurately defining the Problem is critical 4
The Military focus has changed 9 -11 -2001 PRE – 911 POST – 911 Major Regional Conflict/ Major Theater War 2 MRC – 2 MTW CP PK PE HA/ DR C-Piracy CT Peace–keeping/ Enforcement Humanitarian Assistance/ Disaster Relief C-Terrorism Strategy and capabilities required for Spectrum of Conflict were subsets of 2 MRC/MTW force structure 5 Strategy and capabilities required for post-9/11 environment are not subsets of MCO force structure
When you change the way you look at things, the things you look at change… 6
An Example of Data Anscombe's Quartet comprises 4 data sets of 11 points each: I II IV x y x y 10 8 13 9 11 14 6 4 12 7 5 8. 04 6. 95 7. 58 8. 81 8. 33 9. 96 7. 24 4. 26 10. 84 4. 82 5. 68 10 8 13 9 11 14 6 4 12 7 5 9. 14 8. 74 8. 77 9. 26 8. 10 6. 13 3. 10 9. 13 7. 26 4. 74 10 8 13 9 11 14 6 4 12 7 5 7. 46 6. 77 12. 74 7. 11 7. 81 8. 84 6. 08 5. 39 8. 15 6. 42 5. 73 8 8 8 8 19 8 8 8 6. 58 5. 76 7. 71 8. 84 8. 47 7. 04 5. 25 12. 50 5. 56 7. 91 6. 89 For all four: • Mean of the x values = 9. 0 • Mean of the y values = 7. 5 • Equation of the least-squared regression line is: y = 0. 5 x + 3 • Sums of squared errors (about the mean) = 110. 0 • Regression sums of squared errors (variance accounted for by x) = 27. 5 • Residual sums of squared errors (about the regression line) = 3. 75 • Correlation coefficient = 0. 82 • Coefficient of determination = 0. 67 (F. J. Anscombe, "Graphs in Statistical Analysis, " American Statistician, 27 [February 1973], 17 -21) 7 What does this mean ? • Analysis of the data show that it is similar • But is it ? • Let’s look at the data when it is plotted
Completely Different Data Sets Using line plots reveals the differences among the data sets I III 16 R 2 = 0. 67 16 12 12 8 8 4 4 0 0 0 5 10 15 0 20 5 II 10 15 IV 16 R 2 = 0. 67 20 R 2 = 0. 67 16 12 12 8 8 4 4 0 0 0 5 10 15 20 0 5 10 15 This quartet is used as an example of the importance of looking at your data before analyzing it in Edward Tufte's book, The Visual Display of Quantitative Information. 8 20
History of OA • • • Tanker torpedoed off US coast - 1942 First wide use of OA methods was during the Battle of the Atlantic in World War II Anti-Submarine Warfare Operations Research Group (ASWORG) established as part of US Atlantic Fleet Headquarters to evaluate: “How to protect a large number of cargo ships against enemy submarines with limited escorts” Study of the ASW problem provided crucial insights regarding: – Convoy size - number of ships in each – Search techniques - how to best employ sensors to maximize the probability of detecting submarines – Screening - where to position escorts to put them in the best position to detect and counter submarine attacks Atlantic convoy at sea
Convoy Protection Problem Assumption: each ship in a convoy takes up a certain area that has to be defended by the escorts (1 mi 2) • • • 5 For a 25 -ship convoy, vulnerable perimeter is 5 X 4=20 miles If an escort can protect 2. 5 miles, this convoy requires 8 escorts 2 similar convoys would require 16 escorts • • • A 49 -ship convoy has a vulnerable perimeter of 7 X 4=28 miles and requires 11 escorts (11. 2) 96% increase in convoy size, but only 40% increase in vulnerable perimeter: requires 33% fewer escorts than same number of ships divided into 2 convoys 7 Application of relatively simple mathematics to a complex warfare problem 10
OA Today • The use of OA has expanded greatly beyond ASW and Naval Warfare to other services and the commercial sector – Department of Defense - uses OA to address the problem of which programs to fund in the face of a changing world with a limited amount of resources – Transportation Industry - OR tools and techniques are readily applicable to the problem of optimizing transportation networks and assets in different areas with varying conditions – Many recent business improvement schools of thought (Total Quality Management, Business Process Re. Engineering, Six Sigma) are all based on Operations Research theory • Military and civilian businesses leadership increasingly turning to analytic process to inform decisions 11
Example Naval Warfare OA Problem • Issue: How many ships should the Navy have in the future? – What types of ships should they be? “The Global Concept of Operations requires a fleet of approximately 375 ships that will increase our striking power from today's 12 carrier battle groups, to 12 Carrier Strike Groups, 12 Expeditionary Strike Groups, and multiple missile-defense Surface Action Groups and guided-missile submarines. These groups will operate independently around the world to counter transnational threats and they will join together to form Expeditionary Strike Forces—the "gold standard" of naval power—when engaged in regional conflict. ” Sea Power 21 The Navy's report provides few details about how many ships the service would have to buy each year to implement either the 260 - or 325 -ship plan--and thus how big a budget it would need for ship construction. Some combinations may not be affordable, feasible or mission effective 12
Spectrum of Analysis Basic Analysis: Investigative Journalism • What or Who is being studied? – Objects or Methods – Materiel or Doctrine Who, What, When, Where, Why, and How Who What • When and Where is it being studied? – Micro or Macro level – Phenomenological or Campaign Where Why How • How and Why is it being studied? – Simple evaluation of one – A choice between two or more – An optimal mix of several
Who and What Who or What is going to be analyzed Military Application: • What is the mission that needs to be accomplished? • Can the mission be accomplish with the current constraints? • Is there a better way to use available resources to accomplish the mission? • If additional resources are applied how will it improve the ability to accomplish to this mission? Example Problem Statement: “How many ships should the Navy have in 2020 -2030? ” 14
When and Where Which missions are important to the overall campaign ? CAMPAIGN What functions are critical in those missions? What parameters are critical to those functions? Does it matter? (SO WHAT? ) MISSION How does it work? ENGAGEMENT ENGINEERING Doctrinal studies are usually conducted at Campaign or Mission levels, and sometimes at the Engagement level 15 Can you use it? What is it that does it? Technical studies are usually conducted at Engineering levels or below, and sometimes at the Engagement level
Why and How • Straight Evaluation of one subject against a set of benchmarks – Grades – Stock price or profit margin • Choice between two or more subjects – Fly-off – Competition • Optimization – Determine the optimal mix of resources to apply to a specific (fixed) situation • Force Structure – Trade Studies measure the effect of different levels of capability in one area or attribute versus another • Cost / performance trade-offs determine the balance between design alternatives based on both performance and cost 16
Total Spectrum of Analysis Optimization Evaluation Material Mission Evaluation Doctrine Mission Evaluation Material Engagement Evaluation Doctrine Engagement Evaluation Material Engineering Evaluation Doctrine Engineering Engagement Doctrine Campaign Evaluation Material When / Where Material Campaign Evaluation Who / What 17 When / Where Campaign Choice Evaluation Optimization Choice Mission H ow /W hy Who / What hy H ow /W • Regardless of where a study falls in the who / what / when / where / how / why spectrum, each has the same characteristics – Foundation is based on scientific methods – Features logical, common study components and stages
Foundation of OA • Regardless of subject, level or method, all Operations Analysis is based on scientific methods • A more thorough definition: – “. . . the application of scientific knowledge toward the solution of problems which occur in operational activities (in their real environment). Its special technique is to invent a strategy of control by comparing, measuring, and predicting possible behavior through a scientific model of [the] activity” Naval Operations Analysis, USNI Press, 1984 • • • “Solution of problems” - the study Objective “Operational activities” - the study Subject “Real environment” - the study Context “Comparing” - the study Basis of Comparison “Measuring” - the study Metrics 18 Breaking a problem down into these factors is the first step in solving it
A Typical OA Study Plan 2. Define Studies 3. Execute Studies 1. Establish Plan Foundation Defense Planning Guidance Prioritize Study Areas Establish Study Spectrum Develop Basis of Comparison Select Metrics May 2001 Define Objective This type of plan is scalable to small and large issues Generate Data 5 a. Refine / Focus Plan 4. Synthesize Results Establish Study Subject Develop Alternatives Define Context Is the Study still supporting the Objective? 5 b. Check Impact What Happened? Did it support the Objective? 19 5. Facilitate the Decision Process Data Generate Information Review with Decision-Makers
Descriptive Statistical Measures • Graphical displays provide a better sense of the data by transferring numerical information into a picture - another way is to use Descriptive Measures • Measures of Central Tendency (or location) – Descriptions on where the middle lies – Generally attempt to estimate the underlying population mean – Examples include the mean, median & mode • Measures of Dispersion (or spread) – Descriptions on the spread of the data – Examples include the range, variation, standard deviation & percentile Central Tendency looks for the middle X XX 20 X X X XX X X Dispersion looks for the spread X XX X X
Interpreting Statistics • Sampling distribution is obtained by computing statistics for a large number of samples drawn from the same population Student Group A AVG = 77 Student Group B AVG = 77 Student Group C AVG = 77 – Measures of Central Tendency / Location • Mean • Median • Mode – Measures of Dispersion • • Range Variance Standard Deviation Percentile • Once observations have been made and statistics derived, what can we infer about the population from which the sample was collected? 21
Trend Analysis Stock Price over time C 1/C 2 Readiness and Top 5 Manning • An element of Time Series Analysis • An investigation of data (i. e. system performance) over a spectrum of one variable – usually time • Trend Analysis can estimate or predict… – within the data set (interpolation) – outside the data set (extrapolation)
Analysis with Regression • Regression Analysis evaluates the relationship between a variable of interest (a dependent or response variable) and one or more independent or predictor variables – Distance from target and kill probability – Years of training and promotion rates – Time and target movement • Often used to predict the response variable from the knowledge of the independent variables – Can evaluate either a linear relationship or more complex one • Regression methodologies are complicated, but the EXCEL spreadsheet package can perform both simple and complex regression analysis quickly A quick indicator is the coefficient of determination (R 2) in the results. You want this number to be close to one. 23
Regression: Interpolation Example Missile X Target Aspect vs. Miss Distance • Test performance data set for Missile X • In addition to the miss distance data, the target aspect at launch time was also recorded • The chart shows the relationship between the target aspect at launch and the missile miss distance at intercept – For example for Test # 7, target aspect at launch was 20 degrees and miss distance was 15 feet Target Aspect (Degrees) Miss Distance (Feet) • Is there a mathematical relationship between these two variables such that for any given target aspect one can predict the miss distance? • What would you say the miss distance would be for a target aspect of 25 degrees?
Regression: Extrapolation • Extrapolation estimates or predicts outside the data set – When time is the independent variable, predicts system performance in the future based on how it has functioned in the past • Validity depends on… – …the consistency of the original data set – …the assumption that current conditions will continue outside the data set • Regression works well for interpolation, but not necessarily for extrapolation – R 2 value (a measure of how well the line fits the data) only applies within the data set • When extrapolating, keep your window short and use some common sense How far out would you extrapolate this data set?
Extrapolation Example • The day before the final launch of the Challenger Space Shuttle, Morton Thiokol faxed 13 charts to NASA opposing the launch due to possible O-ring failure. – In the charts, the following information was scattered throughout and summarized in the conclusions: • • O-Ring damage at 53 F O-Ring damage at 75 F Temperature on 1 -28 -86 launch initially estimated to be 29 F - 38 F RECOMMENDATION: O-Ring temperature must be > 53 F at launch – NASA officials recommended reconsideration because: • Analytical graphics failed to communicate the magnitude of risk that was in fact present – Reassessing the situation, Morton Thiokol stated – The evidence presented by their engineers was inconclusive, – Cool temperatures were not linked to O-ring problems, and – They now favored a launch the next day Example from Visual Explanations, Edward R. Tufte, Graphics Press, 1997 26
Extrapolation Example (Cont. ) Analysis illustrates risk with launch day temperature forecast of 26 -29 degrees F R 2 is the square of the residual errors from the line How well the line approximates the points 27
Why is basic probability theory important? – Probability is the basis for statistics, data analysis, hypothesis testing, sampling, surveys, experimentation, prediction … other useful tools in the experimentation trade. – Decision making under uncertainty is largely based on application of statistical data analysis for probabilistic risk assessment of decisions …probability provides theoretical framework to be certain of how uncertain you are. Probability theory provides necessary foundations for the ability to generate statistics 28
General Concepts • Basic Probability Theory – What is it? • That branch of mathematics that is concerned with calculating the likelihood of outcomes of experiments -- modeling the phenomenon of chance or randomness • That way of thinking in which we make inferences from a sample to a population, and then measure the accuracy of those inferences Probability of a strike by a tropical storm Katrina 29
Operational Issues How many enemy aircraft are detected in a four hour period? Number of Enemy Aircraft Detected in a four hour period Poisson Distribution How reliable is my Air Search Radar? Probability that radar will fail Exponential Distribution Are there any anti-ship mines located in this area? Mine Location in Chokepoint Uniform Distribution 30 What are the chances that I can destroy an inbound missile raid with my overhead CAP? Inbound Missile Raid destruction Binomial Distribution What is the Pk for my missiles? Single Shot missile kill Bernoulli Trial How well does my Flight deck crew perform? Probability that flight deck crew will maintain advancement rates Normal Distribution
General Concepts • What are statistics? – Classical definition: a science of inferring generalities from specific observations , i. e. , a way of working with numbers to answer questions about various phenomena 1 • E. g. Analyzing missile test firings to evaluate overall missile performance • Goals of statistics are to: – Descriptive: Describe a set of numbers, and – Inferential: Make accurate inferences about groups based upon incomplete information • E. g. , Does the type of missile meet key performance parameters • Making accurate inferences requires groundwork. In order to reach an informed conclusion, one must: – Gather data (numerical information); then, – Organize it (sometimes in a graphic); then, – Analyze it Statistics use “observed data” to draw conclusions about an “unobserved population” 1 -Dixon and Massey, Introduction to Statistical Analysis 31
Missile Ao. A Dispersion Calculations Let’s look at the spread miss distance dispersion data for our 3 candidate missile systems. Accuracy of 5 m with the spread data? Accuracy of 10 m with spread data? Least amount of dispersed shots? Greatest amount of dispersed shots? Miss distances in meters (m) 32
Value Statement You can build a fairly comprehensive warfighting model using Probability Distributions and Statistical Analysis 33
Operations Research and the Role of Probability and Statistics in Military Analysis 21 February 2014 Dr. Rafael E. Matos WBB, Inc. (703) 448 -6081 x 108
Скачать презентацию Operations Research and the Role of Probability and
3ad212cf10ea420245da0f0af42ae889.ppt