Multinational Experiment 4 on Effects Based Approach to

Multinational Experiment 4 on Effects Based Approach to Operations A CFEC Report Philip S. E. Farrell, Ph. D. A presentation for IAMWG Meeting Friday 11 May 2007 Defence Research and Development Canada Recherche et développement pour la défense Canada

Presentation • Introduction – Human Experimentation • MNE 4 CFEC Report • Discussion – Can war-fighting experimentation be done in a controlled laboratory setting? Defence R&D Canada • R & D pour la défense Canada

Introduction Not “The Bear” Now “The Ball of Snakes” New concepts need to be tested Defence R&D Canada • R & D pour la défense Canada

Scientific Method High School Science Report – Background – Purpose – Hypothesis (related to concept) – Analysis – Apparatus – Procedures – Results – Discussion – Conclusions Defence R&D Canada • R & D pour la défense Canada

Human Experimentation Defence R&D Canada • R & D pour la défense Canada

Human Experimentation Job Similarity Index concept Crew Selection Experiment (2006) TTPs Recommendations Defence R&D Canada • R & D pour la défense Canada

Human Experimentation in MNE 4 • Too many people? • Too many variables? • Too many confounds? • Too many constraints? Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment Effects Based Approaches to Operations Refinement and Assessment of EBAO POT Desired End State + _ Planning desired (desired actions effects) Execution actions + + disturbances Operational Environment Current situations World states Assessment (effects) Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment (CONOPS 0. 9) Planning RG Rs Execution Re + ESA ED _ Cs Ra + ADRM SPR PREPO _ CA “situation” and “effects” comparators IDI “action” comparators Ce EBAP + Ca_ C&C A + + D + + W MMA Analysis Function KBD, SOSA, KM AAMOP S AAMOE Assessment QCE Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment Aim Experiment Objectives Hypothesis Critical Operational Issues Measures Data Experiment Execution Data Collection Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment Competing Aims! • to explore concepts and supporting tools for EBAO • to address the Canadian MNE 4 objectives Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment • Canadian Objectives – Refine, assess, and analyze elements of EBAO that Canada and the CF could apply to achieve integrated strategic effort in operations. – Determine whether EBAO represents a means of applying the Canadian 3 -D approach to operations – Develop and refine a Multinational Knowledge Management Plan that can be tailored and applied to meet national operational requirements Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment The Hypothesis of this experiment was: If an operational commander and staff employ effects based operations, then they will have the ability to: Create a holistic understanding (KBD) Develop and refine an operational plan (EBP) Harmonize the full spectrum of DIME planning (MNIG) Coordinate, direct, and monitor operations (EBE) Assess actions, effects, and end state (EBA) Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment • Experimental Design Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment Types of Analyses • Observations – JOT (Joint Observation Tool) – JDCAT (Joint Battle Centre Data Collection Analysis Tool) Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment Types of Analyses • User surveys and network data Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment Types of Analyses • Box Plots (KBD) Figure C. 1 Distribution of correct responses for SA surveys, by organization Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment Types of Analyses • Multi-Dimensional Scaling (KBD) Figure C. 4: Multi-dimensional scaling of KBD 3 answers by Group Reference Point (KBD 3) Reported same incorrect answers Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment Types of Analyses • Count data (KM) Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment Types of Analyses • Whitney – U (Culture) Q 3. 2 b. To what degree is the decision of the leader contingent on the work and advice of others? (this question was for those who did not answer Leaderless Team in Q 3. 2 a) Mann-Whitney U = 2212, p <. 07 Not at all Little Somewhat Much Completely Not applicable 2 NRF 0% 5% 44% 36% 11% 4% 2 = 38. 18, p <. 001 CTF 1% 8% 19% 54% 15% 3% 2 = 110. 13, p <. 001 Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment Types of Analyses • Vector Method (Common Intent) Date 28 Feb Name Vector (7, 5, 6, 7) v 1 2 Mar v 2 (7, 6, 5, 7) 7 Mar v 3 (7, 6, 7, 7) 9 Mar v 4 (7, 7, 7, 6) 14 Mar v 5 (7, 6, 7, 7) 16 Mar v 6 (7, 7, 7, 6) Reference r (7, 7, 7, 7) Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment Types of Analyses • Vector Method (Common Intent) Table 3. Angle between vector pairs in degrees (Duncan & Farrell, 2007) v 1 v 2 v 3 v 4 v 5 v 6 r v 1 v 2 0 6. 4 4. 7 10. 0 7. 6 v 3 6. 4 0 7. 8 10. 0 7. 6 v 4 4. 7 7. 8 0 6. 0 0. 0 6. 0 3. 7 v 5 10. 0 6. 0 0. 0 3. 7 v 6 4. 7 7. 8 0. 0 6. 0 3. 7 Defence R&D Canada • r 10. 0 6. 0 0 3. 7 7. 6 3. 7 0 R & D pour la défense Canada

MNE 4 Experiment • Results, Discussion, Conclusions, and Recommendations are just as fuzzy as the objectives and hypothesis. Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment Insights related to Canadian Objectives • The notion of planning, execution, and assessment in an iterative fashion until the Current State reaches the End State is a robust concept. • The relationship between MNIG, EBAO, and the 3 D concepts requires further development and experimentation. • Some of the basic KM and KBD definitions and concepts still have not been resolved. Defence R&D Canada • R & D pour la défense Canada

MNE 4 Experiment Recommendations • More research is required! Defence R&D Canada • R & D pour la défense Canada

Discussion Can war-fighting experimentation be done in a laboratory controlled setting? Defence R&D Canada • R & D pour la défense Canada

Discussion Military buy-in vs. Scientific reputation • Do you need SMEs to conduct the experiment? Defence R&D Canada • R & D pour la défense Canada

Discussion Human Experimentation Ethics • Is voluntary consent necessary? Defence R&D Canada • R & D pour la défense Canada

Discussion political Considerations • Who makes the decisions? Defence R&D Canada • R & D pour la défense Canada

Discussion Concept complexity • Are there too many variables? Defence R&D Canada • R & D pour la défense Canada

Discussion Experimental Design • Whatever happened to Hypothesis-testing? Defence R&D Canada • R & D pour la défense Canada

Summary • Human Experimentation is not new. • MNE 4 has refined our processes for war-fighting experimentation. • We need to pay attention to the scientific method if our recommendations are to be credible Defence R&D Canada • R & D pour la défense Canada

Questions Defence R&D Canada • R & D pour la défense Canada