Ultra-Large-Scale ULS Systems Their Impact on Technology

Ultra-Large-Scale (ULS) Systems & Their Impact on Technology & Society Dr. Douglas C. Schmidt d. schmidt@vanderbilt. edu www. dre. vanderbilt. edu/~schmidt Institute for Software Integrated Systems Vanderbilt University Nashville, Tennessee

The Road Ahead CPU & network performance has increased by orders of magnitude in past decades 10 Megahertz to 4+ Gigahertz Extrapolating these trends another decade or so yields • ~10 Gigahertz desktops • ~100 Gigabits/sec LANs • ~100 Megabits/sec wireless • ~10 Terabits/sec Internet backbone Unfortunately, software quality & productivity hasn’t improved as rapidly or predictably as hardware – especially for mission- & safetycritical systems 1, 200 bits/sec to 10+ Gigabits/sec

Example Mission- & Safety-Critical Systems

Why Hardware Improves So Consistently Advances in hardware & networks stem largely from R&D on standardized & reusable APIs & protocols x 86 & Power PC chipsets TCP/IP

Why Software Fails to Improve as Consistently In general, software has not been as standardized or reusable as hardware, especially for DRE systems Proprietary & Stovepiped Application & Infrastructure Software Application Software Standard/COTS Hardware & Networks Application Software

What Makes Software Hard? “Human Nature” Technical Complexities Accidental Complexities • Low-level APIs & debug tools • Organizational impediments • Algorithmic decomposition • Economic impediments Inherent Complexities • Administrative impediments • Political impediments • Function & quality property assurance • Scheduling & synchronization • Psychological impediments • Deadlock • etc. … www. dre. vanderbilt. edu/~schmidt/reuse-lessons. html

Key Challenges for Software Developers & users of software face challenges in multiple dimensions Logical View Process View Physical View Development View

Key Challenge for Software Developers Determining units of abstraction for system (de)composition, reuse, & validation Logical View • Popular technologies & tools provide inadequate support for • Checking pre-/post-conditions & invariants • Specifying, analyzing, & optimizing dependencies • Expressing design intent more clearly using domain concepts

Our Solution Approach Devising composable abstractions whose interfaces & Qo. S properties can be specified/analyzed/optimized via metadata Logical View 18% reduction Total Footprint 45% reduction www. dre. vanderbilt. edu/~schmidt/IEEE-RTAS-08. pdf

Key Challenge for Software Developers • Popular technologies & tools provide inadequate support for • Configuring & customizing components for application requirements & run-time environments • Automated optimized mapping of components onto nodes in target environments Physical View Integrating/deploying diverse new & reusable application components in a networked environment to ensure endto-end Qo. S requirements

Our Solution Approach Physical View Model-driven development & analysis techniques for optimizing, verifying, & automating the deployment & configuration process Gigabit Ethernet www. dre. vanderbilt. edu/~schmidt/JSS-06. pdf

Key Challenge for Software Developers • Popular technologies & tools provide inadequate support for • Identifying & reducing performance & robustness risks earlier in system lifecycle • Satisfying multiple (often conflicting) Qo. S demands • e. g. , secure, real-time, reliable • Satisfying Qo. S demands in face of fluctuating/insufficient resources • e. g. , mobile ad hoc networks (MANETs) Devising execution architectures, concurrency models, & communication styles that ensure multi-dimensional Qo. S & correctness of new/reusable components Process View

Our Solution Approach Skoll Server(s) Distributed continuous quality assurance techniques run on virtual computing grid subtask res. subtask req. register subtask client kit Process View Intelligent Steering Agent Configuration Automatic (ISA) Model Adaptation strategies Visualization characterization Subtask Code Adaptation e. g. , scoreboard e. g. , classification trees e. g. , nearest neighbor. Configuration www. dre. vanderbilt. edu/scoreboard Strategies … Intelligent Steering Agent Skoll clients(s) Model Subtask Results Output: Subtask Client Characteristic www. dre. vanderbilt. edu/~schmidt/IEEE-TSE-07{a, b}. pdf

Key Challenge for Software Developers • Popular technologies & tools provide inadequate support for avoiding • Cyclic dependencies, which make unit testing & reuse hard • Excessive link-time dependencies, which bloat the size of executables • Excessive compile-time dependencies, where small changes trigger massive recompiles (De)composing systems into reusable modules (e. g. , packages, subsystems, libraries) that achieve/preserve required Qo. S properties Development View

Our Solution Approach Standards-based, General-purpose, Layered Middleware Architecture Customized Middleware Stack Feature-Oriented CUStomizer (FOCUS) Average end-to-end measures improved by ~43% Development View Feature-oriented specialization optimization tool, e. g. , layer folding, deployment platform, memoization, constant propagation, etc. www. dre. vanderbilt. edu/~schmidt/ACM-Euro. Sys-06. pdf

New Challenges: Ultra-Large-Scale (ULS) Systems A ULS System has unprecedented scale in some of these dimensions: • Lines of code • Amount of data stored, accessed, manipulated, & refined • Number of connections & interdependencies • Number of hardware elements • Number of computational elements • Number of system purposes & user perception of these purposes • Number of routine processes, interactions, & “emergent behaviors” • Number of (overlapping) policy domains & enforceable mechanisms • Number of people involved in some way • … ULS systems are socio-economic ecosystems comprised of software-intensive systems, people, policies, cultures, & economics ULS systems report is available from www. sei. cmu. edu/uls

Example Emerging ULS Systems ULS systems are systems of systems at Internet scale ULS system examples from Linda Northrop’s OOPSLA 2006 keynote talk

Scale Changes Everything Characteristics of ULS systems that arise because of their scale include • Decentralization • Inherently conflicting, unknowable, & diverse requirements • Continuous evolution & deployment • Heterogeneous, inconsistent, & changing elements • Erosion of the people/system boundary • “Normal” failures • New paradigms for acquisition & policy • These characteristics appear in some of today’s systems, but in ULS systems they dominate • These characteristics undermine the assumptions that underlie today’s technology approaches, rendering incremental solutions inadequate

Serialized Phasing is Common in ULS Systems System infrastructure components developed first Level of Abstraction Application components developed after infrastructure is sufficiently mature Software Lifecycle Timeline

Serialized Phasing is Common in ULS Systems Level of Abstraction System integration & testing is performed after application development is finished Integration Surprises!!! Software Lifecycle Timeline

Complexities of Serialized Phasing Level of Abstraction Still in development Ready for testing Complexities • System infrastructure cannot be tested adequately until applications are done Development Timeline

Complexities of Serialized Phasing Level of Abstraction End-to-end performance of critical path? System bottleneck? Complexities • System infrastructure cannot be tested adequately until applications are done • Entire system must be deployed & configured (D&C) properly to meet endto-end Qo. S requirements • Existing tools & platforms have poor support for realistic “what if” evaluation Development Timeline Qo. S needs of components in ULS systems often unknown until late in lifecycle

Promising Approach for ULS System Challenges: System Execution Modeling (SEM) Tools to express & validate design rules • Help applications & developers adhere to system specifications at design-time Tools to ensure design rule conformance • Help properly deploy & configure applications to enforce design rules throughout system lifecycle Tools to conduct “what if” analysis • Help analyze Qo. S concerns prior to completing the entire system, i. e. , before system integration phase SEM tools should be applied continuously when developing software elements

SEM Tool Approach for Planning Aspect Approach • Develop Component Workload Emulator (Co. Work. Er) Utilization Test Suite (CUTS) so architects & systems engineers can conduct “what if” analysis on evolving systems by 1. Software architects, developers, & systems engineers specify structure of an enterprise DRE system 2. They associate necessary Qo. S characteristics with individual components (e. g. , CPU utilization) or system as a whole 3. Tools synthesize emulation code & configuration metadata to deploy system in target environment 4. Metrics are collected & analyzed to explore how design alternatives from multiple computational & valuation perspectives affect system Qo. S CUTS integrates nicely with continuous integration servers & test-driven development

Level of Abstraction Emulating Computational Components in CUTS • Application components are represented as Component Workload Emulators (Co. Work. Ers) • Co. Work. Ers can be interconnected by the PICML tool to form operational strings • Co. Work. Ers are programmed using the Component Behavior Modeling Language (CBML) Software Lifecycle Timeline www. dre. vanderbilt. edu/~schmidt/IEEE-RTCSA-06. pdf www. cs. wustl. edu/~schmidt/PDF/CUTS. pdf

Level of Abstraction Emulating Computational Components in CUTS High-level Overview of WML’s Modeling Structure • Workload Modeling Language (WML) is used to parameterize CBML actions with realistic & executable workload Software Lifecycle Timeline

Level of Abstraction Emulating Computational Components in CUTS • Workload Modeling Language (WML) is used to parameterize CBML actions with realistic & executable workload • Fully parameterized models are used to generate emulation code for the target architecture • i. e, spec. compliant code Software Lifecycle Timeline www. dre. vanderbilt. edu/~schmidt/WML-book-chapter. pdf www. cs. wustl. edu/~schmidt/PDF/Qo. SPML-WML. pdf

Level of Abstraction Analyzing Systemic Performance in CUTS • Benchmark Manager Web (BMW) Utility provides analytical capabilities for CUTS • e. g. , showing performance of actions in each Co. Work. Er, or evaluating non-functional concerns of mission critical operational strings Software Lifecycel Timeline www. dre. vanderbilt. edu/~schmidt/IEEE-ECBS-2008. pdf

Concluding Remarks • The emergence of ULS systems requires significant innovations & advances in tools & platforms • Not all technologies provide the precision we’re accustomed to in legacy small-scale systems • Advances in computing technology & related disciplines needed to address ULS systems challenges • Significant groundwork layed in various R&D programs • Much more research needed for ULS systems • Currently working with SEI, OSD, & NSF on ULS systems R&D