Sim Millennium Project Overview David E Culler Computer

Sim. Millennium Project Overview David E. Culler Computer Science Division U. C. Berkeley NSF Site Visit March 2, 1998 Sim. Millennium Overview

The Vision • To work, think, and study in a computationally rich environment with deep information stores and powerful services – test ideas through simulation – explore and investigate data and information – share, manipulate, and interact through natural actions • Organized in a manner consistent with the University setting March 2, 1998 Sim. Millennium Overview 2

Sim. Millennium Project Goals • Enable major advances in Computational Science and Engineering – Simulation, Modeling, and Information Processing becoming ubiquitous • Explore novel design techniques for large, complex systems – Fundamental Computer Science problems ahead are problems of scale • Develop fundamentally better ways of assimilating and interacting with large volumes of information – and with each other • Explore emerging technologies – networking, OS, devices March 2, 1998 Sim. Millennium Overview 3

Goals of this talk • Components of the Project – – – Community Cluster-based Resouces Connectivity User Interaction Computational Economics • Specific Infrastructure • Research Agenda March 2, 1998 Sim. Millennium Overview 4

Component 0: Community • An inter-disciplinary community with common interests and shared view of the future – strong momentum in computational science and engineering => Session II – Members of 17 campus units and NERSC in Intel Millennium – Need and commitment required for participation – Key subset represented in this proposal March 2, 1998 Sim. Millennium Overview 5

Component 1: Resources (Millennium) • An environment with vast cluster-based computing power and storage (CLUMPS) behind a personal 3 D desktop Campus Cluster Group Cluster of SMPs Dept. SMP NT 3 D Desktop March 2, 1998 Sim. Millennium Overview 6

Resource Component Support • Computers via Intel Technology 2000 grant – 200 NT desktops – 16 department 4 -way SMPs – 8 5 x 4 Group Clusters, – 1 ~100 x 4 Campus Cluster – PPro => Pentium II => Merced 200 Gflop/s 150 GB memory 8 TB disk • Additional storage via IBM SUR grant – 0. 5 TB this year => 4 TB • • NT tools via Microsoft grant Solaris x 86 tools via SMCC grant Campus provides Technical staff Research provides the prog. and system support March 2, 1998 Sim. Millennium Overview 7

Key NSF investment: Cluster Network • Transforms large collection of individual resources into a powerful system – can be focus on a problem • High Bandwidth – scales with the number of processors • • (Gb/s per proc) Low Latency Low Overhead Low Cost Simple and Flexible Almost no errors Low Risk Today: Myrinet March 2, 1998 Sim. Millennium Overview 8

Cluster Research Agenda • Applications grow into resources – huge range of needs – require Algorithmic Innovation, Prog. Tools, & Performance • Dealing Deep Memory Hierarchy – New numerical algorithms on CLUMPs – New compiler techniques for parallel object language • Fast Multi-protocol Communication • Global system at large scale – Unix vs. NT, single system image vs. objects • Exciting technology turnover – VIA, SANs, Gigabit Ethernet => Session March 2, 1998 Sim. Millennium Overview III 9

Campus Support Enables Research • Technical staff develop and deploy common solution and environment – networked systems designer – unix-based programmer and cluster system admin. – NT-based programmer and tools • Technical computing software developed jointly with NERSC • Participating departments provide system administration and construction costs. March 2, 1998 Sim. Millennium Overview 10

Component 2: Connectivity • Create a richly interconnected pool of resources owned by members of the community – Enable transportation of huge data sets and computation – Enable remote visualization and collaboration – Enable extensive sharing of resources • Expand networking technology CS Cluster EE Cluster CE Cluster ME Cluster Astro/Phys Cluster xport Cluster BIO Cluster Econ/Math Cluster Campus Cluster March 2, 1998 Sim. Millennium Overview 11

NSF Investment: Inter-cluster network • Gigabit Ethernet connecting group clusters and campus cluster • Bay Networks provides 70% discount • Campus provides fiber plant, maintenance, and staff March 2, 1998 Sim. Millennium Overview 12

Physical Connectivity March 2, 1998 Sim. Millennium Overview 13

Inter-Cluster Research Agenda • Vastly expands the scope of systems challenge – integrate well-connected resources according application needs, rather than physical packaging – resource allocation, management, and administration • Network bandwidth matches display BW – Protocols and run-time sys. for visualization, media transport, interaction, and collaboration. • Community can share non-trivial resources while preserving sense of ownership – Bandwidth translates into efficiency of exchange – Data can be anywhere • Important networking technology in its own right. – Layer 3 switching, Qo. S, VLan => Session March 2, 1998 Sim. Millennium Overview III, V 14

Component 3: User Interaction • High-quality 3 D graphics emerging on costeffective platforms – desktops and dedicated cluster nodes – NERSC team provides modern scientific visualization support • Gigabit network allows this to be remote. • New displays create “workbench” environment where large volumes of information can be viewed and manipulated. • Trackers and Haptic interfaces greatly enhance degrees of user input – 3 D capture March 2, 1998 Sim. Millennium Overview 15

NSF Investment: UI Technology • Two Projection Table – large field of view in horizontal (or vertical) orientation • Phantom Haptic Interface – 3 D force feedback • Motion Tracker – untethered position • 3 D Shutter Glasses – low cost visualization March 2, 1998 Sim. Millennium Overview 16

User Interaction Research Agenda • Expand access to 3 D visualization – Explore any data anywhere – Ease development • Develop lab-bench metaphor for Viz – two hands, physical icons • Fast prototyping and exchange through Informal Interfaces – sketching • Dealing with large volumes of information – lenses, brushing and linking • 3 D collaboration and interaction => Session March 2, 1998 Sim. Millennium Overview VI 17

Component 4: Computational Economy • How is this vast, integrated pool of resources managed? • Traditional system approach: empower global OS to provide “optimal” allocation to blind applications – predefined metric, tuned to fixed workload – ignores the inherent adaptation of demand • Computer Center – charge => director-to-user feedback according to cost • Economic view: decentralized allocation according to perceived value – pricing => user-to-user feedback – compatible niches, sense of control, cooperation – idea has been around, why now? March 2, 1998 Sim. Millennium Overview 18

Research Agenda • Natural fit to academic structure – members want control over own resources, and each has varying needs that far exceed dedicated resources – incentive for maintaining resources up to par • Address partial or delayed information, component failure, and user satisfaction from the start • Framework for elevating design from resources to services • Rich body of theory, little empirical validation – experts in several parts of the community • New paradigm for algorithms & perf. Analysis • Complex, large-scale systems March 2, 1998 Sim. Millennium Overview 19

Basic Approach • Desktop an active agent conducting automated negotiation for resources • Servers provide resources to highest bidders – monitor usage and enforce limits within remote execution environment – placement based on economic advantage • Higher level system functions are self-supporting – resource availability, brokering, directories • Useful applications packaged as services – may charge more than resources cost March 2, 1998 Sim. Millennium Overview 20

NSF Investment: Staff Support • Provide enabling technology and let it evolve – monitoring, enforcement – exchange – negotiation tools • Integrate it into users enviroment • Tools and measurements to determine effectiveness March 2, 1998 Sim. Millennium Overview 21

Integrated Research Agenda • Advance the State of Computational Science and Engineering – immerse a community in a computationally rich environment with the right tools: algorithms, programming & system support – Path to exploiting novel techniques and technology • Explore design techniques for robust large-scale distributed systems – economic (or ecologic) approach • Explore new ways of interacting with information – large paste-ups, two hands, sketching, 3 D collaboration • Investigate new technology – SMP nodes, gigabit Ethernet, SANs, VIA – NT, d. COM, Java beans, directory services – workbench displays, 3 D icons, haptics, position sensors March 2, 1998 Sim. Millennium Overview 22

Perspective • Highly leveraged investment in a large scale infrastructure for studying problems of scale • Deep commitment across the campus • Sense of ownership and participation • Rich research agenda March 2, 1998 Sim. Millennium Overview 23