Realizing the Promise of Grid Computing Ian Foster

Realizing the Promise of Grid Computing Ian Foster Mathematics and Computer Science Division Argonne National Laboratory and Department of Computer Science The University of Chicago Presentation to the NSF Advisory Committee on Cyber. Infrastructure, November 30, 2001

The Grid Opportunity l What Grids are about: “Resource sharing & coordinated problem solving in dynamic, multiinstitutional virtual organizations” = entirely new tools, with often revolutionary impacts l The opportunity: advance transition to routine use by multiple years foster@mcs. anl. gov ARGONNE öCHICAGO

Why Grids? l l l A biochemist exploits 10, 000 computers to screen 100, 000 compounds in an hour 1, 000 physicists worldwide pool resources for petaop analyses of petabytes of data Civil engineers collaborate to design, execute, & analyze shake table experiments Climate scientists visualize, annotate, & analyze terabyte simulation datasets An emergency response team couples real time data, weather model, population data foster@mcs. anl. gov ARGONNE öCHICAGO

Why Grids? (contd) l l l A multidisciplinary analysis in aerospace couples code and data in four companies A home user invokes architectural design functions at an application service provider An application service provider purchases cycles from compute cycle providers Scientists at a multinational company collaborate on the design of a new product A community group pools members’ PCs to perform environmental impact study foster@mcs. anl. gov ARGONNE öCHICAGO

Grids: Why Now? l l Moore’s law improvements in computing produce highly functional endsystems The Internet and burgeoning wired and wireless provide universal connectivity Changing modes of working and problem solving emphasize teamwork, computation Network exponentials produce dramatic changes in geometry and geography – 9 -month doubling: double Moore’s law! – 1986 -2001: x 340, 000; 2001 -2010: x 4000? foster@mcs. anl. gov ARGONNE öCHICAGO

The Grid World: Current Status l An exciting time, in many ways – Dozens of major Grid projects worldwide > Deployment, technology, application – Consensus on key concepts & technologies > E. g. , Globus Toolkit as de facto standard – Growing industrial interest l But also: – Funded by an inadequate patchwork of diverse, mostly short-term sources – No long-term coordinated plan aimed at injecting Grid technologies into community – International programs outpacing U. S. efforts! foster@mcs. anl. gov ARGONNE öCHICAGO

PACIs and Grids l PACIs play critical role in Grid development – Act very effectively as nucleation point, bully pulpit, technology explorer – Major resource providers for community l But grid technologies & applications are essentially unfunded mandates for PACIs – “Grids” a tiny fraction of total PACI budget – Situation only worse for Tera. Grid! l Current situation untenable long term – New scientific tools are not created for free foster@mcs. anl. gov ARGONNE öCHICAGO

What is Needed: A National Grid Program l Goal: Accelerate “Grid-enablement” of entire science & engineering communities – Don’t wait the 20 years it took the Internet! l Program components 1) Persistent R, D, outreach, support organization 2) Application-oriented “Grid challenge” projects 3) Infrastructure: campus, national, international 4) Basic research, engaging CS community 5) Explicit international component l Explicit and strong interagency coordination foster@mcs. anl. gov ARGONNE öCHICAGO

A Persistent Grid Technology Organization l We’re talking about a complete retooling of entire science and engineering disciplines – Not a part-time, or three-year, or graduate student business – Also not something we can buy (yet) l We need a persistent national organization that can support this process – Technology R&D, packaging, delivery – Training, outreach, support – GRIDS Center an (unproven) existing model foster@mcs. anl. gov ARGONNE öCHICAGO

Application “Grid Challenge” Projects l Goal: Engage significant number of communities in the transition to Grids – Gri. Phy. N, NVO, NEESgrid existing models l Emphasize innovation in application of technology and impact to community – May be data-, instrumentation-, compute-, and/or collaboration-intensive – Aim is to achieve improvement in the quality and/or quantity of science or engineering – And to entrain community in new approaches foster@mcs. anl. gov ARGONNE öCHICAGO

Upgrade National Infrastructure l Seed nation with innovative Grid resources – i. VDGL one existing model l Encourage formation of campus Grids – Re-think campus infrastructure program? – U. Tenn Sin. RG one existing model l Enhance national & international networks, link with Tera. Grid – Advanced optical nets, Star. Light, etc. l Operations and monitoring foster@mcs. anl. gov ARGONNE öCHICAGO

Basic Research l Engage researchers in imagining & creating new tools & problem-solving methods – In a world of massive connectivity, data, sensors, computing, collaboration, … l l And in understanding and creating the new supporting services & infrastructure needed This is not “CS as usual” foster@mcs. anl. gov ARGONNE öCHICAGO

Explicit International Component l International connections are important, to – Support international science & engineering – Connect with international Grid R&D – Achieve consensus and interoperability l l But international cooperation, especially in technology R&D, is hard An National Grid Program needs to provide explicit support for international work – Infrastructure: networks – Support for projects foster@mcs. anl. gov ARGONNE öCHICAGO

Resource Requirements l Persistent technology/support org: $30 M – ~200 people l Application “Grid challenges”: $20 M – 10 teams, with application & CS involvement l Infrastructure upgrades: $20 M – Tb networks, campus infrastructures l Research: $15 M – Grids of tomorrow, & 100 s of grad students l International projects: $5 M – Support international work in other projects foster@mcs. anl. gov ARGONNE öCHICAGO