Скачать презентацию e-Science and Datacentric Frameworks Hyunseung Choo Sungkyunkwan University Скачать презентацию e-Science and Datacentric Frameworks Hyunseung Choo Sungkyunkwan University

172d2c6c83f48ef0743e91f014985e0a.ppt

  • Количество слайдов: 25

e-Science and Datacentric Frameworks Hyunseung Choo Sungkyunkwan University http: //monet. skku. ac. kr choo@skku. e-Science and Datacentric Frameworks Hyunseung Choo Sungkyunkwan University http: //monet. skku. ac. kr [email protected] ac. kr

e-Science and its examples e-Science and its examples

e-Science ‘e-Science’ is about global collaboration in key areas of science, and the next e-Science ‘e-Science’ is about global collaboration in key areas of science, and the next generation of infrastructure that will enable it. ‘e-Science’ will change the dynamic of the way science is undertaken. Director General of Research Councils Office of Science and Technology John Taylor

GRID vs. e-Science GRID e-Science Enhancing research productivity and acquiring national competitive power based GRID vs. e-Science GRID e-Science Enhancing research productivity and acquiring national competitive power based on R&D infrastructure in 21 st century Goals Organization Roles Resources Characteristics Differences IT based infrastructure for novel computing services Renovation of R&D capability based on proper infrastructure Advanced Networks Middlewares Applications Advanced Users IT Infrastructure Virtual Organizations HPC, mass storages, DB, advanced instruments, human resources, etc Shared data, information and computation by geographically dispersed communities Provider-oriented (Technology-Push) Focus on networks and middlewares Consumer-oriented (Science-Pull) Focus on actual applications

From Networking to Grid Computing ■ Exponential Growth of Network Technology ü Network vs. From Networking to Grid Computing ■ Exponential Growth of Network Technology ü Network vs. Computer Performance u Computer speed doubles every 18 months u Network speed doubles every 9 months u Difference = order of magnitude per 5 years ü 1986 to 2000 u Computers: x 500 u Networks: x 340, 000 ü 2001 to 2010 u Computers: x 60 u Networks: x 4000

The Driver for e-Science ■ More and more data ü Instrument resolution doubling / The Driver for e-Science ■ More and more data ü Instrument resolution doubling / 12 months u Instrument and telemetry speeds increasing u Mobile sensors & radio digital networks ü Storage capacity doubling / 12 months ■ More and more computation ü Computations available doubling / 18 months ■ Faster networks can change methods ü Raw bandwidth doubling / 9 months ■ These integrate and enable ü More interplay between computation and data ü More collaboration: scientists, medics, engineers, etc. ü More international collaboration

The New Behavior ■ Shared Infrastructure ü Intrinsically distributed ü Intrinsically multi-organizational ü Multiple The New Behavior ■ Shared Infrastructure ü Intrinsically distributed ü Intrinsically multi-organizational ü Multiple uses interwoven ■ Shared Software ü A new attempt at making distributed computing economic, dependable and accessible ü Scientists from all disciplines share in its design and use ■ Shared & Automated System Administration ü Replicated farms of replicated systems ü Autonomic management ■ Immediate Benefits ü Faster transfer of ideas and techniques between disciplines ü Amortization of development, operation and education

Examples on e-Science Earth Observation Systems severe weather predictions, climate variations, flood monitoring, earthquakes, Examples on e-Science Earth Observation Systems severe weather predictions, climate variations, flood monitoring, earthquakes, and tsunami (a tidal wave) Virtual Observatories Robotic Telescopes Bioinformatics / Functional genomics Collaborative Engineering Medical / Healthcare informatics Tele. Microscopy, and so on

Example 1 – Earthquake Simulation NEESgrid National infrastructure to couple earthquake engineers with experimental Example 1 – Earthquake Simulation NEESgrid National infrastructure to couple earthquake engineers with experimental facilities, databases, computers, & each other. Argonne, Michigan, NCSA, UIUC, USC

Example 2 – Airspace Simulation NASA Information Power Grid (IPG) Aircraft, flight paths, airport Example 2 – Airspace Simulation NASA Information Power Grid (IPG) Aircraft, flight paths, airport operations and the environment are combined to get a virtual national airspace

e-Science (USA) ■ Cyber infrastructure program like “e-Science community” for federal offices, supercomputing centers, e-Science (USA) ■ Cyber infrastructure program like “e-Science community” for federal offices, supercomputing centers, and research institutes ü Budget in 2003 : U$ 1. 1 billion ■ e-Science Cases ü Telescience Portal : X-ray related applications including Microbioanalysis ü NASA IPG (Information Power Grid) : Aircraft simulation and analysis to reduce the design processing time ü BIRN(Biomedical Informatics Research Network) : Study on human and animal brains for the new era in medical science

BIRN (Biomedical Informatics Research Network) ■ Processing Pipelines for Morphometric Analysis ■ Medical Applications BIRN (Biomedical Informatics Research Network) ■ Processing Pipelines for Morphometric Analysis ■ Medical Applications for HPC ü non-linear registrations ü biomechanical simulations ü statistical analysis of large populations

e-Science Centre (UK) Access. Grid always-on video walls e-Science Centre (UK) Access. Grid always-on video walls

e-Science Pilot Project (UK) (1/2) ■ Many to one project ■ Particle Physics and e-Science Pilot Project (UK) (1/2) ■ Many to one project ■ Particle Physics and Astronomy Research Council (PPARC) ü Grid. PP: A prototype Grid infrastructure for the CERN Large Hadron collider ü Astro. Grid: A Grid based Virtual Observatory ■ Biotechnology and Biological Sciences Research Council (BBSRC) ■ Medical Research Council (MRC) ■ Natural Environment Research Council (NERC) ü Grid for Environmental Systems Diagnostics and Visualization ü Climateprediction. com: Distributed computing for global climate research ü Environment from the Molecular Level: Modeling the atomistic processes involved in environmental issues

e-Science Pilot Project (UK) (2/2) ■ Economic Social Research Council (ESRC) ■ Engineering and e-Science Pilot Project (UK) (2/2) ■ Economic Social Research Council (ESRC) ■ Engineering and Physical Sciences Research Council (EPSRC) ü The Reality Grid: a tool for investigating condensed matter and materials ü Comb-e-chem: Structure-Property Mapping: Combinatorial Chemistry and the Grid ü DAME: Distributed Aircraft Maintenance Environment ü GEODISE: Grid Enabled Optimization and Design Search for Engineering ü Discovery Net: An e-Science Testbed for High Throughput Informatics ü My. Grid: Directly Supporting the e-Scientist ■ Council for the Central Laboratory of the Research Councils (CLRC)

e-Science (JP) ■ IT-based laboratory (ITBL), Grid based fundamental Informatics (A 05), 100 Teraflop e-Science (JP) ■ IT-based laboratory (ITBL), Grid based fundamental Informatics (A 05), 100 Teraflop high performance computing (NAREGI) ü All led by Ministry of Education, Culture, Sports, Science, and Technology (문부과학성) ■ e-Science Cases ü ITBL : Project for virtual research environments ü A 05 : Grid computing project ü NAREGI : Integrating distributed computing resources by high performance networks for 100 Teraflop HPC

ITBL (IT-Based Laboratory) ■ 6 Organizations at ITBL ü ü ü Japan Atomic Energy ITBL (IT-Based Laboratory) ■ 6 Organizations at ITBL ü ü ü Japan Atomic Energy Research Institute (JAERI) 일본원자력 연구소 RIKEN (The Institute of Physical and Chemical Research) 이화학연구소 National Institute for Materials Science (NIMS) 재료 물질 연구 기구 National Aerospace Laboratory of Japan (NAL) 항공우주기술연구소 National Research Institute for Earth Science and Disaster Prevention (NIED) 방재과학기술연구소 ü Japan Science and Technology Corporation (JST) 과학진흥 사업단 ■ Massive collaborative research environment for remote researchers by Super. SINET based on IT infrastructure

e-Science (CN) ■ Grid Projects in China (2002 -2005) ü The Ministry of Science e-Science (CN) ■ Grid Projects in China (2002 -2005) ü The Ministry of Science & Technology 863 Grid Project u Grid Enabling Cluster (>4 Tflop/s) u Grid Nodes (Total 6 -10 Tflop/s) u Grid Software (Grid OS, Developer and User Environment) u Grid Applications in Science, Manufacturing, Service industry, and Environment/Resource sector ü The “Next Internet” Project (led by Chinese NSF) u Upgrade network infrastructure u Basic research in computing, data and access grids ü The Chinese Academy of Sciences e-Science Grid ü The Beijing City Manufacturing Grid

Datacentric Applications Datacentric Applications

Three different kinds of grids ■ Computational grids ü These represent the natural extension Three different kinds of grids ■ Computational grids ü These represent the natural extension of large parallel and distributed systems, and exist to provide high-performance computing ■ Access grids ü This requires managing access to many specific, small resources that are actually located inside large, complex, organizational computer systems and networks ■ Data grids ü These exist in order to allow large datasets to be stored in repositories and moved about with the same ease that small public files can be moved today ☼ Datacentric grids

Facts about online data ■ They are big and growing fast ü Data stored Facts about online data ■ They are big and growing fast ü Data stored online quadruples every 18 months. ü Process power ‘only’ doubles every 18 months. ■ They are naturally distributed ü Data is captured via multiple channels ü Operating systems struggle to handle files larger than a few GB ■ They are hard to move ü Pragmatics: Few sites have enough swap space to handle the arrival of a terabyte dataset for temporary use ü Performance ü Politics: Data about individuals cannot be moved out of jurisdictions with strong privacy rules

Implications of datasets that are large, distributed, and immovable ■ It’s much more effective Implications of datasets that are large, distributed, and immovable ■ It’s much more effective to divide programs into separated pieces and send them to data ■ This requires a datacentric view of computation, rather than the conventional processor-centric view. ü A new programming model is needed ü Applications must be decomposable ü The results of (partial) computations must be small enough to move around ü These condensed forms are worth keeping ü Execution nodes must be able to provide both computing cycles and high-performance data access.

Some properties ■ Users can be productive even from a thin client ■ Applications Some properties ■ Users can be productive even from a thin client ■ Applications require only thin pipes within the internet ■ Code mobility is essential ■ The format and content of a data repository will often be unknown to an application until it actually starts accessing it ■ Applications will tend to be standardized ■ Applications will often be built from templates, perhaps even expressed using a query language ■ Re-execution of an application on a different or updated dataset will be common ■ There will be increased sensitivity about information leakage

A typical datacentric application A typical datacentric application

Conclusion ■ e-Science and datacentric grid are strongly coupled ■ Meteorology data require dataqcentric Conclusion ■ e-Science and datacentric grid are strongly coupled ■ Meteorology data require dataqcentric grid computing in the future ü ü Typical e-Science characteristics Huge data size Poor data site accessibility Experts are spread over the country/world ■ Basically all are based on reliable networks ü Exact computing on network probabilistic connectivity (one aspect of reliability measures) is theoretically hard ü Fast approaches and good enough approximation algorithm are developed (will be published)