Grid-based Information Architecture for i. SERVO International Solid Earth Research Virtual Organization Western Pacific Geophysics Meeting (WPGM) Beijing Convention Center July 26 2006 Geoffrey Fox Computer Science, Informatics, Physics Pervasive Technology Laboratories Indiana University Bloomington IN 47401 http: //grids. ucs. indiana. edu/ptliupages/presentations/ gcf@indiana. edu http: //www. infomall. org
APEC Cooperation for Earthquake Simulation n ACES is a seven year-long collaboration among scientists interested in earthquake and tsunami predication • i. SERVO is Infrastructure to support work of ACES • SERVOGrid is (completed) US Grid that is a prototype of i. SERVO • http: //www. quakes. uq. edu. au/ACES/ n Chartered under APEC – the Asia Pacific Economic Cooperation of 21 economies
Participating Institutions n n n n n CSIRO Australia Monash University Australia University of Western Australia, Perth, Australia University of Queensland Australia University of Western Ontario Canada University of British Columbia Canada China National Grid Chinese Academy of Sciences China Earthquake Administration China Earthquake Network Center n n n n Brown University Boston University Jet Propulsion Laboratory Cal State Fullerton San Diego State University n n UC Davis UC Irvine UC San Diego University of Southern California University of Minnesota Florida State University US Geological Survey Pacific Tsunami Warning Center PTWC Hawaii National Central University, Taiwan (Taiwan Chelungpu-fault Drilling Project) University of Tokyo Institute of Technology (Titech) Sophia University National Research Institute for Earth Science and Disaster Prevention (NIED) Japan Geographical Survey Institute, Japan
Role of Information Technology and Grids in ACES Numerical simulations of physical, biological and social systems Engineering design Economic analysis and planning Sensor networks and sensor webs High performance computing Data mining and pattern analysis Distance collaboration Distance learning Public outreach and education Emergency response communication and planning Geographic Information Systems Resource allocation and management
Grids and Cyberinfrastructure n n Grids are the technology based on Web services that implement Cyberinfrastructure i. e. support e. Science or science as a team sport • Internet scale managed services that link computers data repositories sensors instruments and people There is a portal and services in SERVOGrid for • Applications such as Geo. FEST, RDAHMM, Pattern Informatics, Virtual California (VC), Simplex, mesh generating programs …. . • Job management and monitoring web services for running the above codes. • File management web services for moving files between various machines. • Geographical Information System services • Quaketables earthquake specific database • Sensors as well as databases • Context (dynamic metadata) and UDDI system long term metadata services • Services support streaming real-time data
Repositories Federated Databases Database Sensors Streaming Data Field Trip Database Sensor Grid Database Grid Research Compute Grid Data Filter Services Research Simulations SERVOGrid ? GIS Discovery Grid Services Education Customization Services From Research to Education Analysis and Visualization Portal Grid of Grids: Research Grid and Education Grid Computer Farm
SERVOGrid has a portal The Portal is built from portlets – providing user interface fragments for each service that are composed into the full interface – uses OGCE technology as does planetary science VLAB portal with University of Minnesota
Semantically Rich Services with a Semantically Rich Distributed Operating Environment SOAP Message Streams OS SS FS Another Service Data FS Raw Data MD OS OS Information SS FS FS MD SS OS FS FS SS MD Knowledge FS SS OS F S Information SS Raw Data Grids of Grids Architecture SS MD OS SS SS Other Service OS OS FS MD OS Data. FS FS al FS MD Data MD SS SS FS FS SS SS FS rt F S FS Information FS Raw Data FS Decisions Po OS MD Data OS SS OS FS Knowledge FS SS Another Database Grid Wisdom MD SS Another Service Filter Service FS SS Raw Data SOAP Message Streams Another Grid SS Meta. Data Sensor Service is same as outward facing application service
Linking Grids and Services n n Linkage of Services and Grids requires that messages sent by one Grid/Service can be understood by another Inside SERVOGrid all messages use • Web service system standards we like (UDDI, WS-Context, WSDL, SOAP) and • GML as extended by WFS so that data sources and simulations all use same syntax All other Web service based Grids use their favorite Web service system standards but these differ from Grid to Grid • Furthere is no agreement on application specific standards – not all Earth Science Grids use OGC standards • OGC standards include some capabilities overlapping general Web Services • Use of WSDL and SOAP is agreed although there are versioning issues So there is essentially there is no service level interoperability between Grids but rather interoperation is at diverse levels with shared technology • SQL for databases, PBS for Job scheduling, Condor for job management, GT 4 or Unicore for Grids
Grids in Babylon n Presumptuous Tower of Babel (from the web) • In the Bible, a city (now thought to be Babylon) in Shinar where God confounded a presumptuous attempt to build a tower into heaven by confusing the language of its builders into many mutually incomprehensible languages. n For Grids, everybody likes to do their own thing and Grids are complex multi-level entities where no obvious points of interoperation • so one does not need divine intervention to create multiple Grid specifications • But data in China, Tsunami sensors in Indian ocean and simulations in USA etc. will not be linked for better warning and forecasting unless the national efforts can interoperate n Two interoperation strategies: • Make all Grids use the same specifications (divine harmony) • Build translation services (filters!) using say OGF standards as a common target language (more practical) n Don’t need computers (jobs) to be interoperable (although this would be good) as each country does its own computing • Rather need data and some metadata on each Grid to be accessible from all Grids
Interoperability Summary n n Need to define common infrastructure and domain specific standards • Build Interoperable Infrastructure gatewayed to existing legacy applications and Grids Generic Middleware • Grid software including workflow • Portals/Problem Solving environments incl. visualization • We need to ensure that we can make security, job submission, portal, data access (sharing) mechanisms in different economies interoperate Geographic Information Systems GIS • Use services as defined by Open Geospatial Consortium (Web Map and Feature Services) http: //www. crisisgrid. net/ Earthquake/Tsunami Science Specific • Satellites, sensors (GPS, Seismic) • Fault, Tsunami … Characteristics stored in databases need GML extensions - Schema for Quake. Tables developed by SERVOGrid can be used Internationally
ACES Components Country and/or Economies Data (shared as part of a collaboration) Earthquake Forecast/Model Wave Motion Infrastructure Institutions Australia Seismic data, fault database, GPS Finley, LSM PANDAS prototype Access Canada Polaris Radarsat Pattern Informatics P. R. China Seismic GPS LURR CAS China National Grid Japan GPS Seismic Daichi (In. SAR) Geo. FEM JST-CREST Earth Simulator Naregi Chinese Taipei FORMOSAT 3/COSMIC (F/C) U. S. A. Quake. Tables Sesismic In. SAR PBO (GPS) Pattern Informatics ALLCAL Geo. FEST, PARK, Virtual. California Tera. Shake SERVOGrid GEON SCECGrid Vlab International IMS Pacific Rim Universities (APRU ) PRAGMA
National Earthquake Grids of Relevance n n n APAC –GT 2 GT 4 g. Lite ACc. ESS – Some link to SERVOGrid China National Grid – GOS GT 3 GT 4 China. Grid – CGSP built on GT 4 CNGI – China’s Next Generation Internet has significant earthquake data component Naregi – Uses GT 4 and Unicore with much enhancements Japanese Earthquake Simulation Grid – unclear K*Grid Korea Enhanced SRB, GT 2 to GT 4 TIGER Taiwan Integrated Grid for Education and Research unclear technology and unclear earthquake relevance SERVOGrid – Uses WS-I+ simple Web Services Tera. Grid – Uses GT 4 but not a clear model except for core job submittal
Tera. Grid: Integrating NSF Cyberinfrastructure Buffalo Wisc UC/ANL Utah Cornell Iowa PU NCAR IU NCSA Caltech PSC ORNL USC-ISI UNC-RENCI SDSC TACC Tera. Grid is a facility that integrates computational, information, and analysis resources at the San Diego Supercomputer Center, the Texas Advanced Computing Center, the University of Chicago / Argonne National Laboratory, the National Center for Supercomputing Applications, Purdue University, Indiana University, Oak Ridge National Laboratory, the Pittsburgh Supercomputing Center, and the National Center for Atmospheric Research. Today 100 Teraflop; tomorrow a petaflop; Indiana 20 teraflop today.
Portal Tools: APAC National Grid Core Grid Services Grid. Sphere QPSF Info Services: (JCU) APAC Registry INCA 2? ACc. ESS at UQ (ACES Partner) outside APAC Security: APAC CA My. Proxy VOMRS Systems: QPSF APAC National Facility IVEC ANU SAPAC Gateways Partners’ systems Network: Grange. Net / AARNet APAC Private Network (AARNet) ac 3 VPAC TPAC CSIRO
National “Grid Projects” in China Plan Research Develop Production Procure Deploy Operate Manage China e-Nation Strategy (2006 -2020) Virtual Comp. Env. CAS e. Science Net-based Res. Env. China National Grid Semantic Grid Edu. & Res. Grid Next-Generation Network Initiative € 10 M’s NSFC CAS Science and Technology R &D Assets Foundation Platform €M’s State Council Mo. E Mo. ST National Planning Commission Grid activities still growing
CNGrid (2006 -2010) • HPC Systems – 100 Tflop/s by 2008, Pflop/s by 2010? • Grid Software Suite: CNGrid GOS – Merge with international efforts – Emphasize production • CNGrid Environment – Nodes, Centers, Policies • Applications – – Science Resource & Environment Manufacturing Services – Domain Grids
Cyber Science Infrastructure toward Petascale Computing (planned 2006 -2011) International Collaboration - EGEE - UNIGRIDS -Teragrid -GGF etc. Operaontional Collaborati R&D Collaboration Nano Proof, Eval. R&D Collaboration Joint Project IMS （Bio） Project Joint Osaka-U Feedback AIST ナノ分野 Delivery ナノ分野 実証・評価 Nano 実証・評価 Proof of al. Concept 分子研 Eval. 分子研 Delivyer IMS Feedback Project. Oriented VO NII NAREGI Site Core Site Customization Operation/Maintenance Domain Specific VO (e. g ITBL) Cyber-Science Infrastructure（ＣＳＩ） Middleware CA Research βver. Dev. （ ） V 1. 0 V 2. 0 Collaborative Operation Center （IT Infra. for Academic Research and Education) Delivery Feedback R&D Collaboration Operation/ Maintenance （Middleware） Operation/ Maintenance （UPKI, CA） Peta-scale System VO Industrial Projects Domain Specific VOs Univ. /National Supercomputing VO IMS, AIST, KEK, NAO, etc. Networking Infrastructure (Super-SINET） Note: names of VO are tentative） Networking Contents Project-oriented VO
Japanese Earthquake Simulation Grid Data-Server NIED 48 x. G 5, 15 TB Data-Server GSI 8 x. Opteron 20 TB Integrated Observation-Simulation Data Grid Super SINET (10 Gbps) Earth Simulator 5, 120 x. SX 6 PC Cluster ERI, 64 x. Opteron para. AVS PC Cluster EPS, 64 x. Opteron para. AVS
JST-CREST Integrated Predictive Simulation System Strong Motion and Tsunami Generation Earthquake Generation Tsunami Generation Plate Motion Tectonic Loading Earthquake. Rupture Wave Propagation. Structure Oscillation Artificial Structure Oscillation Crustal Movement Data Analysis Seismic Activity Data Analysis Strong Motion Data Analysis GONET Hi-net K-NET Platform for Integrated Simulation Database for Model Construction Data Processing, Visualization, Linear Solvers PC clusters for small-intermediate problems Earth Simulator for large-scale problems GIS Urban Information Simulation Output
Current PTWC Network of Seismic Stations (from GSN & USNSN & Other Contributing Networks)
The NCES/WS-*/GS-* Features/Service Areas I Service or Feature WS-* GS-* NCES (Do. D) Comments A: Broad Principles FS 1: Use SOA: Service Oriented Arch. WS 1 Core Service Architecture, Build Grids on Web Services. Industry best practice FS 2: Grid of Grids Strategy for legacy subsystems: modular architecture B: Core Services (Mainly Service Infrastructure and W 3 C/OASIS focus) FS 3: Service Internet, Messaging WS 2 NCES 3 Core Infrastructure including reliability, publishsubscribe messaging cf. FS 13 C FS 4: Notification WS 3 NCES 3 JMS, MQSeries, WS-Eventing, Notification FS 5: Workflow WS 4 NCES 5 Grid Programming FS 6: Security WS 5 NCES 2 Grid-Shib, Permis Liberty Alliance. . . FS 7: Discovery WS 6 NCES 4 UDDI and extensions FS 8: System Metadata & State WS 7 FS 9: Management WS 8 FS 10: Policy WS 9 ECS FS 11: Portals and Users WS 10 NCES 7 Globus MDS Semantic Grid, WS-Context GS 6 NCES 1 CIM Portlets JSR 168, NCES Capability Interfaces
The NCES/WS-*/GS-* Features/Service Areas II Service or Feature WS-* GS-* NCES Comments B: Core Services (Mainly Higher level and OGF focus) FS 12: Computing GS 3 Job Management major Grid focus FS 13 A: Data as Repositories: Files and Databases GS 4 NCES 8 Distributed Files, OGSA-DAI Managed Data is FS 14 B FS 13 B: Data as Sensors and Instruments FS 13 C: Data Transport OGC Sensor. ML WS 2, 3 GS 4 NCES 3, 8 Grid. FTP or WS Interface to non SOAP transport FS 14 A: Information as Monitoring GS 4 Major Grid effort for job status etc. FS 14 B: Information, Knowledge, Wisdom part of D(ata)IKW GS 4 NCES 8 VOSpace for IVOA, JBI for Do. D, WFS for OGC Federation at this layer major research area NCOW Data Strategy FS 15: Applications and User Services GS 2 NCES 9 Standalone Services Proxies for jobs FS 16: Resources and Infrastructure GS 5 Ad-hoc networks; Network Monitoring FS 17: Collaboration and Virtual Organizations GS 7 NCES 6 XGSP, Shared Web Service ports FS 18: Scheduling and matching of Services and Resources GS 3 Current work only addresses scheduling “batch jobs”. Need networks and services
Скачать презентацию Grid-based Information Architecture for i SERVO International Solid
b0d2463435f4b71beed302693e21233c.ppt