Скачать презентацию Report from Amsterdam Massimo Sgaravatto INFN Padova Скачать презентацию Report from Amsterdam Massimo Sgaravatto INFN Padova

0c61877b000976aba21ce1fe328715a0.ppt

  • Количество слайдов: 30

Report from Amsterdam Massimo Sgaravatto INFN Padova Report from Amsterdam Massimo Sgaravatto INFN Padova

Amsterdam events n n Global Grid Forum (GGF) 1 Data. Grid workshop Amsterdam events n n Global Grid Forum (GGF) 1 Data. Grid workshop

Global Grid Forum n n Focuses on the promotion and development of Grid technologies Global Grid Forum n n Focuses on the promotion and development of Grid technologies and applications via the development and documentation of "best practices, " implementation guidelines, and standards Global GF efforts are also aimed at the development of a broadly based Integrated Grid Architecture that can serve to guide the research, development, and deployment activities of the emerging Grid communities www. globalgridforum. org

Global Grid Forum History 1999 GF BOF (Orlando) GF 1 (San Jose — NASA Global Grid Forum History 1999 GF BOF (Orlando) GF 1 (San Jose — NASA Ames) GF 2 (Chicago — i. CAIR) e. Grid and GF BOFs (Portland) GF 3 (San Diego — SDSC) e. Grid 1(Posnan — PSNC) GF 4 (Redmond — Microsoft) Asia-Pacific GF Planning (Yokohama) e. Grid 2 (Munich — at Europar) GF 5 (Boston — Sun) Global GF BOF (Dallas) 2000

Global Grid Forum 2001 GGF 1 (Amsterdam) March 4 -7, 2001 w/ Data. Grid Global Grid Forum 2001 GGF 1 (Amsterdam) March 4 -7, 2001 w/ Data. Grid “Kickoff” (March 7 -9) Hosts: Amsterdam Science and Technology Center (SARA, Univ. Amsterdam, NIKHEF) 325 participants 192 organizations 28 countries GGF 2 (Washington, DC) July 15 -18, 2001 GGF 3 (Rome) October 7 -10, 2001 W/ Data. Grid workshop (October 2 -5) Frascati (Rome)

Global GF Working Groups n n n n Account Management Applications and Testbeds Grid Global GF Working Groups n n n n Account Management Applications and Testbeds Grid Information Services Remote Data Access Security Advanced Collaborative Environments Grid Protocol Architecture n n n Advanced Programming Models Grid Computing Environments Grid Performance Scheduling User services JINI

How to Be Involved n Electronic Participation n n Each working group uses email How to Be Involved n Electronic Participation n n Each working group uses email distribution lists for discussions Contributing to Grid Working Drafts n n Volunteer to help write a planned paper Submit an abstract/outline to propose a paper n n Architectures / Models API, Protocol, or other specification Survey or Current Practice Description Lead a Group n Volunteer to start a working group or a sub-group within an existing working group

Scheduling & Resource Management WG n Charter: generate best practice scheduling and resource management Scheduling & Resource Management WG n Charter: generate best practice scheduling and resource management documents and API specifications to enable interoperability n n Co-chairs: Jenny Schops, Bill Nitzberg, Uwe Schwiegelshohn Web site: http: //www. cs. nwu. edu/~jms/sched-wg Mailing list: [email protected] org General meetings at Grid forum events and teleconferences

Scheduling WG @ GGF 1 n 3 documents discussed n 10 actions for superscheduling Scheduling WG @ GGF 1 n 3 documents discussed n 10 actions for superscheduling n n n Advanced reservation API n n n Goal: list the basic steps and capabilities involved in resource reservation, acquisition and use Basically done Just an API because not yet advanced for a protocol Basically done Attributes for communication about scheduling instances n n Goal: define a set of attributes that a higher level scheduler can ask of a lower level scheduler In progress

Scheduling WG @ GGF 1 n Suggested new documents n Advance reservation protocol n Scheduling WG @ GGF 1 n Suggested new documents n Advance reservation protocol n n Allow advance reservation between systems Scheduling dictionary n n n List of tokens and definitions to coordinate with the dictionary group (common glossary of important terms across the Grid forum) Already started (terms only, no definitions) (www. sdsc. edu/~mthomas/GF/terms/sched-terms. cgi) Document to move this effort forward

Distributed Accounting WG n Purpose n n n n Develop a prototype Grid accounting Distributed Accounting WG n Purpose n n n n Develop a prototype Grid accounting architecture such as is required for resource economies Define interactions with other Grid components and existing systems Create a specification that can be used to build and evaluate prototype systems Co-chairs: Tom Hacker, William Thigpen Web site: http: //www. nasa. gov/~thigpen/accounts-wg Mailing list: [email protected] org Working drafts we may be interested in: n n Distributed accounting on the Grid An economy Grid architecture for service-oriented grid computing

GGF and WP 1 n Should we/how do we contribute to the GGF effort GGF and WP 1 n Should we/how do we contribute to the GGF effort ? n n n Dictionary ? Accounting ? …?

Data. Grid n n ATF Progress Report Joint Parallel meetings n n n WP Data. Grid n n ATF Progress Report Joint Parallel meetings n n n WP 1 -5 (Middleware) WP 6 -7 (Testbed) WP 8 -10 (Applications)

ATF Progress Report n Documents (http: //gridatf. web. cern. ch/gridatf/documents. html) n n Data. ATF Progress Report n Documents (http: //gridatf. web. cern. ch/gridatf/documents. html) n n Data. Grid Architecture version 1 Month 9 release We have to provide reactions Implementation plan for M 9 release by the next PTB (April 19)

WP 1 – 5 n Information Services n n Relational vs. LDAP Scope n WP 1 – 5 n Information Services n n Relational vs. LDAP Scope n n WP 3 is going to provide only APIs (for producers and consumers) and interfaces What will be there in M 9 ? n n n Current Globus GIS implementation (WP 3) WP 1: many shortcomings and open issues (performances, incomplete schema, …) in the current Globus GIS implementation, and therefore it is probably unsuitable Discussion with S. Tuecke (Globus team) n n n Aware of problems We may use/test Globus alpha releases Workshop (Convenior WP 3 ) n April 28, 29 @ Cern

WP 1 - 5 n Job n Partition/Checkpoint of jobs ? n n n WP 1 - 5 n Job n Partition/Checkpoint of jobs ? n n n At GRID level but also inside fabric ? Applications have to help us, otherwise it is difficult how to split/checkpoint jobs Job input/output n n n Files ? Objects ? …? Is the input/output known at schedule time ? Scheduling policies ? n n Deadline? Best fit ? , …? Applications should provide their view

WP 1 – 5 n Storage Model n n n Boundaries between WP 2, WP 1 – 5 n Storage Model n n n Boundaries between WP 2, WP 4, WP 5 Global vs. local filenames Who does the house keeping … Workshop: Convenior WP 2 n n March 27, 2001 @ Cern ? Accounting n n Economic model (WP 1) ? WP 1 to investigate and circulate thoughts

WP 6 – 7 n Testbed organization n Working model n n n Permanent WP 6 – 7 n Testbed organization n Working model n n n Permanent Group (partner representatives, technical committee, site contacts, testbed support) Release team (one team for each release validation phase) WP test activity (WPx) WP release (WPx + testbed +appl. ) Testbed release (WPs + testbed + appl. ) Official testbed release (+ users) Waiting for test requests

WP 6 – 7 n Testbed 0 n n Grid basic services deployment, based WP 6 – 7 n Testbed 0 n n Grid basic services deployment, based on Globus toolkit Resources n n n Mainly Linux machines (also Suns, SGI) LSF, Condor, PBS, BQS > 100 people > 40 sites Main issues n n n Information Services Resource access (certification, security and policies management) Installation toolkit

GIS Topology EU GIIS o=datagrid 0 (root server at Cern) (Top level server) LIP GIS Topology EU GIIS o=datagrid 0 (root server at Cern) (Top level server) LIP (Portugal) dc=lip, dc=pt, o=datagrid 0 INFN (Italy) dc=infn, dc=it, o=datagrid 0 France dc=fr, o=datagrid 0 . . . dc=in 2 p 3, dc=fr, o=datagrid –Support for GIS deployment in place . . .

WP 6 – 7 n Globus installation toolkit n n n 3 toolkits available WP 6 – 7 n Globus installation toolkit n n n 3 toolkits available (INFN, UK, FR) Working group to converge on common tools and common environment (within 3 weeks) Security n CA group (chaired by D. Kelsey) already in place n n Access policies for Testbed 0 n n n Support for certificates signed by the various national CA’s Globus grid-mapfiles Only one account per certificate (no group accounts) Requests for accounts for Testbed 0 via WP managers to WP 6 General security group (chaired by D. Kelsey) is going to set up Security Workshop (29, 30 March @ Cern, Convenior: D. Kelsey) n n n Representatives from WP’s Security experts/officers/reps from Testbed sites Other interested people

WP 8 - 10 Specific application layer VO common application layer ALICE ATLAS CMS WP 8 - 10 Specific application layer VO common application layer ALICE ATLAS CMS HEP LHCb WP 9 WP 10 Earth Obs. Biology Data. GRID middleware PPDG, Gri. Phyn, Euro. GRID Bag of Services (GLOBUS) OS & Net services WP 8 -910 TWG

WP 8 – 10 n User cases for testbed 0 & 1 n n WP 8 – 10 n User cases for testbed 0 & 1 n n n User cases for testbed 2 & 3 n n Basically ready for WP 8 Available for WP 9 Draft in one month for WP 8 Common (WP 8 -10) user requirements n n First draft in one month First release at M 6

WP 8 – 10 n Preliminary reactions to ATF n n Objects or files WP 8 – 10 n Preliminary reactions to ATF n n Objects or files when interacting with the middleware ? Access to middleware services from applications via API (C++ API) RW databases Interactive jobs n n n Required job resources unknown at schedule time Middleware portability n n They will propose use cases Linux and Solaris is not enough The code should be written to run on any machine with a reasonable cc/java MPI and Corba Worries about the use of Condor (open source)

Other issues (not tackled) n n Grid global queues or local queues Gateway functionality Other issues (not tackled) n n Grid global queues or local queues Gateway functionality n Sharing of WP 1 and WP 4 tasks on the gateway n n n Language translation Translation of Grid credentials to local credentials WP 4: Will other WPs only query for information of the current state or is there a need to query for future changes ? Central middleware repository Coding convention and tools …

Informal discussions n WP 2 n n Agent-based query optimization WP 4 n n Informal discussions n WP 2 n n Agent-based query optimization WP 4 n n For month 9 they basically agree to consider queues as resources Resource monitoring info n n n We are consumer, they are producer We have to specify the attributes needed for M 9 release, and they will tell us if they can implement the corresponding information providers … depends on what exactly WP 3 plans to deliver concerning information and monitoring service

Condor n n In the first year we are allowed to use the Condor Condor n n In the first year we are allowed to use the Condor stuff Then we can rely on open source software n n n Probably possible for the Condor matchmaking library More complicated for Condor-G Possibility to have the source code

Condor: what’s new n New Condor grid-manager and old Globus job manager (basically ready) Condor: what’s new n New Condor grid-manager and old Globus job manager (basically ready) n n n New Globus job manager (in testing phase) n n n Fixes for existing shortcomings (scalability in the submitting machine, logging, …) Globus GRAM client APIs 2 -phase submit protocol Support for reattaching to an existing job New Condor grid-manager (4 -6 weeks) n Able to exploits the new Globus job manager

Agenda n n n n n Data. Grid topical (IS, DM, Security) workshops at Agenda n n n n n Data. Grid topical (IS, DM, Security) workshops at CERN (week of March 26) Data. Grid technical visit to Globus at ANL April 5 -7 Agent technology workshop in Trento (ITC/IRST) May 17 INET 2001 5 -8 June in Stockholm Euro. Globus 16 -22 June EU-US coordination in Rome on 23/6/2001 Next project conference at Oxford 2 -5 of July GGF 2 in Washington 15 -18 of July HPDC 7 -10 of August in the US

Agenda n n n CHEP 2001 in Beijing (Grid sessions) 3 -7 of September Agenda n n n CHEP 2001 in Beijing (Grid sessions) 3 -7 of September CERN school of Computing (Grid session) 16 -29 September in Santander, Spain 6 th Globus retreat and i. DG-1 17 -21 of September in Chicago Data. Grid+GGF 3 in Frascati+Rome 2 -5 + 7 -10 of October IST 2001 in Brussels end of October-early November (Data. Grid booth, presentations and demos? ) SC’ 2001 10 -16 November (Data. Grid participation? )