Team 8 ISSGC 05 Project Claudia CORONNELLO University

Team 8 ISSGC’ 05 Project Claudia CORONNELLO University of Palermo Italy Christos FILIPPIDIS NCSR “Demokritos” Greece Dimitrios KORENTZELOS Glasgow Caledonian University UK Homayoun POURHEIDARI Serono Switzerland Lin YANG University of Edinburgh UK Jun ZHAO University of Manchester UK

The “ONE” Problem Dealing with Massive Amounts of Geographically Distributed Data Across Organizations

Global Presence We are a global organisation of 140 employees supporting 4’ 600 users in more than 60 locations and from 30 different countries in the world. North America Ø Canada Ø USA Ø Puerto Rico Japan Asia-Pacific Ø Singapore Ø Korea Ø Hong-Kong Ø Taiwan Ø Thailand Ø China Latin America Ø Argentina Ø Brazil Ø Uruguay Ø Venezuela Ø Colombia Ø Mexico Oceania Ø Australia Ø New-Zealand Europe, Middle-East, Africa Ø Ø Ø Switzerland France UK Germany Austria Netherlands Ø Ø Ø Sweden Danemark Finland Norway Czech Rep. Slovakia Ø Ø Ø Poland Lithuania Russia Croatia Italy Spain Ø Ø Ø Portugal Greece Israel South Africa Egypt Tunisia Ø Ø Ø Algeria Morocco Turkey Jordan Saudi Arabia United Arab Em.

Issues and Solutions Data Management and Manipulation Ø Transfer Ø Replication Ø Coordination Ø Collaboration Organize the Problem into 2 Tracks Ø Compute Intensive (e. g. cycles to run the jobs, etc. ) Ø Data Intensive (e. g. replication, access, etc. )

Compute Intensive Requirements Ø Resource Management – Condor (Persistant Resource Pools) , GT 4 (Indexing based on WSRF) , Unicore Ø Job Management (Create and Deploy Jobs) – Condor, GT 4, GLite/LCG, Uni. Core Ø Schedule Jobs – Condor, GLite (WM) Ø Prioritize Jobs – Condor (Match. Maker Community Policy), GT 4 (Community Scheduler), Glite (WM) Ø Security (Authn/Authz, Safe Execution) – GT 4 (GIS), Uni. Core, Condor Sand. Box (via GIS) Ø Monitor the jobs and their progress – Condor, GLite (BDII), Uni. Core (WSRF operations) Ø Scalability of the System – Condor (Agents, Resource, and Match. Makers are independent), GT 4, Uni. Core

Data Intensive Requirements Ø Replication Management – GT 4 (Replica Mgmt. Services) provides • • Creation Registration Location Mgmt of dataset replicas Ø Data Transfer (High Speed, Reliable, and Just in Time) – GT 4 (Grid. FTP) • Parallel data transfer over TCP streams • Stripped and Partial file transfer Ø Data Access (Discoverable, Reliable) – GT 4 (OGSA-DAI) Ø Security (Authentication, Authroization, and Secure Transfer) – GT 4 (GSI, GSS, Kerberos, etc. ) – HTTPS, X 509

Middleware Puzzle USER APPLICATIONS Condor-G Globus Toolkit Community Authorization Service Portal HLS UNICORE / GS OGSA-DAI WS Authentication Authorization RTF Python WS Core C WS Core MANGEMENT Pre-WS Authentication Authorization Grid. FTP Credential Management RLS Pre-WS GRAM SECURITY DATA MANAGEMENT EXECUTION MANGEMENT WS GRAM Condor GRID FABRIC MDS 4 MDS 2 Java WS Core C Common Libraries INFORMATION SERVICES VDT My. Proxy, Py. Globus , Monalisa, … ECONO Broker OGSA BES Delegation Service g. Lite Workload Manager BDII RGMA Cataloguing

Progressive Exercise Ø Task: to find the pillars on the surface and the texts that are embossed or etched on the top surface of each pillar Ø Progress – Running scanner over the fifth and sixth points from the deep. Thought. II. txt data file, with radius = 10. 0 and step = 1. 0 – This returned points that are on the top of the pillars – Chose one of the points from the result set and defined a small scoped box to perform “Regular” function over this box – Repeatedly shrinking the scale of the box boundary in order to amplify the pillar and the text Ø Results – Could not find any texts on these two points – Then we tried the first point in this text data file Ø Future work – Write a script to automate this searching process in order to scan points from these two data files and find the knowledge

Lessons Learned Ø Grid technology – – File. Store Big. Probe Regular – Computation intensive – Data intensive – Team collaboration: shared knowledge and human resource – Cross-team collaboration: shared workspace, resources Scanner Ø “People Grid” worked Searching Knowledge Client Pillars. Of. Wisdom Web service J 2 EE XML, WSDL, SOAP Distributed job scheduling, allocation and management – Distributed data management – Managed services Ø Facing reality – Un-stability: broken server delays work progress – Security is not always there File. DB 1 File. DB 80

Middleware Puzzle Team Leader WS Java Coding Presentation xx Algorithm xx Claudia CORONNELLO xx Christos FILIPPIDIS x xx Dimitris KORENTZELOS x xx x x Homayoun POURHEIDARI x x xx Lin YANG xxx xx x Jun ZHAO xxx xx x

Feedback Ø Collaborative Environment Ø Lectures (need improvement) Ø Better Preparation/Organization Ø Exposure & Training to Grid Technologies Ø Good Facilities (Excellent Lab) Ø Good Location Ø Better Food (need better wine too : )