Integrated Service and Desktop Grids for Scientific Computing

Integrated Service and Desktop Grids for Scientific Computing Robert Lovas Computer and Automation Research Institute, Hungarian Academy of Sciences Budapest, Hungary rlovas@sztaki. hu Tamas Kiss Centre for Parallel Computing, University of Westminster London, UK kisst@wmin. ac. uk EDGe. S is supported by a Grant from the European Commission's FP 7 IST Capacities programme under grant agreement RI-211727 1

Overview • Bottom-up approach • • Desktop Grids as computational infrastructure • ETICS-2 project (special support for application build/test/validation) • Applications EDGe. S project (bridge between Desktop and traditional grids + application support) 2

Prelude - what do people and SMEs think about grid computing • • Questionnaires all across Europe • Get an idea of the interest in running a Grid inside an SME Get an idea of the interest in people and SMEs to donate computing time for science to a Grid 3

Some results 4

Survey - Conclusions • • Overall: there is interest in Desktop Grid computing in Europe. • • Need to generate trust in the organisation that manages the Grid. • • People want feedback on the application they are running. • Overall the respondents were rather positive about donating computing time for a Grid or about running applications However, that people are willing to change their current practice and say that they want to participate in Grid efforts does not mean that they are actually going to do that. People want to donate computing time for scientific applications, especially medical applications. They do not like to donate computing time to commercial or defense applications. No clear technical barriers perceived by the respondents: so this does not need much attention. 5

A Grid definition Connecting a large number of resources and make them available for a set of applications Resources belong to different owners Heterogeneous system (HW, SW, network) Typically for Desktop Grids: it is the Desktop Grid computer that initiates all communications there is no central control over the desktop computer Most computers in a Desktop Grid are desktop computers, but they can also be notebooks, or servers, or handhelds 6

What is Grid? • Using physical resources belonging to different organisations for a set of applications • Main problem: crossing (administrative) boundaries between the organisations • • security workload management fault-tolerence etc. 7

Resources that can be shared • • Processors - computational Grid HD/Storage - storage Grid Data bases, repositories - data Grid Sensors - sensor Grid 8

Different owners of resources • Citizens’ and SMEs’ PCs Desktop Grid • • Company divisions Enterprise Grid • • Thousands or even millions of owners A dozen or so organisations within one enterprise Universities and research institutes e. Science Grid • Up to a few hundred (in the case of EGEE, a pan-European Grid) 9

e. Science Grid example: EGEE • > 200 sites in 40 countries • > 100 000 CPUs • > 5 PB storage • > 100 k jobs/day • > 200 Virtual Organizations • g. Lite middleware • The World’s largest multi-disciplinary Grid http: //www. eu-egee. org/ 10

Desktop Grid example Almere. Grid - World’s first City. Grid • • Almere is a new town in the Netherlands • Back-Up Grid experiment to share hard disk space (as part of EU BEin. GRID project) • Build-on-Grid and Virtual Reality based environment to allow people to build their own house in virtual space Desktop Grid used for scientific applications (medical research of local hospital): sharing CPU power 11

For science and beyond Applications on the Grid Science Grid e-Education using Grid applications BEin. GRID backup/ restore Build-on-Grid Own house construction Broadband network Health. Grid diagnosis agent based Almere. Grid infrastructure Strengthen knowledge infrastructure of the city 12

Desktop Grids - introduction • Agenda • • Basic techniques Open source examples • • • Xtrem. Web BOINC Commercial examples • • • LSF Desktop Datasynapse Parabon 13

Basic structure • Resource computers • • User system or interface • • The resource computers (also called workers, clients, . . ) execute the work Interface for the Grid user to submit his/her application, monitor the progress, and retrieve results Grid server • Handles the requests from the resource computers to get new work or submit the results • Handles the requests from the users to execute jobs 14

Basic desktop Grid resources Grid server Grid resources Grid users 15

Volunteer desktop Grids • Uses a pull model: it is always the resource computer that asks for work and initiates all communication with the server • There is an application repository: users can only run validated trusted applications • The resources, however, are untrusted and volatile: you are not sure they deliver the correct answer (or even deliver answer at all) 16

Local Deskop Grids • Organisations (universities, companies) can also run an internal Desktop Grid. • • • Can be composed of trusted resources Use spare capacity Can also include servers and clusters 17

Desktop Grid applications • There is – in general – no communication between the resources • Hence you can only run a subclass of parallel programmes: master-worker, parameter sweep or monte-carlo • In volunteer computing the network bandwidth and the HDD/CPU capacity can be rather limited • But there can be large numbers of resources 18

BOINC • Middleware designed for volunteer Desktop grid computing • • http: //boinc. berkeley. edu/ • More than 5 million computers world wide have BOINC installed • Many other projects use BOINC Developed for the largest Grid in the world: seti@home 19

SETI@home • SETI@home • • 3. 8 M users in 226 countries 1200 CPU years/day 38 TF sustained (Japanese Earth Simulator is 32 TF sustained) Highly heterogeneous: >77 different processor types 20

Some other Desktop Grids World Community Grid - IBM (http: //www. worldcommunitygrid. org/) 300. 000 PCs Leiden Classical Grid - Education on Grid (http: //boinc. gorlaeus. net/) 17. 000 PCs SZTAKI - Hungarian Grid (http: //www. desktopgrid. hu/) 70. 000 PCs Almere. Grid (http: //almeregrid. nl) 3. 000 PCs PS 3 GRID (Based on Playstations) (http: //ps 3 grid. net/) Xtrem. Web http: //xtremweb. net 21

BOINC Structure • BOINC is organised around projects: a project is typically one application • Resource computers (BOINC clients) can connect to one or more projects • A Grid server can run more projects: but most only have one or perhaps a few • Web-base user interface is integrated with the project 22

BOINC installation • • Installing a BOINC Grid is not click and play • Easiest way: get one of the distributions. For instance the SZTAKI BOINC distribution. This installs relatively fast if you start with a Debian Linux www. desktopgrid. hu Collection of C++ programmes, PHP scripts and other sources, with many dependencies on operating system, the database, and other tools 23

BOINC Client on resource • • Customisable, easy-to-install Easy view and complex view with a lot of details on what is happening 24

BOINC applications • BOINC applications have to be compiled with calls to a special BOINC API. • There are some wrappers for executables, like genwrapper (developed by MTA SZTAKI) • More extensive APIs exist for instance for master worker-applications: DC-API (developed by MTA SZTAKI) 25

Commercial desktop Grids • • Desktop Grids have commercial success, too • But they do not tell too much about it: it is a competitive advantage… • Several companies provide Desktop Grid middleware; although some now call in differently because they do not want to be associated with the Globus and g. Lite based Grids that by many are seen as failed in the commercial area Many large companies (biotech, engineering, banks, . . ) have installed large-scale Desktop Grids 26

Datasynapse • More than 1 million computers run Datasynapse software in the world. • Making it, probably, the most successful Grid middleware after BOINC • Acquired by TIBCO (Cloud provider) recently for US$28 million 27

And there are more. . . • • • Platform LSF Desktop Parabon Pro. Active: Active. Eon, France Digipede, USA, … 28

Desktop Grid QA • Q. A Desktop Grid consists of otherwise unused computing sources. So it is for free? • • Q. Are Desktop Grids Green? • • A. Yes: less computers have to be built. Q. So we do not need supercomputers anymore? • • A. It is much cheaper than any other solution, but of course if you use a computer it uses power and produces heat A. No. Although there are many applications that can run on a Desktop Grid, some require fast communication between the processors. For those supercomputers are more appropriate Q. It took me a long time as a user to get used to Glite within EGEE. Do I have to learn a new Grid technology • A. No. The EDGe. S project provides a seamless bridge between EGEE and Desktop Grids 29

EDGe. S project background • Service Grids have typically order 10. 000 trusted computer nodes; Desktop Grids have order 100. 000 untrusted computers. • Largest Service Grid: EGEE: 120. 000 processors • Largest Desktop Grid: SETI@Home: 3. 800. 000 processors • Service Grids have untrusted applications but trusted resource providers; Desktop Grids have trusted applications, but untrusted resource providers • http: //edges-grid. eu 30

The problem we address • Very expensive, OGF GIN Supercomputer based SGs (DEISA, Tera. Grid) • small number of sites and large number of CPUs • MPI/Open. MP appls • Moderately expensive, EDGe. S Cluster based service grids (SGs) (EGEE, OSG, etc. ) Desktop grids (DGs) (volunteer DGs – home computers, organizational DGs – institutional desktops) • moderate number of sites and CPUs • any appls • Inexpensive, • very large number of CPUs (~100 K – 1 M) • Bag of task appls 31

EDGe. S Enabling Desktop Grids for e-Science • Connect Service Grids (EGEE) with Desktop Grids (SZTAKI Desktop Grid, Almere. Grid, Extramadura Grid…) to provide even more alternative resources to scientists • Provide a Bridge between these types of Grids for automatic job sharing • Provide an Application Development Methodology to port applications to the Grid; Port a number of (new) applications to the Grid • • Organise users and industry Two year, EU funded project (started 1 -1 -2008) 32

EDGe. S Partners Partner Participant short name Country 1 Computer and Automation Research Institute of the Hungarian Academy of Sciences MTA SZTAKI Hungary 2 Centro de Investigaciones Energéticas Medio Ambientales y Tecnológicas CIEMAT Spain 3 Foundation for the Development of Science and Technology in Extremadura Fundecyt Spain 4 The French National Institute for Research in Computer Science and Control INRIA France 5 University of Westminster Uo. W UK 6 Cardiff University CU UK 7 Faculty of Sciences and Technology of the University of Coimbra FCTUC Portugal 8 Stichting Almere. Grid The Netherlands 9 Centre National de la Recherche Scientifique Institut National de Physique Nucleaire et de Physique des Hautes Energies IN 2 P 3 France 33

EDGe. S - Grid infrastructure 34

Generic Grid-Grid (3 G) Bridge to integrate SGs and DGs Job source g. USE submitter DC-API Plugin g. USE/ WSPGRADE portal Queue Manager BOINC EGEE Plugin EGEE Grid. Handler Interface Job source EGEE GRAM Job Database Job Handler Interface BOINC Job source BOINC client EGEE Xtrem. Web Plugin 3 G Bridge Xtrem Web

For EGEE users wanting to run on Desktop Grids Bag-of-tasks type of EGEE applications are selected and ported to BOINC and Xtrem. Web Porting is a joint work of EDGe. S and the application developer No certificate support in BOINC: all applications are validated by the EDGe. S team Validated applications are placed in the EDGe. S Application Repository Desktop Grids connected by EDGe. S can get applications from the EDGe. S Application Repository EGEE users can take applications from the Application Repository and submit via the EGEE->Desktop Grid bridges 36

For Desktop Grid users wanting to run jobs on EGEE EDGe. S did create a new Virtual Organisation in EGEE (VO) called as EDGe. S VO Sites in the EDGe. S VO support the applications of the EDGe. S connected Desktop Grids that are available in the Application Repository Jobs of the connected Desktop Grids can be seamlessly executed in the EDGe. S VO 37

EADM - EDGe. S Application Development Methodology 1. Analysis of current application 2. Requirements analysis 3. Systems design 4. Detailed design 5. Implementation 6. Testing 7. Validation 8. Deployment 9. User support, maintenance & feedback

The ETICS System INFSO-RI-223782 • • • Continuous software build, test and QA verification system Set of core tools and services Easily extensible with additional plugins Scheduled (cron) or on-demand build and test jobs User computers or remote distributed infrastructures Built-in connectors to distribute build and test jobs on different types of infrastructures from standard job management systems to the grid • Open repository of configuration metadata, packages and build, test and QA reports • Support for standard package management systems like YUM • It’s multi-platform and independent from any specific language, build or test tool ETICS Overview - Module 01 OGF 25 - Catania, 2 - 6 March 2009 39

Architecture ETICS Overview - Module 01 INFSO-RI-223782 OGF 25 - Catania, 2 - 6 March 2009 40

The Configuration Web Client ETICS Overview - Module 01 INFSO-RI-223782 OGF 25 - Catania, 2 - 6 March 2009 41

Local and Remote Operations INFSO-RI-223782 • Build and test jobs can be executed on the user computer • Or sent to remote distributed infrastructures • ETICS has connectors for different types of infrastructures (like the grid or clouds) and different job management system (like PBS, LSF or Condor) • The jobs can be scheduled using standard cron jobs and the ETICS command-line client • All necessary dependencies are handled by ETICS • Developers can build or test locally their software on a single platform and then submit the same build or test to be executed in parallel on multiple platforms ETICS Overview - Module 01 OGF 25 - Catania, 2 - 6 March 2009 42

The Repository INFSO-RI-223782 • ETICS provides an online repository where all software packages and build, test and QA reports are stored • The Repository generates dynamic YUM repository at every build to automate the execution of deployment, integration and functional tests • It can be browsed and queried using a web application with a REST interface, a command-line client or a mountable file system (AFS-based) ETICS Overview - Module 01 OGF 25 - Catania, 2 - 6 March 2009 43

The Repository Web Application ETICS Overview - Module 01 INFSO-RI-223782 OGF 25 - Catania, 2 - 6 March 2009 44

Extensibility and Plugins INFSO-RI-223782 • ETICS comes preconfigured with extensive build and test features • It can also be extended by adding additional plugins • The plugins are provided by ETICS and by its user community and can be easily installed from the online repository ETICS Overview - Module 01 OGF 25 - Catania, 2 - 6 March 2009 45

INFSO-RI-223782 Examples of Plugins Metrics Type Programmi ng languages/ technologi es Complexity static Java Python Javancss JCcn. Plugin Py. Complexity. Pl ugin. py Design quality static Java Jdepend JDepend. Plugin N of potential bugs static C/C++ Python Perl PHP Java Flawfinder, RATS PMD Findbugs CFlawfinder. Plug in CPy. Php. Rats. Plu gin JPmd. Plugin JFindbugs. Plugin N of potential bugs dynamic C/C++ Valgrind CValgrind. Plugin Lines of code static All SLOCCou nt SLOCCount. Plug in dynamic Java Emma Cobertura JUnitemma. Plugi n JCobertura. Plugi n JUnit Py. Unit JUnit. Plugin JUnitreports. Plug- Catania, 2 - 6 March 2009 OGF 25 in. py Py. Unit. Plugin. py Coverage Unit tests dynamic Java success rate. Overview - Module 01 Python ETICS Tool ETICS Plugin 46

Distributed Testing INFSO-RI-223782 • One of most powerful features of ETICS is the “Distributed Testing” or “Parallel Testing” • This features allows to design complex tests composed of several interacting nodes and to deploy automatically complete test environments • This feature is currently prototyped and will be available to the ETICS user community in the end of 2009 • The solution is based on MTA SZTAKI’s workflow designer ETICS Overview - Module 01 OGF 25 - Catania, 2 - 6 March 2009 47

INFSO-RI-223782 Benefits from EDGe. S’ point of view Software quality and reliability of the Desktop Grid enabled applications to be executed on donor’s computer is a key issue • highly heterogeneous computational platform with limited capacities and some vulnerabilities Mostly third-party applications must be investigated before their execution on the production infrastructures • EDGe. S application validation procedure Migration of some key components of EDGe. S Grid middleware (SZTAKI Desktop Grid, 3 G Bridge) under the ETICS would ease the build, deployment, test and maintenance of these components 48

EDGe. S Components for testing in ETICS INFSO-RI-223782 • BOINC server Developed by Uni. of Berkeley • The core of the DG system • Packaged by SZTAKI for Debian (or source packages) • • • DC-API Requires: My. SQL, Apache, … Developed by SZTAKI • For easier programming of desktop grid applications • • Requires: BOINC packages • Applications • • • Developed by third-parties Client side must be compiled on various platforms Master side only for the server • Requires: DC-API • BOINC Client • Developed by Uni. Of Berkeley 49

Minimal test scenario 1 server with dependencies 1 application (Upper. Case) 1 client 1 test for join/exec INFSO-RI-223782 Node 1 Node 2 Node 1: Boinc server installation with its dependencies server IP address to the Boinc client on Node 2 directory for application registration Node 1: Application registration Application/project name to the test on Node 2: Client install with config 50

INFSO-RI-223782 More information: http: //www. eticsproject. eu

The EDGe. S Application Support Service • Aims and objectives: – facilitate porting of applications to the EDGe. S infrastructure by targeting • user communities already associated with DGs or EGEE • new user communities • Activities: – identify user communities that require the resources of the EDGe. S infrastructure – develop a generic methodology for application porting – provide a service in order to aid the migration to and running of applications on the EDGe. S infrastructure 52

The EDGe. S User Community • Academic user community: – 16 research communities and/or teams to be supported • 10 teams with links with project partners • 6 new teams identified by the project consortium – Establishing the EDGe. S user forum • Industrial user community – 8 companies to be supported • 5 companies with links with project partners • 3 new companies identified by the project consortium – Establishing the EDGe. S industry forum 53

Application Porting community number of applications academic non-academic bioscience engineering 7 5 6 5 1 physics business 3 1 2 1 1 audio & image processing TOTAL 4 3 1 20 16 4 54

Application Porting I. Application Organisation Runs on DG Runs from DG to EGEE to DG Video Stream Analysis in Correlation a Grid Environment Systems Ltd (VISAGE) Israel √ √ Digital Alias-free Signal Processing University of Westminster √ √ Analysis of Genotype Data (Plink) Atos Origin, Spain √ √ E-Marketplace Model MTA SZTAKI Integrated with Logistics (EMMIL) √ √ √: actual √: plan 55

EMMIL: e-marketplace modeling • • • buy several products from suppliers find logistic service providers who can deliver the goods minimize overall price: price of goods + price of delivery 56

Parameters for the optimization • Number of different products • Quantity of each product • Number of suppliers • Container size • Price bids from suppliers: – Product price – Delivery fix cost – Delivery variable cost 57

Mathematical model Qi k P ik Purchased quantity of product i from seller k Unit-price of product i at seller k as a step function of quantity Δk Discount given as a step function after the total purchase cost at seller k xjl є {0, 1} decision variable xjl =1 offer j of 3 PL l is selected as winner 58

Application Porting II. Application Organisation Runs on DG Runs from DG to EGEE to DG Anti-cancer Drug Design SZTAKI (Cancer. Grid) √ √ Cellular Automata based University of Laser Dynamics (CALD) Seville and Uo. W √ √ Signal and Image Processing using GT 4 Tray Forschungs. Zentrum Karlsruhe √ √ Protein Molecule Simulation using Autodock University of Westminster √ √ √: actual √: plan 59

Protein Molecule Simulation PDB file 1 (Receptor) PDB file 2 (Ligand) Phase 2 Check (Molprobity) Phase 3 Energy Minimization (Gromacs) Validate (Molprobity) Perform docking (Auto. Dock) Phase 1 Phase 4 Molecular Dynamics (Gromacs) 60

Protein Molecule Simulation g. USE WS PGRADE portal Uo. W Local Desktop Grid G-USE DG BOINC Server Submitter submits jobs and retrieve results via 3 G Bridge Pulls work units Adds user credentials Define input parameters and number of work units Submits as job to EGEE WMS BOINC workers End User BOINC to EGEE bridge Workers: Download executable and input files Upload: result EGEE EDGe. S VO 61

Application Porting III. Application Organisation Runs on DG Runs from DG to EGEE to DG Distributed Audio Cardiff Retrieval using TRIANA University (DART) √ √ Fusion Plasma Application (ISDEP) BIFI √ √ √ 3 -D Video Rendering using Blender University of Westminster √ √ √ Profiling Hospitals in the University of UK based on Patient Westminster Readmission √ √: actual √: plan 62

3 D Video Rendering (Blender) • Goal: create images and videos from models designed in Blender (one of the most popular Open Source 3 D graphics tools) 63

Application Porting IV. Application Organisation Runs on DG Runs from DG to EGEE to DG Extraction of x-ray diffraction profiles University of Extramadura and Uo. W √ √ UC-Explorer (Universality Classes of Non-equilibrium systems) KFKI-RMKI and SZTAKI √ √ Protein Functional Analysis CNR √ √ Crystal defect aggregation in metals Kurdymov Institute √ √ √: actual √: plan 64

Application Porting V. Application Organisation Runs on DG LHC@home – Large Hadron Collider Queens Mary and Uo. W √ WISDOM – Drug Discovery WISDOM team and Uo. W √ √ Vis. IVO - Visualisation Interface to the Virtual Observatory Catania and Uo. W √ √ DEMOS - Discrete Event NTNU and Modeling on Simula SZTAKI Runs from DG to EGEE to DG √ √: actual √: plan 65

Conclusion • Build your on cost-efficient Desktop Grid based on BOINC • Join the EDGe. S community / forum • Port your application with EADM • Apply ETICS services for testing/validation • Run your simulation / analysis on EDGe. Sbased infrastructure 66

Thank you for your attention … Any question? For more information please visit the EDGe. S Website: Join the EDGe. S User & Industry Forum here! http: //www. edges-grid. eu/ Please contact us if you need support in porting your application to EDGe. S! Email: Tamas Kiss, kisst@wmin. ac. uk 67