Science Gateways and Clouds Dr Wibke Sudholt Cloud

Science Gateways and Clouds Dr. Wibke Sudholt Cloud. Broker Gmb. H Technoparkstrasse 1, CH-8005 Zurich, Switzerland Phone: +41 44 633 79 34 Email: wibke. sudholt@cloudbroker. com Web: http: //www. cloudbroker. com SCI-BUS is supported by the FP 7 Capacities Programme under contract no. RI-283481

Presentation Outline • Cloud computing • Science gateways • Cloud. Broker Platform • SCI-BUS project • Gaa. S = Gateway as a Service © Cloud. Broker Gmb. H All rights reserved. EGI Technical Forum 2012 20. 09. 2012 / 2

Cloud Computing SCI-BUS is supported by the FP 7 Capacities Programme under contract no. RI-283481

Cloud Definition • Self service: Access to computer resources on demand, without much initial investment in time, money or expertise • Pay per use: Only pay for what you actually use in small steps, Op. Ex instead of Cap. Ex, micropayments • Scalability: Nearly unlimited elasticity, grow and shrink the resource size upon request ⇒Not only change in technology, but change in business model ⇒Interface between providers and users set at the right place © Cloud. Broker Gmb. H All rights reserved. EGI Technical Forum 2012 20. 09. 2012 / 4

Cloud Services Saa. S = Software as a Service • Scientific / office / business / etc. applications • Salesforce, Google Apps, etc. Paa. S = Platform as a Service • Programming / deployment frameworks, distribution / messaging / monitoring systems, databases, etc. • Google App Engine, Windows Azure, etc. Iaa. S = Infrastructure as a Service • Virtual machines, storage, network, etc. • Amazon EC 2 and S 3, Rackspace, etc. © Cloud. Broker Gmb. H All rights reserved. EGI Technical Forum 2012 20. 09. 2012 / 5

Types of Clouds Public Clouds Private Clouds • In-house or hosted • Just for internal use • Multi-tenant across organization sites, departments, groups and users • Focus on selfservice and accountability © Cloud. Broker Gmb. H All rights reserved. Hybrid Clouds • Mixture of public and private clouds • Focus on scalability and failover • Offered by partner organizations or cloud providers • Available to certain external user communities or to everybody • Multi-tenant across organizations • Focus on ondemand payper-use EGI Technical Forum 2012 20. 09. 2012 / 6

Science Gateways SCI-BUS is supported by the FP 7 Capacities Programme under contract no. RI-283481

Gateway Description • • Portals to scientific applications for scientific end users, hiding the infrastructure complexity in the background Usually focus on a certain scientific domain, application area, geographic region, user group, organization or similar Front-ends might be web-based, client software, mobile apps, APIs, CLIs, etc. Back-ends are typically distributed computing infrastructures (DCIs) such as clusters, supercomputers, grids or clouds, but can also include sensors, etc. Additional functionalities such as data handling, workbenches, workflows, parameter sweeps, visualization, graphs, user management, community interaction, social media, publications, reporting, etc. Often have domain-specific functionalities such as naming, annotation and display of items, metadata, item connections, access to content databases, etc. Example scientific domains include molecular science, proteomics, medical imaging, astrophysics, seismology, rendering, etc. Example providers include academic institutes, scientific communities, research projects, NGIs, companies, etc. © Cloud. Broker Gmb. H All rights reserved. EGI Technical Forum 2012 20. 09. 2012 / 8

© Cloud. Broker Gmb. H All rights reserved. Private Public Desktop Grids (BOINC, Condor, etc. ) Institutional Grids (Globus, UNICORE, etc. ) Clusters, Supercomputers (SGE, PBS, LSF, etc. ) User Management, Community Visualization, Reporting Workflows, Parameter Sweeps Data Handling, Metadata Applications, Databases Gateway Architecture User Science Gateway Technology (Liferay, WS-PGRADE/g. USE, etc. ) DCIs Clouds Saa. S Paa. S Iaa. S EGI Technical Forum 2012 20. 09. 2012 / 9

Cloud. Broker Platform SCI-BUS is supported by the FP 7 Capacities Programme under contract no. RI-283481

Cloud. Broker Platform • Web-based application store for the deployment and execution of scientific and technical software in the cloud • On demand, pay per use, browser / programmatic / command-line access, cross-domain • Uses infrastructure as a service (Iaa. S) from resource providers • Offers platform as a service (Paa. S) for software vendors and software as a service (Saa. S) for end users • Easy to use, speeds up time to market, no need for own HPC infrastructure, Op. Ex instead of Cap. Ex © Cloud. Broker Gmb. H All rights reserved. EGI Technical Forum 2012 20. 09. 2012 / 11

Platform Architecture End Users, Software Vendors, Resource Providers CLI Chemistry Applications Java Client Library REST Web Service API Biology Applications Health Applications Engineering Applications … Applications Cloud. Broker Platform Amazon Cloud © Cloud. Broker Gmb. H All rights reserved. IBM Cloud Open. Stack Cloud Eucalyptus Cloud. Broker Integration Web Browser UI User Tools … Cloud EGI Technical Forum 2012 20. 09. 2012 / 12

Clouds and Applications Cloud Resources • • Use your own resources and/or resources offered by others Configuration in platform needed for own resources: – • • • Amazon EC 2 / S 3 IBM Smart. Cloud Enterprise © Cloud. Broker Gmb. H All rights reserved. Use your own software on your own resources and/or software offered by others on resources offered by others Any batch-oriented Linux command-line application: – • • Amazon EC 2 IBM Smart. Cloud Enterprise Software offered by Cloud. Broker: – • Executables, deployments, prices Auto-deployment with installation script: – – • Multi-threading and MPI possible Configuration in platform needed for own software: – Open. Nebula Resources offered by Cloud. Broker: – – • Open. Stack (via EC 2 API) Eucalyptus / Walrus Support in preparation: – • Amazon EC 2 / S 3 IBM Smart. Cloud Enterprise / Nirvanix Currently supported private clouds / storages: – – • Instance types, regions, storages, prices Currently supported public clouds / storages: – – Application Software BLAST, Auto. Dock, Gromacs, X! Tandem, Open. FOAM Also already ported: – Rosetta, Trans. AT, Siesta, T-Coffee, Fold. X, R EGI Technical Forum 2012 20. 09. 2012 / 13

Platform Features • Computations: • – Job creation, submission and control – Data file upload and download – Automatic queuing, instance startup and shutdown, data transfer (via cloud storage) and fault tolerance – Pipeline jobs, archives and tagging • • – Registration (with user check in public platform) – User authentication with email and password – SSL transport-layer encryption – Dashboard, activity logs – User and organization management – Different organization, user and visibility levels – Resource access and software license limits – Invoicing and billing, pre-payment or monthly payment – Documentation Interfaces: – – Web browser UI REST web service API Java client library CLI Marketplace: – Platform usage surcharges as percentage of resource and software prices – Free resources and software have zero surcharges © Cloud. Broker Gmb. H All rights reserved. Further features: • Platform flavors: – Public installation under https: //platform. cloudbroker. com – Also hosted and in-house installations offered EGI Technical Forum 2012 20. 09. 2012 / 14

Reference Examples • Protein modeling with Rosetta (collaboration with ETH Zurich and IBM) • Computational fluid dynamics web UI for Trans. AT (collaboration with ASCOMP and HSR) • Platform node for KNIME (Konstanz Information Miner) workflow framework • Access to clouds in SCI-BUS (SCIentific gateway Based User Support) EU FP 7 project • Further projects in industry, government and academia © Cloud. Broker Gmb. H All rights reserved. EGI Technical Forum 2012 20. 09. 2012 / 15

SCI-BUS Project SCI-BUS is supported by the FP 7 Capacities Programme under contract no. RI-283481

SCI-BUS Setup • • EU FP 7 project http: //www. sci-bus. eu Running 10/2011 -09/2014 Consortium: – 15 partners from 10 countries (10 academic, 5 industrial) – 6 subcontractors – Additional associated partners (via Mo. U) • Goals: – Provide researchers seamless access to major computing, data and networking infrastructures and services – Develop gateway technology – Support community gateways – Focus on scientific workflows © Cloud. Broker Gmb. H All rights reserved. • Generic-purpose technologies: – WS-PGRADE/g. USE from MTA SZTAKI (see http: //www. guse. hu and presentation of Zoltan Farkas in the morning) – Cloud. Broker Platform • Initial gateway communities: – Systems biology, computational chemistry, astrophysics, heliophysics, seismology, medicine, rendering, electronic document handling, business process optimization, SMEs, software testing • SCI-BUS booth at EGI Technical Forum 2012: – Cloud. Broker Platform demo at 15: 40 today EGI Technical Forum 2012 20. 09. 2012 / 17

SCI-BUS Architecture Source: SCI-BUS © Cloud. Broker Gmb. H All rights reserved. EGI Technical

WS-PGRADE/g. USE Source: http: //www. guse. hu/? m=architecture&s=0 © Cloud. Broker Gmb. H All

Cloud. Broker Platform Integration in WS-PGRADE/g. USE • Method: – Use of the Cloud. Broker Platform Java client library • Existing features: – Cloud. Broker DCI Bridge backend plugin and configuration interface – Cloud. Broker security portlet – Concrete workflow portlet extension for Cloud. Broker jobs • Upcoming features: – Support for running any kind of application via wrapper script – Display of prices, costs and invoices – Different types of user mapping Source: Zoltan Farkas, MTA SZTAKI © Cloud. Broker Gmb. H All rights reserved. EGI Technical Forum 2012 20. 09. 2012 / 20

Gaa. S = Gateway as a Service SCI-BUS is supported by the FP 7 Capacities Programme under contract no. RI-283481

Gateway Sustainability • Over time, science gateways need to become sustainable • It does not make sense to offer a gateway if there is not enough user interest in it • There are more and more moves towards also metering and billing academic services • Many governments don’t have enough money for large Cap. Ex anymore and thus prefer stepwise Op. Ex • Sponsors are hard to attract to science gateways • Not always gateways can be outsourced or a spin-off company generated from them ⇒ Base government funding to build, operate and support science gateways is needed ⇒ Beyond that, apply the cloud business model to science gateways © Cloud. Broker Gmb. H All rights reserved. EGI Technical Forum 2012 20. 09. 2012 / 22

Gateway Business Models • Build science gateways from cloud building blocks (model used in the Cloud. Broker Platform and SCI-BUS) • Charge for not freely accessible resource consumption and application usage (model used in the Cloud. Broker Platform and SCIBUS) • Have a freemium model with extra charges for additional services (model used in the Cloud. Broker Platform and SCI-BUS) • Offer user subscriptions with different features • Let commercial and other external users pay for gateway services • Have a shareholder charge-back model • Provide professional consulting, training and support ⇒ Develop your own business model © Cloud. Broker Gmb. H All rights reserved. EGI Technical Forum 2012 20. 09. 2012 / 23

Thank you! – Questions? Try out the Cloud. Broker Platform under https: //platform. cloudbroker. com Visit SCI-BUS under http: //www. sci-bus. eu SCI-BUS is supported by the FP 7 Capacities Programme under contract no. RI-283481