EGI-In. SPIRE EGI Applications Database Status Report Marios Chatziangelou, et al. mhaggel@iasa. gr Institute of Accelerating Systems and Applications (IASA) www. iasa. gr 30/05/14 EGI-In. SPIRE RI-261323 1 www. egi. eu
Latest developments (1) Multi-content layout – Software Marketplace (apps, tools, m/w products, science GWs and WFs ) – Cloud Marketplace (apps devel/servers/stacks, big data, infra, etc. . ) – People/Researchers registry (coord, sw eng, net eng, sysadmins etc. . ) EGI-In. SPIRE RI-261323 www. egi. eu
Latest developments (2) • Virtual appliances (VAs) – – – VA Registration VMIs Registration per VA VMCatcher compatible Image lists creation Required devels for: • User interface/Portal • Backend (db) • API (xml/json) • Development of a separate sub-service for handling submitted Image lists using vmcaster command line tool, as an alternative to the graphical way through the App. DB portal (documentation). [ https: //vmcaster. appdb. egi. eu/ ] • Deployment of an App. DB dedicated Wiki sub-service [ https: //wiki. appdb. egi. eu/ ] – Documentation in good shape. Suitable for: • Users/visitors or submitters • Resource Providers/Site admins • VO managers EGI-In. SPIRE RI-261323 www. egi. eu
Latest developments (3) VO wide image lists Editable only by VO managers VO wide image lists always ‘private’ – a personal access token required Simplified workflow The user submits on or more VAs – a VO manager is able to include or not to a VO wide image list The user updates a VA – a VO manager is able to include the udpate or not. EGI-In. SPIRE RI-261323 www. egi. eu
Latest developments (4) Integration with the information system (top. BDII) Retrieve info related to: – Sites which are part of the Fed. Cloud infra – Available templates per site – Available images per sites that supports a specific VO Usage details through App. DB UI: – site endpoint – template ID – occi id EGI-In. SPIRE RI-261323 www. egi. eu
Latest developments (5) • Authentication – – – Three sub-services into App. DB ecosystem: • Portal: https: //appdb. egi. eu • VMcaster CLI dashboard: https: //vmcaster. appdb. egi. eu • Wiki: https: //wiki. appdb. egi. eu Technology used: simple. SAMLphp N Service Providers (SP) – one per appdb sub-service 1 Identity Provider (Id. P) – restricted for *. appdb use only Single-Sign-On for the *. appdb sub-services Support for many authentication sources (egi ldap, fb, linkedin, g+, …, plus x 509 support) – SP ↔ Id. P communication using SAML 2 protocol (Shibboleth is also an option) – • Easy to integrate *. appdb SPs with 3 rd-party Id. P(s) – Fed. Cloud AAI User Profile/Account mapping – – N accounts (of any authentication sources kind) ↔ 1 App. DB profile EGI SSO ↔ x 509 auto-connect functionality “Connect to an existing” or “Create new profile” is up to the user Secured connection process (send confirmation code with 30’ TTL) EGI-In. SPIRE RI-261323 www. egi. eu
Latest developments (6) Authorization & Privacy: Group-based, default permission set (groups: admins, managers, NILs, power users, owners, contacts, users) Adjustable permissions per user per associated item (software or virtual appliance) • • edit, information & publication related info manage, software releases or VA versions respectively access, VA versions private data full control Privacy is available only for a subset of VA related metadata – easily to be extended to software item metadata as well Easy to use, self-explanatory, GUI NOTE: to collect all the Auth. Z VO related attributes (i. e. VO roles & membership) an integration with Operations portal has been realized EGI-In. SPIRE RI-261323 www. egi. eu
Latest developments (7) Only for Virtual Appliances Set permissions for the Contacts Set permissions for explicit users (no contacts) EGI-In. SPIRE RI-261323 www. egi. eu
Latest developments (8) By policy, system groups have additional permissions to the item EGI-In. SPIRE RI-261323 www. egi. eu
Latest developments (9) • ‘Personal Access Tokens’ for Authoritative API calls Personal Access Tokens for API calls using vmcatcher for subscribing at private image lists [details] EGI-In. SPIRE RI-261323 www. egi. eu
Cloud marketplace: The Workflow • Search for v. Appliances RPs/Sites • Download images cloud mw • Get usage details • Start/Stop/…. an image push info site-bdii perform occi cmds vmcatcher subscribe and/or get image_list updates (optional) subscribe and/or get image_list updates users Store: VO retrieve info Store: v. Appliance vmcaster sub-service App. DB portal Top-BDII generate image_list App. DB Cloud Marketplace • Republish VO-wide image list • Publish new v. Appliance version • Select/update/exclude v. Appliances • Insert/update/remove image(s) • Create a new v. Appliance version • Register a v. Appliance App. DB portal EGI-In. SPIRE RI-261323 VO Manager submitter www. egi. eu
Current situation • Not well established development roadmap yet • In the process of: – Gathering requirements – Collecting ideas EGI-In. SPIRE RI-261323 www. egi. eu
Ideas/requirements until now (1) 1 Support for ELIXIR project – Creation and support of the ELIXIR VO – Integrate with EBI Id. Ps 2 Support of Fed. Cloud AAI & edu. GAIN – Integrate with Fed. Cloud AAI – Support edu. GAIN 3 Extend the App. DB notification system – Submitters notify VO managers to include their VAs into VO-wide image list (manual action) – Users & VO managers, get notified that their VAs is about to expire (automated action) Started – VO Managers notified that a VA has been deleted by the user (automated action) – VO Managers notified about an update of a VA that is already included into the VO-wide image list (automated action) 4 Started Finalize the integration with the information system – The work on the App. DB side is ready Started – Sites should perform an auto update of the information system on metadata or image change (if there are no any surprises, the actual development needed on site-bdii scripts and vmcatcher fronts) EGI-In. SPIRE RI-261323 www. egi. eu
Ideas/requirements until now (2) 5 Monitoring – App. DB as a service – VMCatcher installations to sites 6 Brokering functionality – Offer (basic? Start/stop/…. ) brokering features through the App. DB service – Monitor the instantiated VMs by the user 7 Started Contextualization – Associate contextualization scripts with registered images – Images that are reference to other images (this is still under discussion) 8 App. DB & CVMFS – Deploy a dedicated stratum 0 node for the software hosted by the App. DB – Provide access (read/write) through the portal 9 Improve usability on App. DB repositories – Change the UI on the repositories segment of the service for better usability EGI-In. SPIRE RI-261323 www. egi. eu
Ideas/requirements until now (3) 10 Site & Project wide views (as it is now for VOs) – Introduce entities such as Sites & Projects within the App. DB – Provide info about which SW or VAs is provided/offered/supported by each registered Site and/or Project 11 VMCatcher related developments – issue on new image additions to a subscribed image list – a couple of bug fixes 12 VMCaster related developments – use PAT (default - in parallel with SSO and x 509) for uploading image lists to the App. DB – Option for uploading non-signed image lists to the App. DB (will be needed for ELIXIR case) – Discuss the default expiration date that is set by vmcaster 13 RT tickets - 15 open tickets (as of 29/5/2014) – 5 should be set as closed – 6 need to be re-evaqluated – 4 should stay and be part of the development plan (an email has been sent to Diego + UCST on 29/5/2014) EGI-In. SPIRE RI-261323 www. egi. eu
Effort allocation • Total effort: 6 PMs • List of the involved personnel and the PMs that will be allocated for each one, will be available in the upcoming days EGI-In. SPIRE RI-261323 www. egi. eu
Thank you!! Questions EGI-In. SPIRE RI-261323 www. egi. eu
Скачать презентацию EGI-In SPIRE EGI Applications Database Status Report Marios
7680e21275d55eee552e5135bdd82d83.ppt