INFN/IGI contributions Federated Clouds Task Force F 2 F meeting November 24, 2011, Amsterdam
Deployment Scenario • 1 shared CREAM-CE We will move to 1 dedicated CREAM-CE within the end of November • • • Torque/MAUI batch scheduler 1 Name Server (WNo. De. S dedicated service) 2 physical servers: Virtual Machin e e Could host up to 24 concurrent virtual nodes • Scientific Linux SL release 5. 7 (Boron) as Operating System – 1 Core per machine – 1. 5 GB of memory We can have different images one per each VO – • Standard job WN WNo. De. S jobs s CREAM CE Non WNo. De. S WNs job Batch Server Manager ard Functional test => OK – Both with local account submission and Grid submission Stress test => OK – Few submission of ~2000 jobs in one shot, all of them completed fine WNo. De. S enabled WNs nd Test status: WNo. De. S Name. Server Sta Each image is shared among all the VOs (i. e. , DTEAM, enmr. org) Customized images need to be decided together with the site • WN WN WN At the moment we have one single image with
Scenario 1 • • The provided testbed supports the provisioning of VMs by using GRID Prerequisite – – • Users MUST belong to a given VO (i. g. , DTEAM) Users MUST have a x. 509 digital certificate How to submit user’s job to CE (i. g. , cremino. cnaf. infn. it) directly as follows: – editing run_job. jdl file: Executable="run_job. sh"; Arguments=“arguments”; Std. Output="report_out. txt"; Std. Error="report_err. txt”; Input. Sandbox={"run_job. sh"}; Output. Sandbox={"report_out. txt", "report_err. txt"}; Output. Sandbox. Base. Dest. URI = "gsiftp: //gftp 2. ba. infn. it/lustre/cms/test_grid/”; – running commands: > voms proxy init voms dteam > glite ce job submit a r cremino. cnaf. infn. it: 8443/cream pbs cloudtf run_job. jdl > glite ce job status https: //cremino. cnaf. infn. it: 8443/CREAM 422160619
Scenario 1 • • • The provided testbed supports the provisioning of VMs by using GRID Prerequisite – Users MUST belong to a given VO (i. g. , DTEAM) – Users MUST have a x. 509 digital certificate How to submit user’s job to CE (i. g. , cremino. cnaf. infn. it) as follows: – editing run_job. sh file: #!/bin/bash hostname date sleep 20 printenv – editing run_job. jdl file: Executable="run_job. sh"; #Arguments="blast_pesole_09_2011 5"; ## Please put your VO as Argument Std. Output="report_out. txt"; Std. Error="report_err. txt"; Input. Sandbox={"run_job. sh"}; Output. Sandbox={"report_out. txt", "report_err. txt"}; requirements = other. Glue. CEPolicy. Max. Wall. Clock. Time > 100 && Reg. Exp(". *cremino. *cloudtf", other. Glue. CEUnique. ID); Rank= ( other. Glue. CEState. Waiting. Jobs == 0 ? other. Glue. CEState. Free. CPUs : -other. Glue. CEState. Waiting. Jobs) ; – running commands: >voms proxy init voms dteam >glite wms job status a e https: //egee rb 02. cnaf. infn. it: 7443/glite_wms_wmproxy_server job. jdlglite ce job submit a r cremino. cnaf. infn. it: 8443/cream pbs cloudtf run_job. jdl
Scenario 1 • The testbed will support the OCCI interface integrated in a Webbased Portal within the middle of December – to retrieve information about VM by using VM’s UUID – to cancel VM by using VM’s UUID • Prerequisite – Users MUST belong to a given VO (i. g. , DTEAM) – Users MUST have a x. 509 digital certificate previously installed in the Web portal • How to interact with the Web-based Portal – A user, after selecting the VO he belongs to, is authenticated via VOMS service – Then user can select a compute resource – Then user can specify VM characteristics , like CPU, RAM, and OS – Then user can submit his/her request to instantiate VM, which can be assessed via passowrdless ssh
Scenario 2 • The provided testbed supports shared NFS filesystem with an available storage space of 2 TB.
Scenario 3 • The provided testbed publishes the Glue. Host. Application. Software Run Time Environment attribute for publishing VM information by using egee bdii. cnaf. infn. it as BDII • For example, a user that belongs to the VO DTEAM can get information about Tags configured on the queue cremino. cnaf. infn. it: 8443/cream pbs cloudtf as follows: $ lcg info vo dteam list ce query 'CE=cremino. cnaf. infn. it: 8443/cream pbs cloudtf' attrs Tag CE: cremino. cnaf. infn. it: 8443/cream pbs cloudtf Tag CNAF GLITE 3_0_0 GLITE 3_1_0 GLITE 3_2_0 SF 00 Mean. Per. CPU_951 SI 00 Mean. Per. CPU_1039 VO dteam Scientific. Linux. SLrelease 5. 7 • This could be used in future to let the user know which OS images is available on that queue
Scenario 4 • The provided testbed supports accounting at batch system level (i. e. , pbs) and integration with the DGAS Accounting System used by the Italian Grid infrastructure – Information is available via HLRMon tool http: //dgas. cnaf. infn. it/hlrmon/report/charts. php – A given user can ONLY access to his/her data – Aggregated data is sent to CESGA http: //www 3. egee. cesga. es/accounting/egee_view. p hp
Scenario 5 • WNo. De. S has an internal monitoring system for hypervisors in relation with the supported batch system. – The provided testbed uses the CREAM probe to integrate WNo. De. S monitoring in Nagios as shown at the Web page https: //mon it. cnaf. infn. it/nagios/cgi bin/status. cgi? navbarsearch=1&host=cremino. cnaf. infn. it – It is accessible by users that belong to the VO dteam • In addition, it uses what described in scenario 3
Scenario 5
Скачать презентацию INFN IGI contributions Federated Clouds Task Force F 2
ce3adfb5899c1962d13c31574769d5c1.ppt