Grid Services NERSC demo Shreyas Cholia Open

Grid Services @ NERSC demo Shreyas Cholia Open Software and Programming scholia@lbl. gov NERSC User Group Meeting September 19, 2007 NERSC User Group Meeting, September 17, 2007 0

Overview • • • Getting a certificate Initializing a certificate Registering certificate with NIM Submitting a fork job Submitting a serial job Submitting an MPI job Moving files with Grid. FTP Third party transfers with Grid. FTP Using myproxy to store your certificate NERSC User Group Meeting, September 17, 2007 1

Getting a Certificate • Instructions at: – http: //www. doegrids. org/pages/cert-request. html • Process: – – http: //pki 1. doegrids. org/ca/ Click on Retrieval->New User. Fill out form. You will receive an email once the certificate is ready. Click on the link. – Click on the import button in your browser. – Export/Backup certificate to local file – Convert file to usercert/key. pem using openssl • Renew Annually - repeat above process NERSC User Group Meeting, September 17, 2007 2

Initializing Your Cert • Load the globus module. One of: module load globus module load osg. /usr/common/osg/0. 6. 0/setup. sh • Your certificate is in: $HOME/. globus/usercert. pem $HOME/. globus/userkey. pem • Initialize the cert: grid-proxy-init (Creates temporary “proxy” cert valid for 12 hours) • View cert: grid-cert-info grid-proxy-info NERSC User Group Meeting, September 17, 2007 3

Adding Cert Info to NIM • • https: //nim. nersc. gov Click on Grid certificates tab Click on Add Certificate Enter information from: grid-cert-info -subject grid-cert-info -issuer NERSC User Group Meeting, September 17, 2007 4

For More Information • NERSC Website – http: //www. nersc. gov/nusers/services/Grid/ • Email – consult@nersc. gov NERSC User Group Meeting, September 17, 2007 5

Fork jobs • Run a job for immediate execution: # globus-job-run davinci/jobmanager /bin/hostname davinci • Submit a job # globus-job-submit davinci/jobmanager /bin/hostname https: //davinci. nersc. gov: 60005/6772/1190142432/ • Query submitted job # globus-job-status https: //davinci. nersc. gov: 60005/6772/1190142432/ DONE • Get output # globus-job-get-output https: //davinci. nersc. gov: 60005/6772/1190142432/ davinci NERSC User Group Meeting, September 17, 2007 6

Serial Jobs # globus-job-submit pdsfgrid/jobmanager-sge /bin/date https: //pdsfgrid. nersc. gov: 60005/6772/119014243 2/ # globus-job-get-output https: //pdsfgrid. nersc. gov: 60005/6772/119014243 2/ Tue Sep 18 14: 47: 06 PDT 2007 NERSC User Group Meeting, September 17, 2007 7

Parallel Jobs • Create RSL script & (host_xcount=4) (xcount=2) (jobtype=mpi) (directory=/usr/common/homes/s/shreyas/sandbox) (executable=/usr/common/homes/s/shreyas/sandbox/cpi) (stdout=x-gasscache: //$(GLOBUS_GRAM_JOB_CONTACT)stdout an. Extra. Tag) (stderr=x-gasscache: //$(GLOBUS_GRAM_JOB_CONTACT)stderr an. Extra. Tag) NERSC User Group Meeting, September 17, 2007 8

Parallel Jobs • Submit Job # globusrun -r jacquardgrid. nersc. gov/jobmanager-pbs -f cpi. rsl -b https: //jacquardgrid. nersc. gov: 60019/7027/1190152305/ • Query Q # globus-job-run jacquardgrid/jobmanager /usr/common/nsg/torque/bin/qstat • Get results # globus-job-get-output https: //jacquardgrid. nersc. gov: 60019/7027/1190152305/ NERSC User Group Meeting, September 17, 2007 9

Output PBS Leader node is jaccn 203 Job setup time: Tue Sep 18 14: 51: 47 PDT 2007 Setting up security Job startup at Tue Sep 18 14: 51: 52 PDT 2007 --------------------------------pi is approximately 3. 1415926535899423, Error is 0. 0000001492 wall clock time = 0. 041841 --------------------------------Jobs exit status code is Job STDIN/432828. jacin 03 -m. nersc. gov completed Tue Sep 18 14: 51: 54 PDT 2007 Submitted by shreyas/shreyas using mpccc Job Limits: ncpus=1, nodes=4: ppn=2, walltime=00: 30: 00 Job Resources used: cput=00: 00, mem=5340 kb, vmem=33288 kb, walltime=00: 02 Nodes used: jaccn 181 jaccn 183 jaccn 194 jaccn 203 Killing any leftover processes. . . NERSC User Group Meeting, September 17, 2007 10

File Transfer • Get File from HPSS # globus-url-copy -nodcau gsiftp: //garchive. nersc. gov//nersc/ccc/shreyas/Animals/gnu file: ///home/shreyas/gnu • Third party copy # globus-url-copy gsiftp: //pdsfgrid. nersc. gov//home/shreyas/Animals/gnu gsiftp: //davinci. nersc. gov//u/s/shreyas/Animals/gnu • Uberftp – Provides interactive access to gridftp server NERSC User Group Meeting, September 17, 2007 11

My. Proxy • If you use several client systems • Copying your. globus directory everywhere can be tedious • Myproxy provides central certificate repository • Storing cert # myproxy-init -s myproxy. nersc. gov (Valid for 1 week. Use -c for non-default lifetime) • Getting cert # myproxy-logon -s myproxy. nersc. gov (Same effect as grid-proxy-init - you have a temporary proxy valid for 12 hours) NERSC User Group Meeting, September 17, 2007 12