Скачать презентацию COP 4991 Component-Based Software Development Grid Computing and Скачать презентацию COP 4991 Component-Based Software Development Grid Computing and

75ce94bdc6cbcb63e51d25c8f1768f1c.ppt

  • Количество слайдов: 76

COP 4991 Component-Based Software Development Grid Computing and LA Grid Instructor: Masoud Sadjadi http: COP 4991 Component-Based Software Development Grid Computing and LA Grid Instructor: Masoud Sadjadi http: //www. cs. fiu. edu/~sadjadi/Teaching/ 8 th Lecture

Acknowledgement Andrew Tanenbaum Laukik Chitnis Sanjay Ranka Onyeka Ezenwoye Jorge Rodriguez COP-4991: Component-Based Software Acknowledgement Andrew Tanenbaum Laukik Chitnis Sanjay Ranka Onyeka Ezenwoye Jorge Rodriguez COP-4991: Component-Based Software Development 8 th Lecture 2

Agenda Motivation Grid Computing Overview Grid Middleware LA Grid COP-4991: Component-Based Software Development 8 Agenda Motivation Grid Computing Overview Grid Middleware LA Grid COP-4991: Component-Based Software Development 8 th Lecture 3

Speed up Using Parallel Processing (a) A program has a sequential part and a Speed up Using Parallel Processing (a) A program has a sequential part and a parallelizable part. (b) Effect of running part of the program in parallel. COP-4991: Component-Based Software Development 8 th Lecture 4

Speed up Using Parallel Processing Real programs achieve less than the perfect speedup indicated Speed up Using Parallel Processing Real programs achieve less than the perfect speedup indicated by the dotted line. COP-4991: Component-Based Software Development 8 th Lecture 5

Parallel Computer Architectures (a) On-chip parallelism. (b) A coprocessor. (c) A multiprocessor. (d) A Parallel Computer Architectures (a) On-chip parallelism. (b) A coprocessor. (c) A multiprocessor. (d) A multicomputer. (e) A grid. COP-4991: Component-Based Software Development 8 th Lecture 6

Instruction-Level Parallelism (a) A CPU pipeline. (b) A sequence of VLIW instructions. (c) An Instruction-Level Parallelism (a) A CPU pipeline. (b) A sequence of VLIW instructions. (c) An instruction stream with bundles marked. COP-4991: Component-Based Software Development 8 th Lecture 7

Multiprocessors (a) A multiprocessor with 16 CPUs sharing a common memory. (b) An image Multiprocessors (a) A multiprocessor with 16 CPUs sharing a common memory. (b) An image partitioned into 16 sections, each being analyzed by a different CPU. COP-4991: Component-Based Software Development 8 th Lecture 8

Multicomputers (a) A multicomputer with 16 CPUs, each with its own private memory. (b) Multicomputers (a) A multicomputer with 16 CPUs, each with its own private memory. (b) The bit-map image of Fig. 8 -17 split up among the 16 memories. COP-4991: Component-Based Software Development 8 th Lecture 9

Taxonomy of Parallel Computers (1) Flynn’s taxonomy of parallel computers. COP-4991: Component-Based Software Development Taxonomy of Parallel Computers (1) Flynn’s taxonomy of parallel computers. COP-4991: Component-Based Software Development 8 th Lecture 10

Taxonomy of Parallel Computers (2) A taxonomy of parallel computers. COP-4991: Component-Based Software Development Taxonomy of Parallel Computers (2) A taxonomy of parallel computers. COP-4991: Component-Based Software Development 8 th Lecture 11

UMA Multiprocessors Using Crossbar Switches (a) An 8 × 8 crossbar switch. (b) An UMA Multiprocessors Using Crossbar Switches (a) An 8 × 8 crossbar switch. (b) An open crosspoint. (c) A closed crosspoint. COP-4991: Component-Based Software Development 8 th Lecture 12

NUMA Multiprocessors A NUMA machine based on two levels of buses. The Cm* was NUMA Multiprocessors A NUMA machine based on two levels of buses. The Cm* was the first multiprocessor to use this design. COP-4991: Component-Based Software Development 8 th Lecture 13

Blue. Gene (1) The Blue. Gene/L custom processor chip. COP-4991: Component-Based Software Development 8 Blue. Gene (1) The Blue. Gene/L custom processor chip. COP-4991: Component-Based Software Development 8 th Lecture 14

Blue. Gene (2) The Blue. Gene/L. (a) Chip. (b) Card. (c) Board. (d) Cabinet. Blue. Gene (2) The Blue. Gene/L. (a) Chip. (b) Card. (c) Board. (d) Cabinet. (e) System. COP-4991: Component-Based Software Development 8 th Lecture 15

Google (1) Processing of a Google query. COP-4991: Component-Based Software Development 8 th Lecture Google (1) Processing of a Google query. COP-4991: Component-Based Software Development 8 th Lecture 16

Google (2) A typical Google cluster. COP-4991: Component-Based Software Development 8 th Lecture 17 Google (2) A typical Google cluster. COP-4991: Component-Based Software Development 8 th Lecture 17

Agenda Motivation Grid Computing Overview Grid Middleware LA Grid COP-4991: Component-Based Software Development 8 Agenda Motivation Grid Computing Overview Grid Middleware LA Grid COP-4991: Component-Based Software Development 8 th Lecture 18

Grid Computing Grid computing is an emerging computing model that – – – tries Grid Computing Grid computing is an emerging computing model that – – – tries to solve large-scale computation problems by taking advantage of many networked computers to model a virtual computer architecture that is able to distribute process execution across a parallel infrastructure. Source: www. wikipedia. org COP-4991: Component-Based Software Development 8 th Lecture 19

Ian Foster and Carl Kesselman “A computational grid is a hardware and software infrastructure Ian Foster and Carl Kesselman “A computational grid is a hardware and software infrastructure that provides dependable, consistent, pervasive, and inexpensive access to high-end computational capabilities. ” 1998 “The sharing that we are concerned with is not primarily file exchange but rather direct access to computers, software, data, and other resources, as is required by a range of collaborative problem solving and resource-brokering strategies emerging in industry, science, and engineering. This sharing is, necessarily, highly controlled, with resource providers and consumers defining clearly and carefully just what is shared, who is allowed to share, and the conditions under which sharing occurs. A set of individuals and/or institutions defined by such sharing rules form what we call a virtual organization. ” 2000 COP-4991: Component-Based Software Development 8 th Lecture 20

A Grid Checklist coordinates resources that are not subject to centralized control … … A Grid Checklist coordinates resources that are not subject to centralized control … … using standard, open, general-purpose protocols and interfaces … … to deliver nontrivial qualities of service. Virtual Organizations – Group of individuals or institutions defined by sharing rules to share the resources of “Grid” for a common goal. – Example: Application service providers, storage service providers, databases, crisis management team, consultants. COP-4991: Component-Based Software Development 8 th Lecture 21

The Grid isn’t a new concept Using multiple distributed resources to cooperatively work on The Grid isn’t a new concept Using multiple distributed resources to cooperatively work on a single application has been around for decades – – – Networked OS (70 s) Distributed OS (80 s and 90 s) Heterogeneous Computing http: //www. acis. ufl. edu/hcw 2006/ Parallel Distributed Computing Metacomputing http: //www. cnds. jhu. edu/research/metacomputing/ COP-4991: Component-Based Software Development 8 th Lecture 22

How is a grid different? Grids focus on site autonomy Grids involve heterogeneity Grids How is a grid different? Grids focus on site autonomy Grids involve heterogeneity Grids involve more resources than just computers and networks Grids focus on the user COP-4991: Component-Based Software Development 8 th Lecture 23

Grid Computing A dynamic multi-institutional network of computers that come together to share resources Grid Computing A dynamic multi-institutional network of computers that come together to share resources for the purpose of coordinated problem solving. resource application Achieved through: 1. Open general-purpose protocols institutional boundary 2. Standard interfaces COP-4991: Component-Based Software Development 8 th Lecture 24

So, what is a Grid again? ! Simply put, a Grid is a congregation So, what is a Grid again? ! Simply put, a Grid is a congregation of different sites collaborating to increase productivity For example, Grid 3 A total of 35 sites Over 3500 CPUs COP-4991: Component-Based Software Development 8 th Lecture 25

Broad Division of Grid Data Grids – Managing and manipulating large amounts of data. Broad Division of Grid Data Grids – Managing and manipulating large amounts of data. – Main objective is to share large amounts of data that is otherwise impossible with out the grid Compute Grids – For compute-intensive tasks. – Emphasis is on the federation of CPU cycles and distribution of compute intensive tasks There is no consensus on these categorization and it only aids in understanding the requirements COP-4991: Component-Based Software Development 8 th Lecture 26

Examples of Distributed Apps. High Energy Physics applications – Monte Carlo simulations – CMS Examples of Distributed Apps. High Energy Physics applications – Monte Carlo simulations – CMS experiment Finding interesting astronomical patterns – Sloan Digital Sky Survey Coastal ocean monitoring and predicting – SURA Coastal Ocean Observing and Prediction (SCOOP) Prime number generator – Cracking DES Cannot be done on a single machine You want to divide the application and run it on a distributed and decentralized environment COP-4991: Component-Based Software Development 8 th Lecture 27

Why do you want a grid? Different perspectives – User: I want to run Why do you want a grid? Different perspectives – User: I want to run my scientific application on the grid so that I can get results in 10 hours instead of 10 days – Organization: Our next big experiment will generate tera-bytes of data and we want to distribute, share and analyze the data – Organization: We want to tap into the existing grids and share resources COP-4991: Component-Based Software Development 8 th Lecture 28

A User Perspective I need – – – More CPU cycles More disk space A User Perspective I need – – – More CPU cycles More disk space More bandwidth Better software tools All of the above Run my app in 10 hrs that usually takes 10 days on my pentium Alternatives to grid – Simple CPU cycle stealer – Simple SRM (Storage Resource Manager) COP-4991: Component-Based Software Development 8 th Lecture 29

Sys admin perspective How do I distribute the load on the machines? How do Sys admin perspective How do I distribute the load on the machines? How do I reduce the overhead on the central server How do I manage local and remote users? What should be the policies? I got root ! COP-4991: Component-Based Software Development 8 th Lecture 30

Organizational Perspective Federation of scientists – distributing, sharing and analyzing data Tapping into existing Organizational Perspective Federation of scientists – distributing, sharing and analyzing data Tapping into existing grids Cost-effective: A grid can be built from commodity software and hardware without spending millions on the next super duper computer. Reliability: If a site fails, we can simply move our jobs to another site (this can be seen as a user perspective as well) Where do you want to run your job today? COP-4991: Component-Based Software Development 8 th Lecture 31

Distributed App. Requirements Requires – – A lot of resources Reservation of resources at Distributed App. Requirements Requires – – A lot of resources Reservation of resources at a particular time Monitoring of status of the submitted jobs to multiple sites Storage that is not easily available at a single place COP-4991: Component-Based Software Development 8 th Lecture 32

Grid Building Blocks Computational Clusters Storage Devices Networks Grid Resources and Layout: – – Grid Building Blocks Computational Clusters Storage Devices Networks Grid Resources and Layout: – – User Interfaces Computing Elements Storage Elements Monitoring Infrastructure… Some slides from Jorge Rodriguez’s presentation on “Building, Monitoring and Maintaining a Grid” COP-4991: Component-Based Software Development 8 th Lecture 33

Computer Clusters Cluster Management “frontend” Tape Backup robots I/O Servers typically RAID fileserver Disk Computer Clusters Cluster Management “frontend” Tape Backup robots I/O Servers typically RAID fileserver Disk Arrays A few Headnodes, gatekeepers and other service nodes The bulk are Worker Nodes Dell Cluster at UFlorida’s High Performance Center COP-4991: Component-Based Software Development 8 th Lecture 34

A Typical Cluster Installation WAN Ø Computing Cycles Ø Data Storage Ø Connectivity Pentium A Typical Cluster Installation WAN Ø Computing Cycles Ø Data Storage Ø Connectivity Pentium III Head Node/Frontend Server Ø Cluster Management • OS Deployment • Configuration • Many options ROCKS (kickstart) OSCAR (sys imager) Sysconfig • • COP-4991: Component-Based Software Development I/O Node + Storage Pentium III Worker Nodes Pentium III Network Switch Pentium III 8 th Lecture 35

Layout of Typical Grid Site A Grid Site Computing Fabric User Interface Compute Element Layout of Typical Grid Site A Grid Site Computing Fabric User Interface Compute Element Authz server => + Grid Middleware OSG VDT Storage Element Monitoring Element + Grid Level Services Data Management Services Monitoring Clients Services COP-4991: Component-Based Software Development Grid Operations => T h e G r i d 8 th Lecture 36

A Typical Grid Application Workflow A Small Montage Workflow ~1200 node workflow, 7 levels A Typical Grid Application Workflow A Small Montage Workflow ~1200 node workflow, 7 levels COP-4991: Component-Based Software Development Mosaic of M 42 created on the Teragrid using Pegasus 8 th Lecture 37

Simple Expectations from the Grid Simply put, we need the grid to do the Simple Expectations from the Grid Simply put, we need the grid to do the following operations for each transformation: – Find the input datasets – Apply the transformations (process the input) – Store the output datasets and publish its “presence” so that collaborating scientists can find it COP-4991: Component-Based Software Development 8 th Lecture 38

Qualities expected from the grid And of course, we would like these operations to Qualities expected from the grid And of course, we would like these operations to be performed: – Efficiently As quickly as possible – Seamlessly for easy collaboration – Fairly fair to all collaborators – Securely security against loss (fault tolerance), unauthorized access COP-4991: Component-Based Software Development 8 th Lecture 39

Agenda Motivation Grid Computing Overview Grid Middleware LA Grid COP-4991: Component-Based Software Development 8 Agenda Motivation Grid Computing Overview Grid Middleware LA Grid COP-4991: Component-Based Software Development 8 th Lecture 40

Grid Middleware In an effort to view the ‘Grid as a Workstation’, a set Grid Middleware In an effort to view the ‘Grid as a Workstation’, a set of grid software and services act as middleware between the user and the grid of machines. These services can be roughly categorized as follows: – – – Security services Information Services Data Management Job Management Virtual Data System COP-4991: Component-Based Software Development 8 th Lecture 41

GLOBUS TOOLKIT 4 – GT 4 Open source toolkit developed by The Globus Alliance GLOBUS TOOLKIT 4 – GT 4 Open source toolkit developed by The Globus Alliance that allows us to build Grid applications. Organized as a collection of loosely coupled components. Consists of services, programming libraries, and development tools. High-level services – – Resource Monitoring and Discovery Service Job Submission Infrastructure Security Infrastructure Data Management Services COP-4991: Component-Based Software Development 8 th Lecture 42

GT 4 Components COP-4991: Component-Based Software Development 8 th Lecture 43 GT 4 Components COP-4991: Component-Based Software Development 8 th Lecture 43

Services offered in a Grid Resource Management Services Data Management Services COP-4991: Component-Based Software Services offered in a Grid Resource Management Services Data Management Services COP-4991: Component-Based Software Development Information Services Security Services 8 th Lecture 44

Security Services Forms the underlying communication medium for all the services Secure Authentication and Security Services Forms the underlying communication medium for all the services Secure Authentication and Authorization Single Sign-on – User need not explicitly authenticate himself every time a service is requested Uniform Credentials Ex: GSI (Globus Security Infrastructure) COP-4991: Component-Based Software Development 8 th Lecture 45

Creates a proxy for single-sign on User Proxy GSI enabled GRAM Plain unix authentication Creates a proxy for single-sign on User Proxy GSI enabled GRAM Plain unix authentication GSI enabled GRAM Kereberos authentication A COP-4991: Component-Based Software Development B 8 th Lecture 46

GSI certificate A GSI certificate includes four primary pieces of information: – A subject GSI certificate A GSI certificate includes four primary pieces of information: – A subject name, which identifies the person or object that the certificate represents. – The public key belonging to the subject. – The identity of a Certificate Authority (CA) that has signed the certificate to certify that the public key and the identity both belong to the subject. – The digital signature of the named CA. A third party (a CA) is used to certify the link between the public key and the subject in the certificate. In order to trust the certificate and its contents, the CA's certificate must be trusted. The link between the CA and its certificate must be established via some non-cryptographic means, or else the system is not trustworthy. COP-4991: Component-Based Software Development 8 th Lecture 47

Grid Proxy Once your user certificate is in place, you need to create a Grid Proxy Once your user certificate is in place, you need to create a grid proxy which is used for accessing the Grid In Globus, you can do this using – grid-proxy-init A proxy is like a temporary ticket to use the Grid, default in the above case being 12 hours. Once this is done, you should be able to run “grid jobs” – globus-job-run site-name /bin/hostname COP-4991: Component-Based Software Development 8 th Lecture 48

Gridmap file A gridmap file at each site maps the grid id of a Gridmap file A gridmap file at each site maps the grid id of a user to a local id – The grid id of the user is his/her subject in the grid user certificate – The local id is site-specific; – multiple grid ids can be mapped to a single local id Usually a local id exists for each VO participating in that grid effort – The local ids are then used to implement site specific policies Priorities etc. COP-4991: Component-Based Software Development 8 th Lecture 49

Gridmap file entry The gridmap-file is maintained by the site administrator Each entry maps Gridmap file entry The gridmap-file is maintained by the site administrator Each entry maps a Grid DN (distinguished name of the user; subject name) to local user names # #Distinguished Name # Local username “/DC=org/DC=doegrids/OU=People/CN=Laukik Chitnis 712960” “/DC=org/DC=doegrids/OU=People/CN=Richard Cavanaugh 710220” “/DC=org/DC=doegrids/OU=People/CN=Jang. Uk In 712961” “/DC=org/DC=doegrids/OU=People/CN=Jorge Rodriguez 690211” COP-4991: Component-Based Software Development ivdgl grid 3 ivdgl osg 8 th Lecture 50

Grid Monitoring A grid is complex and quite big – To efficiently use it, Grid Monitoring A grid is complex and quite big – To efficiently use it, monitoring of the resources is needed Why? – – To check the availability of different grid sites Discovery of different grid services To check the status of “jobs” Maintain information on “health” of the different sites for informed decision making scheduling COP-4991: Component-Based Software Development 8 th Lecture 51

Type of Information The information can be broadly categorized as – Static, and – Type of Information The information can be broadly categorized as – Static, and – Dynamic Static information about a site includes – Its resource tallies Number of worker nodes, processors Storage capacities Architecture and Operating systems – Path of default staging area for grid users $app, $data COP-4991: Component-Based Software Development 8 th Lecture 52

Dynamic Information The dynamic content includes – The number of jobs running on each Dynamic Information The dynamic content includes – The number of jobs running on each site – The CPU utilization of different worker nodes – Overall site “availability” Such time-varying information is more critical for efficient scheduling of grid jobs Monitoring such information presents the question of tradeoff between accuracy (pertinence) and intrusion by the monitoring activities in the processor time share. COP-4991: Component-Based Software Development 8 th Lecture 53

Data Management Grids are usually used for analyzing and manipulating large amounts of data Data Management Grids are usually used for analyzing and manipulating large amounts of data Data management services provide a flexible mechanism to move and share data Ex: Grid. FTP, RLS (Replica Location Service) COP-4991: Component-Based Software Development 8 th Lecture 54

Data Management Motivating factors – Large number of data files Distributed across different sites Data Management Motivating factors – Large number of data files Distributed across different sites – Big files! ~TB, PB? ? Data is a first-class citizen in the Grid! COP-4991: Component-Based Software Development 8 th Lecture 55

Grid. FTP Globus-url-copy Grid. FTP-compliant client from the Globus team Copy files from one Grid. FTP Globus-url-copy Grid. FTP-compliant client from the Globus team Copy files from one URL to another URL – One URL is usually a gsiftp: // URL – Another URL is usually a file: / URL – To move a file from remote Grid. FTP-enabled server to local machine COP-4991: Component-Based Software Development 8 th Lecture 56

globus-url-copy To put file onto server reverse URLs globus-url-copy file: /home/lchitnis/file 1 gsiftp: //grinpc globus-url-copy To put file onto server reverse URLs globus-url-copy file: /home/lchitnis/file 1 gsiftp: //grinpc 03. phys. ufl. edu/data/file 1 By default 1 data channel used – monitor performance using –vb flag $ globus-url-copy -vb gsiftp: //grinpc 03. phys. ufl. edu/ppadala/grid/file 1 file: /tmp/file 1 9437184 bytes 658. 09 KB/sec avg 512. 95 KB/sec inst COP-4991: Component-Based Software Development 8 th Lecture 57

Performance tuning Multiple channels dramatically boosts transfer rate $ globus-url-copy -vb -p 4 gsiftp: Performance tuning Multiple channels dramatically boosts transfer rate $ globus-url-copy -vb -p 4 gsiftp: //ldas- cit. ligo. caltech. edu: 15000/usr 1/grid/largefile: /tmp/largefile 523960320 bytes 5814. 25 KB/sec avg 5568. 27 KB/sec inst Still faster by using large TCP windows $ globus-url-copy -vb -p 4 -tcp-bs 1048576 gsiftp: //ldascit. ligo. caltech. edu: 15000/usr 1/grid/largefile: /tmp/largefile 514392064 bytes 6609. 67 KB/sec avg 8639. 71 KB/sec inst Still faster by using large memory buffers $ globus-url-copy -vb -p 4 -bs 1048576 -tcp-bs 1048576 gsiftp: //ldas-cit. ligo. caltech. edu: 15000/usr 1/grid/largefile: /tmp/largefile 523304960 bytes 7300. 56 KB/sec avg 9311. 99 KB/sec inst Examples in this chapter adapted from David Gehrig’s presentation on “Grid Data Management” COP-4991: Component-Based Software Development 8 th Lecture 58

Job Management Services Provide a uniform, standard interface to remote resources including CPU, Storage Job Management Services Provide a uniform, standard interface to remote resources including CPU, Storage and Bandwidth Main component is the remote job manager Ex: GRAM (Globus Resource Allocation Manager) COP-4991: Component-Based Software Development 8 th Lecture 59

Job Management on a Grid GRAM User Condor VO Site A PBS Site C Job Management on a Grid GRAM User Condor VO Site A PBS Site C VO Site B Narration: note the different local schedulers COP-4991: Component-Based Software Development VO LSF fork VO Site D The Grid 8 th Lecture 60

Commands to run a job globus-job-run – globus-job-run runs in the foreground and defaults Commands to run a job globus-job-run – globus-job-run runs in the foreground and defaults to sending output to your terminal. – In its basic form, it is roughly equivalent to rsh, but has considerably more functionality for running complex jobs on the Grid. globus-job-submit – globus-job-submit is for submitting jobs to a remote batch job scheduler such as LSF or the Portable Batch System (PBS). – With globus-job-submit, you can submit a job, log out, and log back in later to collect the output. That is, globus-job-submit runs in the background and defaults to sending output to the machine running the command. – Retrieve output with globus-job-get-output, and then clean up with globus-job-clean. COP-4991: Component-Based Software Development 8 th Lecture 61

Commands to run jobs on grid globusrun – The globusrun command runs specifications written Commands to run jobs on grid globusrun – The globusrun command runs specifications written in the Globus Resource Specification Language (RSL). – globusrun can run jobs either in the foreground or background, and can send output to your terminal or to the machine running the command. – The trend in Globus software development is toward considering globusrun as middleware, which can be used by application specific shell scripts to manage job submission. – In fact, globus-job-run and globus-job-submit are simply shell scripts that use globusrun for job submission, but present a simpler interface to users. COP-4991: Component-Based Software Development 8 th Lecture 62

Agenda Motivation Grid Computing Overview Grid Middleware LA Grid COP-4991: Component-Based Software Development 8 Agenda Motivation Grid Computing Overview Grid Middleware LA Grid COP-4991: Component-Based Software Development 8 th Lecture 63

WHAT IS LA GRID? “First-ever comprehensive computing grid to link faculty, students, and researchers WHAT IS LA GRID? “First-ever comprehensive computing grid to link faculty, students, and researchers from institutions across the United States, Latin America and Spain to collaborate on complex industry applications for business and societal needs. ” http: //lagrid. fiu. edu/ COP-4991: Component-Based Software Development 8 th Lecture 64

LA-GRID @ CIS. FIU Globus-enabled hosts: – – – ip-comm. cs. fiu. edu la-blade-01. LA-GRID @ CIS. FIU Globus-enabled hosts: – – – ip-comm. cs. fiu. edu la-blade-01. cs. fiu. edu la-blade-02. cs. fiu. edu la-blade-03. cs. fiu. edu la-blade-04. cs. fiu. edu la-blade-05. cs. fiu. edu la-blade-06. cs. fiu. edu la-blade-08 -ppc. cs. fiu. edu la-blade-09 -ppc. cs. fiu. edu tsunami-linux. cs. fiu. edu starscream. cs. fiu. edu COP-4991: Component-Based Software Development 8 th Lecture 65

Resources as of July 2006 COP-4991: Component-Based Software Development 8 th Lecture 66 Resources as of July 2006 COP-4991: Component-Based Software Development 8 th Lecture 66

GETTING READY FOR LAGRID Globus Identity. Get a Globus Identity certificate issued by the GETTING READY FOR LAGRID Globus Identity. Get a Globus Identity certificate issued by the Certificate Authority. http: //www. cs. fiu. edu/~esj/globus. html Security Enrollment. Enroll as a Secure Globus User with Eric Johnson. Setup environment COP-4991: Component-Based Software Development 8 th Lecture 67

How to use Globus@SCIS The directory /etc/grid-security exists on all SCIS machines that are How to use [email protected] The directory /etc/grid-security exists on all SCIS machines that are grid enabled The following should be added to your. cshrc file to ensure that your environment is correctly set up for Globus. if ( -d /etc/grid-security ) then setenv GLOBUS_LOCATION /depot/globus-4 setenv LD_LIBRARY_PATH $GLOBUS_LOCATION/lib: $LD_LIBRARY_PATH ins 2 path $GLOBUS_LOCATION/bin ins 2 path $GLOBUS_LOCATION/sbin setenv MANPATH $GLOBUS_LOCATION/man: $MANPATH if ( `uname -m` == "ppc 64" ) then setenv JAVA_HOME /opt/ibm/java 2 -ppc-50 else setenv JAVA_HOME /depot/J 2 SE-1. 5 endif COP-4991: Component-Based Software Development 8 th Lecture 68

Globus Authentication Model Users have a Globus Identity certificate which is signed by the Globus Authentication Model Users have a Globus Identity certificate which is signed by the local certificate authority. Each system participating will have a mapping of globus ID to local user name. Each system must recognize the certificate authority that signed a users certificate. Here at SCIS we recognize certificate signed by us, and those signed by other LA-GRID sites. COP-4991: Component-Based Software Development 8 th Lecture 69

How to create and use an Identity (1) After setting up your environment as How to create and use an Identity (1) After setting up your environment as specified above, logout and then log in again and run the below command to generate a personal grid identity certificate. grid-cert-request This will create the following files in $HOME/. globus usercert_request. pem (request to sign certificate) userkey. pem (private key - encrypted) usercert. pem (public key - signed) COP-4991: Component-Based Software Development 8 th Lecture 70

How to create and use an Identity (2) After you have created the request How to create and use an Identity (2) After you have created the request then you need to mail it to the local certificate authority: cat $HOME/. globus/usercert_request. pem | mail globus. [email protected] fiu. edu Then the CA (Mr. Eric Johnson) will mail you back a signed certificate which you will want to put into $HOME/. globus/usercert. pem (it can take up to a day for the CA to process the request) COP-4991: Component-Based Software Development 8 th Lecture 71

Commands to log in / logout grid-proxy-init – This Commands to log in / logout grid-proxy-init – This "logs you into" the globus system. grid-proxy-info – Use this to see your status. grid-proxy-destroy – Use this to log out. How to test – When logged into a globus enabled hosts and logged into globus system: globus-url-copy -vb -dbg gsiftp: //OTHERGLOBUSHOST. cs. fiu. edu/dev/zero file: ///dev/null – REPLACE OTHERGLOBUSHOST with host name of another globus host. You should see no errors. COP-4991: Component-Based Software Development 8 th Lecture 72

How to deploy jobs Here is a simple test run on the local machine. How to deploy jobs Here is a simple test run on the local machine. – -s option says to connect stdout and stderr to this local process. globusrun-ws -submit -s -c /bin/hostname globusrun-ws -submit -S -f test 1. rsl globusrun-ws -submit -S -F https: //HOSTNAME: 8443/wsrf/services/Managed Job. Factory. Service -f test 1. rsl globusrun-ws -submit -S -f test 2. rsl globusrun-ws -submit -S -F https: //HOSTNAME: 8443/wsrf/services/Managed Job. Factory. Service -f test 2. rsl COP-4991: Component-Based Software Development 8 th Lecture 73

test 1. rsl This copies over and executable (/bin/echo) and runs it with two test 1. rsl This copies over and executable (/bin/echo) and runs it with two arguments. stdout and stderr are stored to files with those names. – This will only run on Intel arch machines. my_echo ${GLOBUS_USER_HOME} Hello World! ${GLOBUS_USER_HOME}/stdout ${GLOBUS_USER_HOME}/stderr gsiftp: //lion. cs. fiu. edu: 2811/bin/echo file: ///${GLOBUS_USER_HOME}/my_echo file: ///${GLOBUS_USER_HOME}/my_echo COP-4991: Component-Based Software Development 8 th Lecture 74

test 2. rsl This copies over a binary executable and runs it. – stdout test 2. rsl This copies over a binary executable and runs it. – stdout and stderr are stored to files with those names. – This will only run on Intel arch machines. /bin/hostname ${GLOBUS_USER_HOME} ${GLOBUS_USER_HOME}/stdout ${GLOBUS_USER_HOME}/stderr gsiftp: //la-blade-01. cs. fiu. edu: 2811/bin/echo file: ///${GLOBUS_USER_HOME}/my_echo file: ///${GLOBUS_USER_HOME}/my_echo COP-4991: Component-Based Software Development 8 th Lecture 75

References State and Events for Web Services: A Comparison of Five WS-Resource Framework and References State and Events for Web Services: A Comparison of Five WS-Resource Framework and WS-Notification Implementations. M. Humphrey, G. Wasson, K. Jackson, J. Boverhof, M. Rodriguez, Joe Bester, J. Gawor, S. Lang, I. Foster, S. Meder, S. Pickles, and M. Mc. Keown, 4 th IEEE International Symposium on High Performance Distributed Computing (HPDC-14), Research Triangle Park, NC, 24 -27 July 2005. C. Lee and D. Talia. "Grid programming models: Current tools, issues, and directions. ", In Grid Computing: Making The Global Infrastructure a Reality, F. Berman, A. Hey, and G. Fox, editors, John Wiley & Sons, 2003. “Enabling Peer-to-Peer Interactions for Scientific Applications on the Grid, ” V. Matossian* and M. Parashar, Proceedings of the 9 th International Euro-Par Conference (Euro-Par 2003), Lecture Notes in Computer Science, Editors: H. Kosch, L. Boszormenyi, H. Hellwagner, Springer-Verlag, Klagenfurt, Austria, Vol. 2790. Grids: Top Ten Questions. J. M. Schopf and B. Nitzberg, Scientific Programming, special issue on Grid Computing, 10(2): 103 - 111, August 2002. Berry, A. Djaoui, A. Grimshaw, B. Horn, F. Maciel, F. Siebenlist, R. Subramaniam, J. Treadwell, J. Von Reich. Informational Document, Global Grid Forum (GGF), January 29, 2005. COP-4991: Component-Based Software Development 8 th Lecture 76