COMP 3019 Coursework Introduction to Grid SAM Steve

COMP 3019 Coursework: Introduction to Grid. SAM Steve Crouch s. crouch@software. ac. uk, stc@ecs School of Electronics and Computer Science

A Reminder… § For materials (inc. slides, software, materials, handout): – http: //www. ecs. soton. ac. uk/~stc/COMP 301 9/ § Coursework deadline: March 27, 2014 at 4 pm

Grid. SAM Job Submission and Monitoring

Grid. SAM Overview § What is Grid. SAM? – A Job Submission and Monitoring Web Service – Allows compute resources to be accessible securely over the internet – Uses Job Submission Description Language (JSDL) to describe requirements for jobs

Grid. SAM Overview § What is Grid. SAM to the resource owners? – A Web Service to expose heterogeneous execution resources uniformly · · · Single machine through Forking or SSH Condor Portable Batch Scheduler (PBS) Load Sharing Facility (LSF) Sun Grid Engine Globus – Acts as a client to these resources

Grid. SAM Overview § What is Grid. SAM to end-users? – A set of end-user tools and client-side APIs to interact with Grid. SAM Web Services · · · Submit and start jobs Monitor jobs Terminate jobs File transfer Client-side submission scripting Client-side Java API

Grid. SAM Server and Client § Server comprised of: – Web Services Container · Tomcat/Axis - Web Services provider/container · Optional WS-Security - handles authentication (contains X 509 certificate for security) · Can be run over HTTP or HTTPS – Grid. SAM Java web service § Client comprised of: – Client security ‘container’. · Contains Configuration for security (including X 509 certificate security credentials) · Used by client applications to interact with services on OMII server – Grid. SAM Java client

Web Service Definition § “A service is the logical manifestation of some physical or logical resources (databases, programs, devices, humans, etc) and/or some application logic that is exposed to the network” § Service interaction is facilitated by message exchanges

Web Services (WS) I § XML: Platform neutral mechanism to describe data § SOAP: Mechanism to describe message exchange – Simple Object Access Protocol · Not simple and nothing to do with Objects! – Service Oriented Access Protocol · Re-engineering of acronym to fit current use! § WSDL: Defines the service interface – Web Services Definition Language

Web Services (WS) II § Services have to reside in a supporting environment: – Called: hosting environment container or · e. g. Axis on Tomcat – Marshals requests into and response out of the service – Service can discover local configuration parameters – Provides a standard infrastructure for service developers § Processing incoming requests & outgoing responses – Called: Message handlers – Manipulates elements of the message header · Primarily the SOAP header – Handlers can be applied to message traffic into or out of the whole container or a specific service

Architecture … … Grid. SAM … WS-Security AXIS TOMCAT Client

Grid. SAM – Publications & Enabled Activities

Grid. SAM Architecture § A staged event-driven architecture – Submission pipeline is constructed as a network of stages connected by event queues – Each stage performs a specific action upon incoming events Example Pipeline: Condor

Scenario: Condor Pool FTP GSIFTP … WEBDAV HTTP

Example using File Staging § Objectives: submit simple job with data input and output requirements and monitor progress OMII Grid. SAM Client OMII Grid. SAM FTP Server submit JSDL monitor 2 input files 1 output file OMII Grid. SAM Server

JSDL Example § /gridsam/data/examples/remotecat-staging. jsdl: … bin/concat /Executable> < dir 2/subdir 1/file 2. txt

stdout. txt

stderr. txt dir 1/file 1. txt …

JSDL Example dir 2/subdir 1/file 2. txtbin/concat /File. Name> < > overwrite overwrite ftp: //ftp. do: 55521/concat. sh ftp: //ftp. do: 55521/input 2. txt stdout. txt overwrite dir 1/file 1. txt overwrite true ftp: //ftp. do: 55521/input 1. txt /URI ftp: //ftp. do: 55521/stdout. txt < >