CSS 434 Grid Computing Textbook No Corresponding Chapters Professor: Munehiro Fukuda A portion of these slides were compiled from The Grid: Blueprint for a New Computer Infrastructure. CSS 434 Grid Computing 1
Network Infrastructure n n Users login their organizational systems first locally or remotely. If they are affiliated with other organizations, n High-speed Information high way n They can login from the system of their main use to some other systems. (They are given an opportunity to use those resources in parallel). Problems: n n CSS 434 Grid Computing They must orchestrate job execution among the resources they use. Should those resources be limited to such a handful number of researchers? 2
Purposes of Computational Grid n n Use computing resource connected to high-speed information highway as if we use electric power grid n Only 30% utilization in academic/commercial environments. n Many applications have only episodic requirements. So, why don’t we share computation resource? n Computational results and data should be also made available to all users. Users: n Computational scientists and engineers n Experimental scientists n Association and corporations n Training and education n Consumers (E-commerce) CSS 434 Grid Computing 3
Grid Applications Category Examples Characteristics Distributed supercomputing DIS and Stellar dynamics Very large problems needing lots of computing resource at a time High throughput Chip design and parameter studies Harnessing many idle resources to increase aggregate throughput On demand Medical instrumentation Allocating special resource dynamically Data intensive Sky survey Using distributed data and needing high-volume data flows Collaborative design Education Support communication or collaborative work CSS 434 Grid Computing 4
Grid Services Architecture from www. globus. org slide Applications Application Toolkit Layer Grid Services Layer Grid Fabric Layer High-energy Collaborative On-line physics data engineering instrumentation Regional Parameter analysis climate studies Distributed computing Information Security Dataintensive Collab. design Resource mgmt Data access Transport Instrumentation . . . Remote viz . . . Fault detection Multicast Control interfaces CSS 434 Grid Computing Remote control Qo. S mechanisms 5
Programming Model Uniform Access n Paradigm n n n Bag of task or master workers (Condor-MW) Client server (Net. Solve) Object oriented (Legion) Synchronous applications (Not suited for massively parallel computation. ) Language Support n n n MPI-G – message passing (Globus) Open MP – shared memory Math Library – remote procedure (Net. Solve) CSS 434 Grid Computing 6
Resource Management Discovery, Allocation, and Scheduling n Centralized resource manager Systems Globus n Class. Ad and DAGMan Legion n RSL: resource spec. language Condor n Resource descriptions IDL: interface def. language Front-end process Resource manager Job launcher Broker and MDS GRAM Schedd Agent Matchmaker and startd Sandbox (Starter) Scheduler Collection Enactor +: easy to manage –: a bottleneck Decentralized resource manager n n A collection of centralized manager (Condor’s gate flocking) A combination of meta and local schedulers. CSS 434 Grid Computing 7
Fault Tolerance n Check-pointing n n n At the master (Condor) At each node but collected at the master (Catalina) Use a whiteboard (Optimal Grid) Re-execution of fault worker jobs from the beginning (Bayanihan, Optimal Grid) Error code (Net. Solve) n User is responsible to handle errors. CSS 434 Grid Computing 8
Security n Resources covered with security layers n n n Legion (Message/May. I layers) Entropia (Intercepting all system calls) A use of commodity tools n n n SSL Public key Security Certificate Java sandbox Kerberos CSS 434 Grid Computing 9
Net. Solve http: //icl. cs. utk. edu/netsolve/ Network of servers n n RPC-based approach Clients n n Include a set of APIs called as (asynchronous) RPCs Agents n n Client Match client’s requests for services with servers Agent Client Servers n Encapsulates remotely accessed numerical libraries CSS 434 Grid Computing choice request Scalar server reply MPP servers 10
Legion http: //legion. virginia. edu/ n Legion classes n n Act as managers and make policy Core objects Prog request Enactor Scheduler Provide mechanisms that classes use to implement policies: hosts (processors), vaults(memory), context, binding agents, etc. Converted Legion object ID By context objects reserve search Converted Logion object address By binding agents Resource database Class n Per-Program Scheduling Host collection n Participating sites can assure their tty Host local policies. tty Resources n User can choose a scheduling Class tty policy. n CSS 434 Grid Computing 11
Condor http: //www. cs. wisc. edu/condor/ A: User’s local agent R: Each computer resource M: Central manager CSS 434 Grid Computing I/O forwarded to a user’s home 12
Agent. Teamwork at UWB Architecture CSS 434 Grid Computing 13
Paper Review by Students n n n Globus Legion Condor Netsolve Discussions n n What programming or execution model is each system based on? What resource allocation and scheduling algorithm does each system use? Are they fault-tolerant? Did they any special security features for their own? CSS 434 Grid Computing 14
Скачать презентацию CSS 434 Grid Computing Textbook No Corresponding Chapters
c3878044d0afa7be61a8f0f5b64d3bdb.ppt