Скачать презентацию Coordinated Management and Peering of Content Delivery Networks Скачать презентацию Coordinated Management and Peering of Content Delivery Networks

37e240befc051261dbb6f15aec637d92.ppt

  • Количество слайдов: 35

Coordinated Management and Peering of Content Delivery Networks Al-Mukaddim Khan Pathan Grid Computing and Coordinated Management and Peering of Content Delivery Networks Al-Mukaddim Khan Pathan Grid Computing and Distributed Systems (GRIDS) Laboratory Dept. of Computer Science and Software Engineering The University of Melbourne, Australia www. csse. unimelb. edu. au/~apathan Supervisor: A/Prof. Rajkumar Buyya Masters to Ph. D Conversion Seminar

Outline n Introduction n Problem statement n Possible solution n Research Goals n Work Outline n Introduction n Problem statement n Possible solution n Research Goals n Work completed n Research Plan 2 Coordinated Management and Peering of CDNs

Internet & Web: Challenges n With the rapid growth of Internet and Web n Internet & Web: Challenges n With the rapid growth of Internet and Web n n n Services are competing each other for finite network and computing resources High availability and responsiveness are keys to business Web sites Large number of users are trying to simultaneously access the same Web site, causing “Flash Crowd” 3 Coordinated Management and Peering of CDNs

Problems with the Centralized Web Approach n Slow n n Unreliable n n delivery Problems with the Centralized Web Approach n Slow n n Unreliable n n delivery may be prevented by congestion or backbone peering problems Not scalable n n content must traverse multiple backbones and long distances usage limited by bandwidth available at master site Inferior streaming quality n packet loss, congestion, and narrow pipes degrade stream quality Source: Bruce Maggs, CCGrid 2001 Keynote 4 Coordinated Management and Peering of CDNs

Resource Distribution Approach n Content Delivery Networks (CDNs) is the solution to such service Resource Distribution Approach n Content Delivery Networks (CDNs) is the solution to such service degradation n Moving content to the “edge” of the Internet, closer to the end-users 5 Coordinated Management and Peering of CDNs

CDN: Abstract View 6 Coordinated Management and Peering of CDNs CDN: Abstract View 6 Coordinated Management and Peering of CDNs

CDN Commercials and so on… 7 Coordinated Management and Peering of CDNs CDN Commercials and so on… 7 Coordinated Management and Peering of CDNs

Outline n Introduction n Problem statement n Possible solution n Research Goals n Work Outline n Introduction n Problem statement n Possible solution n Research Goals n Work completed n Research Plan 8 Coordinated Management and Peering of CDNs

Limitations of Current CDNs n Existing CDNs are proprietary in nature n n CDN Limitations of Current CDNs n Existing CDNs are proprietary in nature n n CDN expenses have led to consolidation in the market Commercial CDNs sign Service Level Agreements (SLAs) with their customers n n Each has expensive closed delivery network No simultaneous usage of the resources of multiple CDNs Objective is to provide competitive services satisfying Qo. S requirements “Flash Crowd” or “Slash. Dot Effect” cause adverse business impact n n Very often CDNs can not provide Qo. S to end-users requests SLA violation to end-up costing the provider 9 Coordinated Management and Peering of CDNs

Problem Statement n Enable coordinated and cooperative content delivery between existing CDNs n Maximize Problem Statement n Enable coordinated and cooperative content delivery between existing CDNs n Maximize performance and timeliness of Web requests of end-users n Maximize the utilization of resources n Reduce cost and labor 10 Coordinated Management and Peering of CDNs

Related Work n n IETF Content Distribution Internetworking (CDI) model Federated, multi-provider infrastructure for Related Work n n IETF Content Distribution Internetworking (CDI) model Federated, multi-provider infrastructure for content delivery workloads n n CDN brokering n n Performance code to run on client side Coop. Net n n Use of Intelligent Domain Name Server (IDNS) Content Internetworking Router (Ci. Router) n n A peering algorithm Client side mechanism Co. Dee. N, Coral, Globule, Dot. Slash n Collaborative content delivery 11 Coordinated Management and Peering of CDNs

Outline n Introduction n Problem statement n Possible solution n Research Goals n Work Outline n Introduction n Problem statement n Possible solution n Research Goals n Work completed n Research Plan 12 Coordinated Management and Peering of CDNs

Possible Solution n Ad-hoc or planned peering arrangements between CDNs by leveraging existing infrastructures Possible Solution n Ad-hoc or planned peering arrangements between CDNs by leveraging existing infrastructures n Allows a CDN to “scale-out” to meet both flash crowds and anticipated increases in demand n Avoids expense of running a global CDN n Achieves economics of scale, in terms of cost effectiveness and performance for both providers and end-users n Avoids SLA violation n Assists to provide competitive service to catch-up the market 13 Coordinated Management and Peering of CDNs

CDN Peering Scenario 14 Coordinated Management and Peering of CDNs CDN Peering Scenario 14 Coordinated Management and Peering of CDNs

Outline n Introduction n Problem statement n Possible solution n Research Goals n Work Outline n Introduction n Problem statement n Possible solution n Research Goals n Work completed n Research Plan 15 Coordinated Management and Peering of CDNs

Key Issues n When to peer? n n How to peer? n n The Key Issues n When to peer? n n How to peer? n n The strategy taken to form peering between multiple CDNs Whom to peer with? n n The circumstances under which peering should be triggered The decision making mechanism for choosing CDNs to peer with How to manage and enforce policies? n Deploying necessary policies and administering them in an effective way 16 Coordinated Management and Peering of CDNs

Thesis Aims n Create an architectural framework for coordinated and cooperative CDNs n n Thesis Aims n Create an architectural framework for coordinated and cooperative CDNs n n Develop an effective content replication strategy n n Enforce a hierarchical approach to maximize utility of resources Develop an effective request redirection strategy n n Use economic models to develop a cooperative pull-based replication technique Develop an effective load measurement and dissemination technique n n Apply Virtual Organization concepts forming peering CDNs Enforce a multiple level request assignment/redirection technique to make it transparent and efficient Prototype system for peering CDNs n Leverage existing technologies and deploy on real-world testbed 17 Coordinated Management and Peering of CDNs

Outline n Introduction n Problem statement n Possible solution n Research Goals n Work Outline n Introduction n Problem statement n Possible solution n Research Goals n Work completed n Research Plan 18 Coordinated Management and Peering of CDNs

Work Completed n An in-depth analysis and survey of the content delivery networks in Work Completed n An in-depth analysis and survey of the content delivery networks in terms of n Organizational structure, content distribution mechanisms, request redirection techniques, and performance measurement methodologies Publication: A. M. K. Pathan and R. Buyya, “A Taxonomy and Survey of Content Delivery Networks, ” Technical Report, GRIDS-TR-2007 -4, Grid Computing and Distributed Systems Laboratory, The University of Melbourne, Australia, 12 February, 2007. (Submitted to ACM Computing Surveys) n The necessity of peering among CDNs according to a Virtual Organization (VO)-model n Through identifying significance and evaluating case studies Publication: R. Buyya, A. M. K. Pathan, J. Broberg, and Z. Tari, “A Case for Peering of Content Delivery Networks, ” IEEE Distributed Systems Online, 7(10), Los Alamitos, CA, October 2006. 19 Coordinated Management and Peering of CDNs

Work Completed (Cont’d) n A comprehensive architectural framework for peering CDNs n n With Work Completed (Cont’d) n A comprehensive architectural framework for peering CDNs n n With the roles, responsibilities and expected interaction patterns of architectural components Illustrating performance gain through peering Publication: A. M. K. Pathan, J. Broberg, K. Bubendorfer, K. H. Kim, and R. Buyya, “An Architecture for Virtual Organization (VO)-Based Effective Peering of Content Delivery Networks, ” UPGRADE-CN’ 07, In Proceedings of 16 th IEEE International Symposium on High Performance Distributed Computing (HPDC 2007), Monterey, California, USA, 25 -29 June, 2007. n An economy-based replication strategy n For on-demand placement of outsourced content Publication: A. M. K. Pathan and R. Buyya, “Economy-based Content Replication for Peering Content Delivery Networks, ” TCSC Doctoral Symposium, In Proc. of 7 th IEEE International Symposium on Cluster Computing and the Grid (CCGrid’ 07), Rio De Janeiro, Brazil, 14 -17 May, 2007. 20 Coordinated Management and Peering of CDNs

VO-Based Peering of CDNs n Initiator of VO-formation n n Participants n n Content VO-Based Peering of CDNs n Initiator of VO-formation n n Participants n n Content providers and endusers VO types n 21 Primary and any peering CDN(s) Implicit members n n Peering CDNs Explicit members n n Primary CDN n Short-term on-demand Long-term Coordinated Management and Peering of CDNs

System Architecture 22 Coordinated Management and Peering of CDNs System Architecture 22 Coordinated Management and Peering of CDNs

VO Formation (a) Formation of a VO (b) A formed VO 23 Coordinated Management VO Formation (a) Formation of a VO (b) A formed VO 23 Coordinated Management and Peering of CDNs

VO-Lifecycle: Flowchart Start Expansion of Shell VO Hotspot Generation Mi Generates Service Requirements Shell VO-Lifecycle: Flowchart Start Expansion of Shell VO Hotspot Generation Mi Generates Service Requirements Shell VO Creation Mi Passes Service Requirements to Local PA N Policies for Peering Exists? Preexisting Policies Returned Y Sufficient Resources Acquired? Y VO Formation Termination Condition(s) hold? N Short-Term Resource Negotiation Procedure Called Vo terminated 24 Coordinated Management and Peering of CDNs

VO Lifecycle 4 25 Coordinated Management and Peering of CDNs VO Lifecycle 4 25 Coordinated Management and Peering of CDNs

Short-Term Resource Negotiation: Flowchart Start Auction Initiation Mi Calculates Payoff Value Mi Issues a Short-Term Resource Negotiation: Flowchart Start Auction Initiation Mi Calculates Payoff Value Mi Issues a Call for Bid to Local PA With Auction Policy Local PA Passes Call for Bid to Other PAs Bidding Auction Termination Mi’s of Peers Registers With Requesting Mi Auctioneer(s) Collects Bids from Bidders Mi’s Subset of are Chosen as Distributed Auctioneers Winners are Selected Bidders Calculate Bidding Function Winner(s) Set is Returned Winners are Paid the Second Lowest Price Each PA Passes Request to the PA 26 Coordinated Management and Peering of CDNs

Short-Term Resource Negotiation 27 Coordinated Management and Peering of CDNs Short-Term Resource Negotiation 27 Coordinated Management and Peering of CDNs

Negotiated SLA Components n Description of service requirements n n Administration for VO activities Negotiated SLA Components n Description of service requirements n n Administration for VO activities n n Illustrates the steps to be undertaken in face of any problem in providing necessary services Consequences of SLA violation n n Specifies the role of the mediator as an authoritative entity Renegotiation for problem resolution n n Storage requirements, required rate of transfer (delay threshold), preference, expected duration of receiving service Range from imposing penalty through reimbursement of part of the revenues lost, to termination of peering relationship, and to disbanding and/or rearranging the VO SLA bypassing conditions n For situations like the damage of physical resources due to natural disaster, theft etc. 28 Coordinated Management and Peering of CDNs

Policy Management to Support SLAs (a) Basic policy framework (b) Policy mapping Policy framework Policy Management to Support SLAs (a) Basic policy framework (b) Policy mapping Policy framework Component Peering CDNs Component Specified policies Description System Peering CDNs All policies in the system The distributed computing and network infrastructure for peering CDNs Admin domain Formed VO Negotiated VO policies An administrative entity for resource management and access control Policy management tool Administrator dependent – An administrator dependent tool to generate policies Policy repository Web server, VO and mediator policies Storage of policies in the system Policy Enforcement Points (PEPs) Web Services host, Policy Agent, SLA-based allocator Web server policies A logical entity which ensures proper enforcement of policies Policy Decision Points (PDPs) Mediator policies, VO policies 29 An authoritative entity for retrieving policies from the Coordinated Management and Peering repository of CDNs

Performance Gain Through Peering n n Workloads are heavy-tailed in nature characterized by the Performance Gain Through Peering n n Workloads are heavy-tailed in nature characterized by the function, Service capacity follows Bounded Pareto distribution with p. d. f, , (a) Effectiveness of peering (b) Impact of request-redirection 30 Coordinated Management and Peering of CDNs

Three CDNs Peering Scenario n Expected waiting time, Primary CDN Load = 0. 5 Three CDNs Peering Scenario n Expected waiting time, Primary CDN Load = 0. 5 Peer 1 Load = 0. 5 Peer 2 Load = 0. 3 n n n 31 Task variation = 1. 5 Smallest task = 1010. 15 Largest task = 10^7 Service rate = 1/3000 Single request stream Coordinated Management and Peering of CDNs

Outline n Introduction n Problem statement n Possible solution n Research Goals n Work Outline n Introduction n Problem statement n Possible solution n Research Goals n Work completed n Research Plan 32 Coordinated Management and Peering of CDNs

Research Plan n n Evaluate the performance of developed economic model in a realistic Research Plan n n Evaluate the performance of developed economic model in a realistic simulation environment (Apr. 2007 – Aug. 2007) Develop an effective load measurement and dissemination technique (Sept. 2007 – Feb. 2008) n n Develop an effective request redirection strategy (Mar. 2008 – Jun. 2008) n n Enforce a multiple level request assignment/redirection technique to make it transparent and efficient Prototype system for peering CDNs (Jul. 2008 – Dec. 2008) n n Enforce a hierarchical approach to maximize utility of resources Leverage existing technologies and deploy on real-world testbed Complete thesis and submission (Jan. 2009 – Mar. 2009) 33 Coordinated Management and Peering of CDNs

Thanks for your attention! n More information at: http: //www. gridbus. org/cdn/ 35 Coordinated Thanks for your attention! n More information at: http: //www. gridbus. org/cdn/ 35 Coordinated Management and Peering of CDNs

Coordinated Management and Peering of Content Delivery Networks Al-Mukaddim Khan Pathan Grid Computing and Coordinated Management and Peering of Content Delivery Networks Al-Mukaddim Khan Pathan Grid Computing and Distributed Systems (GRIDS) Laboratory Dept. of Computer Science and Software Engineering The University of Melbourne, Australia www. csse. unimelb. edu. au/~apathan Supervisor: A/Prof. Rajkumar Buyya Masters to Ph. D Conversion Seminar