An Efficient Threshold-Based Power Management Mechanism for Heterogeneous

An Efficient Threshold-Based Power Management Mechanism for Heterogeneous Soft Real-Time Clusters Leping Wang, Ying Lu University of Nebraska-Lincoln, USA 15 March 2018

Outline • • • Motivation Related Work Problem Statement Threshold-based approach Evaluation Conclusion 2

Motivation • Why power management (PM) for heterogeneous clusters – The power-related costs dominate the total cost of ownership of a cluster system – Most PM mechanisms are applicable to homogenous systems – Heterogeneous clusters are already widespread 3

Motivation • Opportunities for PM in heterogeneous clusters – Turn off or hibernate idle servers – Dynamically scale operating frequency/voltage (DVS) for underutilized servers – Distribute more requests to power-efficient servers 4

New Challenges • Decide not only how many but also which cluster servers should be used to process current requests, when necessary • Identifying the optimal load distribution for a heterogeneous cluster is a non-trivial task 5

Related Work • PM in homogeneous systems – [Bianchini et al. 2004], [Bohrer et al. 2002], [Chase et al. 2001], [Chen et al. 2005], [Elnozahy et al. 2002], [Rajamani et al. 2003] • PM in heterogeneous systems – [Heath et al. PPo. PP 2005], [Rusu et al. RTAS 2006] 6

Related Work • Current PM approaches for heterogeneous clusters – Search-based algorithms – Extensive performance measurements – Long optimization process • high customization costs upon new installations, server failures, cluster upgrades or other changes 7

Goal and Components • Goal – Near-optimal power consumption – Qo. S (average response time guarantee) – Efficient algorithm • Three components – Vary-on/off – DVS with feedback control – Optimal workload distribution 8

System Model 1. CPU-bounded server clusters (e. g. web server cluster) 2. One front-end server 3. N heterogeneous backend servers 9

Optimization Problem Cast the PM to an optimization problem • Objective: Minimize the total cluster power consumption J • Qo. S constraints: • Decisions on – Which servers should be used to process the current workload cluster , i. e. , decide xi : 0 or 1 – How should the workload cluster be distributed to active back-end servers, i. e. , decide λi such that – According to i, back-end server set its CPU frequency fi 10

Power and Capacity Models • Power Model : Total power consumption : The ith server’s on/off state : The ith server’s constant power consumption : The ith server’s operating frequency : The ith server’s dynamic power consumption • Capacity Model : The ith server’s throughput : The ith server’s performance coefficient 11

Optimization Problem • According to the M/M/1 queuing model and our server capacity model, we have • To make , we know 12

Optimization Problem • The optimization problem is formed as follows – Minimize: Subject to: 13

Optimization Problem • No analytical method to get the closed-form solution on i and xi • Time complexity of search-based algorithm • Basic idea of our efficient PM – Use a heuristic method to decouple decisions on xi and i, then solve them separately to obtain near-optimal solutions. 14

Threshold-Based Approach • An efficient PM heuristic – Efficient offline analysis: • Divides the possible workload range into N sub -ranges • For each sub-range, the PM decisions are derived offline – Online execution: Periodically, 1. Front-end server: workload cluster is predicted and depending on the range cluster falls into, the corresponding PM decisions will be followed 2. Back-end server: applies DVS mechanism to decide fi 15

Offline Analysis 1. Order the heterogeneous back-end servers, i. e. , generates a sequence, called ordered server list 2. Produce server activation thresholds 1, 2, … N such that if cluster ( k-1, k], it is optimal to turn on the first k servers of the ordered server list 3. Optimal workload distribution problem is solved for the N scenarios where cluster ( k-1, k], k=1, 2, …, N (time complexity: (N)) 16

Offline Analysis • When cluster ( k-1, k], the first k servers of the ordered server list are turned on and the optimization problem becomes – Minimize: Subject to: Solution: the optimal workload distribution i 17

Algorithm • Our method, denoted as TP-CP-OP – Server Ordered List Order all back-end servers according to their Typical Power (TP) efficiencies – Server Activation Thresholds Consider both server Capacity constraints and Power efficiencies (CP) – Optimal Workload Distribution (OP) 18

Dynamic Voltage Scaling i errori + - Feedforward M/M/1 Based Controller Feedback PI Controller fi fi + ith Back-end Server Ri 19

Evaluation • A small cluster with 4 back-end servers – Continuous operating frequency ranged in (0, fi_max] – Discrete operating frequency levels in [fi_min , fi_max] • A large cluster with 128 back-end servers in 8 different types 20

Evaluation • Synthetic workload and Real Workload • Desired average response time is set at 1 s • Evaluation metrics: average response time and power consumption • Each simulation lasts 3000 s • Power management in every 30 s 21

Evaluation • Baseline algorithms – Threshold-based approaches: AA−AA−CA, SP−CA−CA, EE-RT-HSC – Optimal power management solution OPT-SOLN obtained by a search-based algorithm 22

Evaluation • Average Response Time 23

Evaluation • Power Consumption 24

Conclusion • A efficient power management algorithm for heterogeneous server clusters – Mathematical models based • Minimum performance profiling – Workload threshold based • Low algorithm time complexity – Balance overhead and optimal solution • Fewer number of server on/off changes • Near-optimal power consumption 25

Technical Report • L. Wang and Y. Lu. Efficient power management of heterogeneous soft real-time clusters. Technical Report TR-UNL-CSE-2008 -0004, University of Nebraska -Lincoln, 2008 26

Questions or Comments? ? Thanks! Leping Wang, Ying Lu

Evaluation • Effect of Feedback Control