UC Berkeley Using Statistical Machine Learning in Cloud Computing ` David Patterson, UC Berkeley Reliable Adaptive Distributed Systems Lab 1 Image: John Curley http: //www. flickr. com/photos/jay_que/1834540/
Datacenter is new “server” • • “Program” == Web search, email, map/GIS, … “Computer” == 1000’s computers, storage, network Warehouse-sized facilities and workloads New datacenter ideas (2007 -2008): truck container (Sun), floating (Google), datacenter-in-a-tent (Microsoft) • How to enable innovation in new services without first building & capitalizing a large company? photos: Sun Microsystems & datacenterknowledge. com 2
Outline • Cloud Computing • RAD Lab • Case Study 1: Director & SCADS – Cloud storage and automatic management • Case Study 2: Automatic Log Analysis – Find anomalous behavior • Case Study 3: Predicting Map Reduce jobs – Help with scheduling • Summary 3
Cloud Computing is Hot • 9/15/09 Federal CIO Vivek Kundra embraces Cloud Computing • “We’ve been building data center after data center, acquiring application after application, and frankly what it’s done is drive up the cost of technology immensely across the board. What we need is to find a more innovative path in addressing these problems. ” 4
But. . . What is cloud computing, exactly? 5
“It’s nothing (new)” “. . . we’ve redefined Cloud Computing to include everything that we already do. . . I don’t understand what we would do differently. . . other than change the wording of some of our ads. ” Larry Ellison, CEO, Oracle (Wall Street Journal, Sept. 26, 2008) 6
Above the Clouds: A Berkeley View of Cloud Computing abovetheclouds. cs. berkeley. edu • 2/09 White paper by RAD Lab PI’s and students – Clarify terminology around Cloud Computing – Quantify comparison with conventional computing – Identify Cloud Computing challenges & opportunities • Why can we offer new perspective? – Strong engagement with industry – Users of cloud computing in our own research and teaching in last 18 months • Goal: stimulate discussion on what’s really new – without resorting to weather analogies ad nauseam 7
Utility Computing Arrives • Amazon Elastic Compute Cloud (EC 2) • “Compute unit” rental: $0. 10 -0. 80/hr. – 1 CU ≈ 1. 0 -1. 2 GHz 2007 AMD Opteron/Xeon core “Instances” Platform Cores Small - $0. 10 / hr 32 -bit 1 1. 7 GB 160 GB Large - $0. 40 / hr 64 -bit 4 7. 5 GB 850 GB – 2 spindles 64 -bit 8 • N XLarge - $0. 80 / hr Memory Disk 15. 0 GB 1690 GB – 3 spindles • No up-front cost, no contract, no minimum • Billing rounded to nearest hour; pay-as-you-go storage also available • A new paradigm (!) for deploying services? 8
What is it? What’s new? • Old idea: Software as a Service (Saa. S) – Basic idea predates MULTICS – Software hosted in the infrastructure vs. installed on local servers or desktops; dumb (but brawny) terminals • New: pay-as-you-go utility computing – Illusion of infinite resources on demand – Fine-grained billing: release == don’t pay – Earlier examples: Sun, Intel Computing Services—longer commitment, more $$$/hour, no storage – Public (utility) vs. private clouds 9
Why Now (not then)? • “The Web Space Race”: Build-out of extremely large datacenters (10, 000’s of commodity PCs) – Build-out driven by growth in demand (more users) => Infrastructure software: e. g. , Google File System => Operational expertise: failover, DDo. S, firewalls. . . – Discovered economy of scale: 5 -7 x cheaper than provisioning a medium-sized (100’s machines) facility • More pervasive broadband Internet • Commoditization of HW & SW – Fast Virtualization – Standardized software stacks 10
Classifying Clouds • • Instruction Set VM (Amazon EC 2) Managed runtime VM (Microsoft Azure) Framework VM (Google App. Engine) Tradeoff: flexibility/portability vs. “built in” functionality Lower-level, Less managed EC 2 Higher-level, More managed Azure App. Engine 11
Cloud Economics 101 • Cloud Computing User: Static provisioning for peak - wasteful, but necessary for SLA Machines Capacity $ Capacity Demand Time “Statically provisioned” data center “Virtual” data center in the cloud Unused resources 12
Cloud Economics 101 Machines Capacity Demand Energy • Cloud Computing Provider: Could save energy Capacity Demand Time “Statically provisioned” data center Real data center in the cloud Unused resources 13
Risk of Under Utilization • Underutilization results if “peak” predictions are too optimistic Capacity Resources Unused resources Demand Time Static data center 14
Resources Risks of Under Provisioning 3 Lost revenue Resources Demand 3 Demand 2 1 Time (days) Capacity Demand 2 1 Time (days) Lost users 3 15
New Scenarios Enabled by “Risk Transfer” to Cloud • “Cost associativity”: 1, 000 CPUs for 1 hour same price as 1 CPUs for 1, 000 hours (@$0. 10/hour) – Washington Post converted Hillary Clinton’s travel documents to post on WWW <1 day after released – RAD Lab graduate students demonstrate improved Hadoop (batch job) scheduler—on 1, 000 servers • Major enabler for Saa. S startups – Animoto traffic doubled every 12 hours for 3 days when released as Facebook plug-in – Scaled from 50 to >3500 servers –. . . then scaled back down 16
Cloud Computing & Statistical Machine Learning? • Before CC, only performance optimization on mostly small scale systems • CC detailed cost-performance model – Optimization more difficult with more metrics • CC Everyone can use 1000+ servers – Optimization more difficult at large scale • Economics rewards scale up AND down – Optimization more difficult if add/drop servers • SML as optimization difficulty increases 17
RAD Lab 5 -year Mission Enable 1 person to develop, deploy, operate next -generation Internet application • Key enabling technology: Statistical machine learning – management, scaling, anomaly detection, performance prediction, . . . • Highly interdisciplinary faculty & students – PI’s: Fox/Katz/Patterson (systems/networks), Jordan (machine learning), Stoica (networks & P 2 P), Joseph (systems/security), Franklin (databases) – 2 postdocs, ~30 Ph. D students, ~5 undergrads 18
RAD Lab Prototype: System Architecture Drivers Automatic Workload Evaluation (AWE) Director Offered load, resource utilization, etc. Training data performance & cost models Log Mining Chukwa & XTrace (monitoring) New apps, equipment, global policies (eg SLA) SCADS Chukwa trace coll. local OS functions Web 2. 0 apps web svc Ruby on APIs Rails environment Chukwa trace coll. local OS functions VM monitor 19
Successes 1. Automatically add/drop servers to fit demand, without violating Service Level Agreement (SLA) 2. Predict performance of complex software system when demand is scaled up 3. Distill millions of lines of log messages into an operator-friendly “decision tree” that pinpoints “unusual” incidents/conditions • Recurring theme: cutting-edge Statistical Machine Learning (SML) works where simpler methods have failed 20
Outline • Cloud Computing • RAD Lab • Case Study 1: Director & SCADS – Cloud storage and automatic management • Case Study 2: Automatic Log Analysis – Find anomalous behavior • Case Study 3: Predicting Map Reduce jobs – Help with scheduling • Summary 21
Automatic Management of a Datacenter • As datacenters grow, need to automatically manage the applications and resources – examples: • deploy applications • change configuration, add/remove virtual machines • recover from failures • Director: – mechanism for executing datacenter actions • Advisors: – intelligence behind datacenter management 22
Director Frameworkload model performance model Advisor monitoring data config Datacenter(s) VM VM 23
Director Framework • Director – issues low-level/physical actions to the DC/VMs • request a VM, start/stop a service – manage configuration of the datacenter • list of applications, VMs, … • Advisors – update performance, utilization metrics – use workload, performance models – issue logical actions to the Director • start an app, add 2 app servers 24
What About Storage? • Easy to imagine how to scale up and scale down computation • Database don’t scale down, usually run into limits when scaling up • What would it mean to have datacenter storage that could scale up and down as well so as to save money for storage in idle times? 25
DC Storage Motivation • Most popular websites follow the same pattern – Rapidly developed on SQLServer / Postgre. SQL / My. SQL / etc. – Become popular and realize scaling limitations – Build large, complicated ad-hoc systems to deal with scaling limitations as they arise • Websites that can’t scale fast enough lose customers
SCADS: Scalable, Consistency. Adjustable Data Storage • Scale Independence - as the user base grows: – No changes to application – Cost per user doesn’t increase – Request latency doesn’t change • Key Innovations 1. Performance safe query language 2. Declarative performance/consistency tradeoffs 3. Automatic scale up and down using machine learning
Scale Independence Arch • Developers provide performance safe queries along with consistency requirements • Use ML, workload information, and requirements to provision proactively via repartitioning keys and replicas
SCADS Performance Model (on m 1. small, all data in memory) 5% writes 1% writes SLA threshold 99 th percentile median
Low workload, low put rate
Outline • Cloud Computing • RAD Lab • Case Study 1: Director & SCADS – Cloud storage and automatic management • Case Study 2: Automatic Log Analysis – Find anomalous behavior • Case Study 3: Predicting Map Reduce jobs – Help with scheduling • Summary 31
Mining Console Logs for Large. Scale System Problem Detection • Why console logs? • Console logs are everywhere – In almost every software system – Hand-picked information by developers – Expressive, convenient to use • Low programmer overhead • Especially useful in large scale Internet services – Open source code + in-house development – Continuously changing system “Detecting Large-Scale System Problem Detection by Mining Console Logs, ” Wei Xu, Ling Huang, Armando Fox, David Patterson, Michael Jordan, 32 Symposium on Operating Systems Principles (SOSP), Oct 2009
Console logs use awkward languages HODIE NATUS EST RADICI FRATER* today unto the Root a brother is born. * "that crazy Multics error message in Latin. " http: //www. multicians. org/hodie-natus-est. html 33
Console logs are not operator friendly Console Logs Operators grep Perl scripts search • Problem – Don’t know what to look for! • Console logs are intended for a single developer • Assumption: log writer == log reader • Today many developers => massive textual logs • Our goal - Discover the most interesting log messages without any prior input 34
Console logs are hard for machines too Machine Parsing Learning Feature Creation Machine Learning Visualization • Problem • Highly unstructured, looks like free text • Not able to capture detailed program state with texts • Hard for operators to understand detection results • Our contribution • A general framework for processing console logs • Efficient parsing and features 35
Key observations • Parsing – Console logs were inherentedly structured • Determined by log printing statement – Constant strings = markers of message structure – Source code is usually available • Features • Common information reported in logs • Identifiers and execution paths • State variables • Correlations among messages • Many ways to group related log messages • i. e. not just by time 36
Case studies • Surveyed a number of software apps – Linux kernel, Open. SSH, Apache, My. SQL, Jetty – Hadoop, Cassandra, Nutch … – Parsing works on 20/22 • In this talk – Hadoop file system – Per block operation logging (usually ignored) • Experiment on EC 2 cloud – 203 nodes * 48 hours – ~ 300 TB HDFS data (550, 000 blocks) – ~24 million lines of console logs 37
Step 1: Using source code analysis to help log parsing • Basic ideas 091012 03 -00 -00 INFO Creating file mydata Log. info(“Creating file ” + filename); Creating file (. *) [filename] • Non-trivial in object oriented languages – to. String() method of objects – Dynamic binding (sub-classing) • Our Solution – Analyze source code to abstract syntax tree (AST) level – Pre-compute all possible message types • Free text -> semistructured text 38
Step 2: Feature Message count vector • Identifiers are common in logs – file names, object keys, user ids • Identifiers can be discovered automatically – Have many distinct values – Appear in multiple message types – Reported many times myfile: Creating Write • Grouping by identifiers – Similar to execution paths Backup Done • Numerical representation of these “paths” – Similar to Bag of words model in IR 39
Message count vector example datanode_r 16 | Receiving block blk_100 src: … dest: . . . namenode_r 10 | allocate. Block: blk_100 namenode_r 10 | allocate. Block: blk_200 datanode_r 16 | Receiving block blk_200 src: … dest: . . . datanode_r 14 | Receiving block blk_100 src: …dest: … datanode_r 16 | Received block blk_100 of size 49486737 from … datanode_r 14 | Received block blk_100 of size 49486737 from … datanode_r 16 | Error Receiving block blk_200 of size 49486737 from … blk_100 0 2 2120020000 blk_200 0 1 01200201000000 40
Step 3: Mining PCA detection 0221200201000000 55, 000 vectors, one per block • This example - detect abnormal vectors • Dimensions highly correlated – Rare correlations indicate abnormal executions – PCA separates normal pattern from abnormal, making anomalies easy to detect • From text -> structured feature – With details of program execution – Can apply many other machine learning algorithms 41
PCA detection results Seq Event Description Actual 1 2 3 4 5 6 7 8 9 10 11 Forgot to update namenode for deleted block Write block exception then client give up Failed at beginning, no block written Over-replicate-immediately-deleted Received block that does not belong to any file Redundant add. Stored. Block request received Trying to delete a block, but the block no longer exists on data node Empty packet for block Exception in receive. Block for block Pending. Replication. Monitor timed out Other anomalies Total (Out of 550, 000 blocks) Seq Description 1 2 False Positives Normal background migration Multiple replica ( for task / jobdesc files ) Total Detected 4297 3225 2950 2809 1240 953 724 476 89 45 108 4297 3225 2950 2788 1228 953 650 476 89 45 107 16916 16808 False Positives 1397 349 1746 Making results easy for operators to understand! 42
Explaining detection results with decision tree write. Block # received exception >=1 1 >=3 1 0 # Starting thread to transfer block # to # <=2 #: Got exception while serving # to #: # >=1 0 Unexpected error trying to delete block #. Block. Info Not found in volume. Map >=1 0 add. Stored. Block request received for # on # size # But it does not belong to any file >=1 0 # starting thread to transfer block # to # >=1 0 #Verification succeeded for # >=1 0 Receiving block # src: # dest: # >=3 0 <=2 0 1 1 0 0 1 43
Summary: Mining Console Logs for Large-Scale System Problem Detection Extract Detect abnormal log segments 200 nodes, >24 million lines of logs Visualize A single decision tree to visualize system behavior Contributions: How to parse logs, How to turn into features so ready for SML, How to make results accessible to operators, Which SML algorithm used matters less 44
Outline • Cloud Computing • RAD Lab • Case Study 1: Director & SCADS – Cloud storage and automatic management • Case Study 2: Automatic Log Analysis – Find anomalous behavior • Case Study 3: Predicting Map Reduce jobs – Help with scheduling • Summary 45
Using SML to Characterize, Predict, Optimize Resources Given a system (e. g. Web Service, multicore …), its workload, and performance metrics: • Answer “what-if” questions for – Changes in workload (mix/rate/distribution) – Changes in system configuration (hw/sw) – Changes in both • 3 Examples for Case Study 3: – Parallel Database Performance Prediction – Hadoop Job Performance Prediction – Multi-core Performance Optimization (if time permits)
Problem Statement Workload SYSTEM Behavior Model • Find relationships between workload and performance • How to generate the model? – Manually constructed queueing model – Automatically constructed • Regression, genetic algorithms, correlation analysis, … – This talk: Canonical Correlation Analysis
Finding Correlations
Kernel Canonical Correlation Analysis (KCCA) • 2003, Bach & Jordan at Berkeley • Find dimensions of maximal correlation between kernelized datasets – “Kernel functions” impose system-specific definition of similarity • Dimensionality reduction + clustering effect Advantages: – Relative distance between datapoints (defines neighborhoods) – Interpolation (preserves neighborhoods across both datasets)
3 Challenges in using SML 1. Mapping function from traces to feature vectors • • Workload -> multidimensional space System behavior -> multidimensional space 2. How to compare similarity of two feature vectors 3. How to leverage results of SML techniques to find actionable answer
Hadoop Job Performance Prediction Jobs Hadoop Performance 51
Motivating Scenarios • Which jobs should I schedule together to avoid resource contention in Hadoop? – Should I run my job now or wait till later? • What is the optimal number of nodes to run a job on? – How many nodes should my job run on to meet the deadline? • Given observed behavior of a job run at small scale, how will the job behave when scaled up? – Are the performance bottlenecks the same? – Which aspects of the system perform non-linearly? 52
Hadoop Facts • Map-reduce jobs have diverse performance profiles - Each job has a different behavioral signature - Input parameters affect scaling slopes of jobs • Map performance is linear wrt number of nodes • Reduce complexity is job specific • Shuffle step is primary source of job unpredictability 53
SML Goals • Predict runtime/resource requirements for jobs – Same jobs, different configs – Same configs, different jobs • Identify which jobs can be co-scheduled – minimize resource contention – maximize shared data • Create Map-Reduce benchmark – Representative workloads – Varying scale factors 54
Data and Testbed: Facebook • Multi-user environment • Data set spans 6 months • Know job names but not job source code • ~600 node cluster • Nodes 1 -300 – 16 GB memory – 5 map slots – 5 reduce slots • Nodes 301 -600 – 8 GB memory – 5 map slots – 0 reduce slots
Recurring Extract Transform Load (ETL) Jobs # of instances 66 unique jobs account for 99% of ETL jobs Job. ID
2008 -07 -18 -23 2008 -07 -22 -09 2008 -07 -25 -22 2008 -07 -29 -06 2008 -08 -02 -23 2008 -08 -10 -04 2008 -08 -16 -02 2008 -08 -23 -10 2008 -08 -27 -15 2008 -08 -30 -18 2008 -09 -05 -19 2008 -09 -09 -03 2008 -09 -12 -14 2008 -09 -15 -21 2008 -09 -19 -03 2008 -09 -22 -09 2008 -09 -25 -14 2008 -09 -28 -20 2008 -10 -02 -11 2008 -10 -06 -20 2009 -02 -10 -06 2009 -02 -13 -17 2009 -02 -17 -10 2009 -02 -21 -03 2009 -02 -24 -16 2009 -02 -28 -16 2009 -03 -04 -19 Total Time Most repeated job 14000 12000 10000 8000 6000 4000 2000 0
Same job, many configs 14000 12000 Total Time 10000 8000 6000 4000 2000 0 0 50 100 150 Number of Maps 200 250
X Y Job Descriptor Vector Hadoop • Job config parameters • Number of Maps • Number of Reduces • Data characteristics • Map Input Bytes • Locality Performance Vector • Map Time • Reduce Time • Total Time • Map Output Bytes
Predicting Job Performance
Total Time Actual time (ms) Perfect prediction Predicted time (ms)
Map Output Bytes Actual bytes Perfect predicti on Predicted bytes
Multi-Core Performance Optimization (if time permits) Application Optimization Configurations Multi-core Performance A. Ganapathi, K. Datta, A. Fox, D. Patterson, "A Case for Machine Learning to Optimize Multicore Performance", First USENIX Workshop on Hot Topics in Parallelism (Hot. Par '09), Berkeley, CA, March 30 -31, 2009.
Problem Structured Grids Dense Linear Algebra Motifs + Sparse Linear Algebra Diverse Sun Intel/AMD IBM Multicore Blue Gene Niagara 2 x 86 Architectures + Compilers alonexlc icc (No code tuning) gcc = Poor Performance!
Solution: Auto-tuning Identify motif-specific optimizations Generate code variants based on these optimizations Search parameter space for best configuration Benefits: Problem: • Portable across diverse Total architectures Optimization Parameters Configs • Easily scalable n! Thread Count 1 4 sio • Tunable for any relevant metric plo 36 Domain Decomposition 4 • Minimizes programmer tuning ex Software Prefetching 18 time ce a 2 sp 1 • Proven track record (e. g. ATLAS, Padding 32 h SPIRAL, FFTW, OSKI) arc Inner Loop 8 480 e S • Already showed speedups of 1. 5 x Total 16 4 x 107 to 5. 6 x
ML Goals Goal: find best value for configuration parameters to simultaneously optimize multiple performance metrics v Require little architecture/application knowledge v Provide actionable suggestions for improving performance v Simultaneously optimize for multiple performance metrics (e. g. energy efficiency and cycle time) v Use small set of samples to build model
Experimental Setup 2563 regular grid x+1 z+1 y-1 (x, y, z) z-1 x-1 Structured Grids (stencil codes) • Used in iterative PDE solvers for image processing, diffusion etc. • For a given point, a stencil is a fixed 3 D 7 -point stencil subset of nearest neighbors • A stencil code updates every point in a regular grid by “applying a stencil” • Stencil codes usually bandwidthbound 3 D 27 -point stencil Architectures • AMD Barcelona 2 sockets x 4 cores/socket, 2. 30 GHz, compiler: gcc • Intel Clovertown 2 sockets x 4 cores/socket, 2. 66 GHz, compiler: icc
Potential Optimizations Parallelization and Capacity Misses +X NX(unit stride) CY RX RY RZ Thread 0 CX Domain Decomposition A[i-1] A[i+1] Processing … Low Memory Bandwidth A[i+dist-1] A[i+dist+1] Retrieving from DRAM … NY TY Software Prefetching Thread Conflict 1 Misses ng … +Y TX CZ NZ +Z TY CZ TX Thread n i d ad p Array Padding for i=1 to x { for i=1 j=1 xto y { for to to y { for i=1 j=1 xk=1 to z { for to y { for j=1 k=1 to z { A[i, j, k]=… for k=1 to z { } }} A[i, j, k]=… }}} }} } Poor Functional Unit Usage and Scheduling Inner Loop Opts.
3 Challenges in using ML 1. Mapping function from traces to feature vectors • • Configuration parameters -> multidimensional space System behavior -> multidimensional space 2. How to compare similarity of two feature vectors 3. How to leverage results of ML techniques to find actionable answer
Configuration Parameters Parameter Number of Threads Block Size Domain Decomposition NThreads {20… 23} 4 {27…NX} {21…NY} CZ Thread Count Number of Configurations CY Type Parameter Range CX Optimization NZ Name 36 Chunk Size Software Prefetching Type Padding {register block, plane, pencil} 3 {0, 25… 29} 6 Padding Size {0… 31} 32 Prefetching Distance RX RY {20… 21} RZ Register Block Size {20… 21} {20… 23} 10 Statement Type {complete, individual} 2 Read From Type {array, variable} 2 Pointer Type {fixed, moving} 2 Neighbor Index Type {register block, plane, pencil} 3 FMA-like Instructions Inner Loop Optimizations {yes, no} 2 X Feature Vector Threads Block Size CX Block Size CY Block Size CZ Padding Size Prefetch type Prefetch distance Statement Type 4 32 128 256 32 Plane 64 individual
Performance Metrics Counter Description PAPI_TOT_CYC PAPI_L 1_DCM PAPI_L 2_DCM Cycles per thread per job L 1 data cache misses per thread L 2 data cache misses per thread PAPI_TLB_DCM PAPI_CA_SHR PAPI_CA_CLN PAPI_CA_ITV Power meter TLB misses per thread Accesses to shared cache lines Accesses to clean cache lines Cache interventions Watts consumed per second Feature Vector Y (total cycles x # of flops) / (clk rate x # of watts) Total Cycles L 1_DCM L 2_DCM TLB_DCM CA_SHR CA_CLN CA_ITV Energy Efficiency 1. 9 E 7 2. 4 E 5 1. 5 E 5 1. 2 E 4 1. 2 E 5 1. 4 E 4 1. 2 E 3 2. 3 E 4
Kernel functions Threads Block Size CX Block Size CY Block Size CZ Padding Size Prefetch type Prefetch distance Statement Type 4 32 128 256 32 Plane 64 Individual 2 64 128 256 32 Plane 64 Individual X 1 X 2 2 64 128 256 32 Pencil 64 complete X 3 Are X 1 and X 2 more similar than X 2 and X 3? Measure of similarity Numeric Columns: Gaussian Kernel K(xi, xj) = exp(-||xi-xj||2 / ) Non-numeric Columns: K(xi, xj) = {1 if xi=xj, 0 if xi≠xj } Similarity(X 1, X 2) = average(K(X 1 i, X 2 i))
Stencil Code X Y Configuration Feature Vector Performance Feature Vector Platform kernel function • compiler N • performance counters 1 …N 1 … optimizations 1 • parameters for memory blocking 1 meter readings • power : : KY : N • Ksoftware prefetching : N X KCCA 0 K Kernel functions. K X Y A = KXKX 0 A K K 0 B B • Numeric Columns: Gaussian. K Kernel 0 K • Non-numeric Columns: K(xi, xj) = {1 if xi=xj, 0 KY*B i≠xj } if x KX*A • Similarity(X 1, X 2) = average(K(X 1 i, X 2 i)) Y X Y Y
Finding Optimal Configurations KCCA Data Space Raw Data Space Configuration features Performance Metrics X Y Inverse Image ? ? ? Problem KX*A KCCA Nearest Neighbors KY*B
Results 75
Auto-tuning Time • Exhaustive Search: – 4 x 107 configs x 0. 08 s/trial x 5 trials > 180 days • Our Technique: – Training data: 1500 configs x 0. 08 s/trial x 5 trials = 10 minutes – Training time using KCCA: 90 minutes – Genetic algorithm: 243 configs x 0. 08 s/trial x 5 trials = 1. 6 min – Total Time < 2 hrs 76
Summary: SML Strengths 1. Train vs. write the logic (handle SW churn) 2. Learns how to handle and make policy during transitions between steady states (unlike queueing theory) 3. Coping with a complex cost function (unlike simple control theory) 4. Finding trends, needles in data haystack 5. Fast enough to run online (today) – Infeasible algorithms from 1960 s fast in 2008 – Computers cheap enough OK to “just” monitor 77
Summary: SML Accommodations • System data usually text vs. numbers – Need to turn text into features – Need to find distance function for close vs. far • SML is not perfect – System must live with <100% accuracy, false positives • Will be asked “Yes, I see that SML works, but why didn’t you try ‘simpler’ Algorithm X? ” – For all X – Simpler = I know X, I don’t know SML? 78
Conclusion • Cloud Computing will transform IT industry – Pay-as-you-go utility computing leveraging economies of scale of Cloud provider – Anyone can create/scale next e. Bay, Twitter… • Cloud Computing better management: cost-performance, scale up/down • SML: making sense from lots of data • RAD Lab applying SML to New Cloud Computing challenges: Director, SCADS, Map Reduce Scheduling, Log Analysys 79
Acknowledgments • Peter Bodik for Director/SCADS experiments • Wei Xu for Log Analysis • Archana Ganapathi for Map Reduce and Multicore Performance Prediction • 30 Ph. D students and 8 faculty of RAD Lab • RAD Lab founding sponsors Google, Microsoft, and Sun Microsystems and affiliates Amazon Web Services, Cloudera, Cisco, Facebook, HP, Net. App, SAP, VMware 80
Backup Slides 81
Evaluation Benchmarks s er et ra m Pa #3 t. “Stencil Computation Optimization and Auto-tuning on State-of-the-Art Multicore Architectures”, Supercomputing 2008. Op – ordered the optimizations – applied them consecutively – more information: Datta et al. , Opt. #2 Parameters • “No Optimization”: running naïve code • “Expert Optimized”: Opt. #1 Parameters • “Random Raw Data”: best performing point in raw data • “Genetic on Raw Data”: permute configs for top three best performing points in raw data
![Cross Validation Prediction Accuracy: 1 – [Σ(predictedi – actuali)2 / Σ(actuali – actualmean)2] Test](https://present5.com/presentation/404ddb843a9d51ab5a26d768064fb87d/image-83.jpg "Cross Validation Prediction Accuracy: 1 – [Σ(predictedi – actuali)2 / Σ(actuali – actualmean)2] Test")
Cross Validation Prediction Accuracy: 1 – [Σ(predictedi – actuali)2 / Σ(actuali – actualmean)2] Test Set Map Time Reduce Time Total Time Map Output Bytes mod 4 (old) 0. 60 0. 80 0. 70 0. 50 mod 3 0. 84 0. 85 0. 86 0. 94 mod 2 0. 81 0. 76 0. 83 0. 94 mod 1 0. 91 0. 77 0. 88 0. 97 mod 0 (new) 0. 96 0. 89 0. 97 0. 91
Train with Sampled Subset Prediction Accuracy: 1 – [Σ(predictedi – actuali)2 / Σ(actuali – actualmean)2] Training Set Map Time Reduce Time Total Time Map Output Bytes All old data 0. 96 0. 89 0. 97 0. 91 sample 1 0. 93 0. 87 0. 94 0. 85 sample 2 0. 62 0. 94 0. 81 sample 3 0. 76 0. 91 0. 72 sample 4 0. 64 0. 83 0. 78 0. 71 sample 5 0. 69 0. 87 0. 82 0. 74
Microsoft’s Chicago Modular Datacenter 85
The Million Server Datacenter • 24000 sq. m housing 400 containers – Each container contains 2500 servers – Integrated computing, networking, power, cooling systems • 300 MW supplied from two power substations situated on opposite sides of the datacenter • Dual water-based cooling systems circulate cold water to containers, eliminating need for air conditioned rooms 86
High workload, low put rate
High workload, high put rate
Low workload, low put rate
2020 IT Carbon Footprint 820 m tons CO 2 360 m tons CO 2 2007 Worldwide IT carbon footprint: 2% = 830 m tons CO 2 Comparable to the global aviation industry Expected to grow to 4% by 2020 260 m tons CO 2 90
Energy & Cloud Computing? • Cloud Computing saves Energy? • Don’t buy machines for local use that are often idle • Better to ship bits as photons over fiber vs. ship electrons over transmission lines to spin disks, power processors locally – Clouds use nearby (hydroelectric) power – Leverage economies of scale of cooling, power distribution 91
Energy & Cloud Computing? • Techniques developed to stop using idle servers to save money in Cloud Computing can also be used to save power – Up to Cloud Computing Provider to decide what to do with idle resources • New Requirement: Scale DOWN and up – Who decides when to scale down in a datacenter? – How can Datacenter storage systems improve energy? 92
Hybrid / Surge Computing • Keep a local “private cloud” running same protocols as public cloud • When need more, “surge” onto public cloud, and scale back when need fulfilled • Saves energy (and capital expenditures) by not buying and deploying power distribution, cooling, machines that are mostly idle 93
Скачать презентацию UC Berkeley Using Statistical Machine Learning in Cloud
404ddb843a9d51ab5a26d768064fb87d.ppt