IBM ILOG CPLEX Advanced Performance Tuning Optimization Technologies

CPLEX Performance Tuning Overview § Speed § Continuous Models (LP, QCP/SOCP) § MIP § Memory § Operating System § Deployment Application § CPLEX § Numerical § Ill conditioning § Numerical Instability § Summary 2 Copyright © ILOG 2008

Tuning for Speed: Continuous Models § Try each algorithm § Primal Simplex § Dual Simplex § Network Simplex § Barrier (only option for QCP/SOCP) § Sifting (LP only) § Concurrentopt (LP only, parallel processors) 3 Copyright © ILOG 2008

Tuning for Speed: Continuous Models § Other tactics § Presolve dual on (apply presolve, then solve the dual model) § LP only Constraints : 1972170 [Greater: 1972170] Variables : [Free: 275036] 275036 Constraint nonzeros: 9234686 Objective nonzeros: 156817 RHS nonzeros: 1487481 § Parallel barrier method § Check for numerical difficulties 4 Copyright © ILOG 2008

Tuning for Speed: Continuous Models § Iteration logs § Degeneracy (simplex methods) § Excessive

Tuning for Speed : Continuous Models § Degeneracy, sample log Iter: 4751 Infeasibility = 8. 000000 Iter: 4870 Infeasibility = 8. 000000 Iter: 4976 Infeasibility = 6. 999999 Iter: 5083 Infeasibility = 6. 000000 Iter: 5191 Infeasibility = 6. 000000 Iter: 5862 Infeasibility = 6. 000000 Iter: 5968 Infeasibility = 6. 000000 . . . Perturbation started. 6 Copyright © ILOG 2008

Tuning for Speed: Continuous Models § Excessive time per iteration, sample log Elapsed time = 138. 23 sec. (37000 iterations) Infeasibility = 387849. 999786 Iter: 39121 Infeasibility = 379979. 999768 Iter: 40295 12 sec. Iter: 37969 Infeasibility = 375639. 999998 Elapsed time = 150. 41 sec. (41000 iterations) . . . Elapsed time = 24318. 58 sec. (138000 iterations) 827 sec. Iter: 138958 Infeasibility = 23. 754244 Elapsed time = 24736. 98 sec. (139000 iterations) Elapsed time = 25145. 43 sec. (140000 iterations) 7 Copyright © ILOG 2008

Tuning for Speed: Continuous Models § Degeneracy § Try each algorithm § Instruct CPLEX to perturb the model § Excessive time per iteration § Try each algorithm § Check for virtual memory usage § Run with more memory available § Turn on memory emphasis parameter § Consider alternate pricing (pgradient/dgradient) § Time per iteration depends on pricing scheme 8 Copyright © ILOG 2008

Tuning for Speed: MIP § Try the Tuning Tool (CPLEX 11 or later) § Primarily for MIPs, but works for all problem types § Works with all CPLEX interfaces, including interactive optimizer § Can make use of internal algorithm information § Let CPLEX and your computer do the work 9 Copyright © ILOG 2008

Tuning for Speed: MIP § Input: Model(s) to tune, tuning instructions § Problem in memory or set of problems in files § Fixed parameter set, parameters which should not be tuned, e. g. , tolerances, threads § Parameters, metrics to control tuning process (e. g. total tuning time, average or worst case run time) § Output: Tuned and fixed parameter settings 10 Copyright © ILOG 2008

Tuning for Speed: MIP § Examine node log (if Tuning Tool doesn’t help) § Excessive iteration counts for node relaxations. § Use different start/sub algorithm. § Barrier method doesn’t support restarts. § Set dual gradient to exact steepest edge norms, file parameter to store node file on disk and compress it. § More progress needed in best integer or best node value? 11 Copyright © ILOG 2008

Tuning for Speed: MIP § Excessive iteration counts for node LPs, sample node log #1 (dual simplex to solve node LPs) Nodes Node Left 0 0 0 2 1 1 2 2 3 1 … 8 2 9 1 … Solution time = Objective IInf -89. 0000 infeasible 6 6 6 -89. 0000 infeasible 8 5 Best Integer Cuts/ Best Node It. Cnt -89. 0000 Fract: 4 -89. 0000 5278 12799 20767 27275 32502 -89. 0000 65717 73714 177. 33 sec. Iterations = 73714 Nodes = 10 (1) 12 Copyright © ILOG 2008

Tuning for Speed: MIP § Excessive iteration counts for node LPs, sample node log #2 (primal simplex to solve node LPs) Nodes Node Left 0 0 0 2 1 1 2 2 3 1 … 8 2 9 1 … Solution time = Objective IInf -89. 0000 infeasible 5 5 5 -89. 0000 infeasible 8 5 Best Integer Cuts/ Best Node It. Cnt -89. 0000 Fract: 5 -89. 0000 6603 7120 9621 10616 12963 -89. 0000 21522 23891 54. 37 sec. Iterations = 23891 Nodes = 10 (1) 13 Copyright © ILOG 2008

Tuning for Speed: MIP § Progress in best node but not best integer: Nodes Node Left Objective IInf Best Integer Cuts/ Best Node 22. 0000 22. 3333 22. 6667 4433 5196 6164 29. 03% 27. 96% 26. 88% 25. 6667 25. 7778 55739 56424 57267 17. 20% 16. 85% It. Cnt Gap . . . 300 400 500 229 309 387 22. 6667 cutoff 26. 5000 40 31 31. 0000 7800 7900 8000 5260 5324 5385 28. 5000 28. 2500 27. 3750 23 26 30 31. 0000 . . . 14 Copyright © ILOG 2008

Tuning for Speed: MIP § Progress in best integer but not best node: Nodes Node Left Objective IInf Best Integer Cuts/ Best Node 2018. 0000 560. 0000 810. 0000 27 0 0 291 175 3780. 0000 2626. 0000 2590. 0000 560. 0000 3703 4405 4553 5747 85. 19% 78. 67% 78. 38% 1544. 0000 944. 0000 1468. 0000 0 110 176 93 1710. 0000 560. 0000 66026 66279 66801 67732 67. 25% It. Cnt Gap … * * 300+ 393 400 500 296 368 372 472 7740+ 7800 7900 8000 5183 5240 5325 5424 . . . * 15 Copyright © ILOG 2008

Tuning for Speed: MIP § Tactics for lack of progress in best integer § Use especially if customer accepts a good, perhaps sub optimal solution. § MIP emphasis § Set to 1 (emphasize integer feasibility) § Set to 4 (emphasize finding hidden integer solutions) § Solution Polishing § Requires an integer feasible solution § Use to improve sub optimal solutions at end of MIP search § Or use as replacement for majority of MIP search when - local search works well - node LPs are time consuming - improving best node appears hopeless 16 Copyright © ILOG 2008

Tuning for Speed: MIP § Lack of progress in best integer (continued) § RINS Heuristic § Local search heuristic - Needs an integer feasible solution § MIP Starts and Solution Repair § CPLEX 10. 0 and later accepts partial MIP starts - Solution repair for (partial) infeasible MIP starts § May make RINS, Solution Polishing more effective § Feasibility Pump Heuristic (CPLEX 11 and later) § Applied first at root node 17 Copyright © ILOG 2008

Tuning for Speed: MIP § Lack of progress in best integer (continued) § Probing § Set to 3 (maximum probing) § Cuts § Set to 2 (aggressive cut generation) § Cuts and Probing don’t explicitly help find feasible solutions, but the resulting tighter formulation may help. § Can be useful when combined with setting MIP emphasis to 1 (emphasize integer feasibility) 18 Copyright © ILOG 2008

Tuning for Speed: MIP § Tactics for lack of progress in best node § MIP emphasis § Set to 2 or 3 to emphasize proving optimality and moving best node respectively. - May need Partial/Infeasible MIP start, Feasibility Pump or RINS Heuristic to compensate for fewer integer feasible solutions § Probing § Set to 3 (maximum probing) § Cuts § Set to 2 (aggressive cut generation) § Cuts and Probing often tighten the model formulation 19 Copyright © ILOG 2008

Tuning for Speed: MIP § Tactics for lack of progress in best node and best integer § Set MIP emphasis to 3 § May cause difficulties finding integer feasible solutions § Compensate by using (partial or infeasible) MIP starts, Feasibility Pump, RINS Heuristic, Solution Polishing to help find more integer feasible solutions 20 Copyright © ILOG 2008

Tuning for Speed: MIP § Tactics for lack of progress in best node and best integer (continued) § Provide a starting solution, regardless of objective value § Can help CPLEX’s local search heuristics § Can yield additional tightening of the model § Consider external methods (e. g. Constraint Programming, Metaheuristics like Simulated Annealing, model specific heuristics) to find a starting solution § Try CPLEX’s feasopt feature 21 Copyright © ILOG 2008

Tuning for Speed: MIP § Other tactics § Specific model knowledge determines parameter settings

Tuning for Speed: MIP § Other tactics (continued) § Specific model knowledge determines priority order. § Time period models - Higher priority to earlier time periods. § Dependencies among integer variables 23 Copyright © ILOG 2008

Tuning for Speed: MIP § Other tactics (continued) § Specific model knowledge determines changes to model § Adding model specific cuts - Tighten formulation - Remove symmetry § Sherali, Smith “Improving Zero-One Model Representations Via Symmetry Considerations” § Model reformulation http: //www. ilog. com/products/cplex/newsletters/cplex_newsletter_9410. cfm 24 Copyright © ILOG 2008

Memory Tuning: Operating System § Virtual memory slows performance § Task Manager (Windows) § Top, ps, vmstat under Unix § Make sure you can allocate all available memory § Java VM limitations § Use –Xmx option to increase heap size § On a 32 -bit machine, the real limit is 2 GB § Limits on individual accounts § Compiler/Linker options § 32 bit AIX has default of 256 MB for memory allocations. § Write a simple program to test available memory (or ask us for one). 25 Copyright © ILOG 2008

Memory Tuning: Deployment Considerations § Compare memory usage § Solve the Deployment application model(s) with: § Interactive CPLEX § Simple example programs § Purify, Insure, Bounds Checker, Valgrind (Linux only) can help with memory leaks § Make sure you solve the correct model § Display problem statistics § Data check parameter § Diagnostic routines in check. c 26 Copyright © ILOG 2008

Memory Tuning: CPLEX Considerations § Sometimes CPLEX really uses all available memory § Default settings emphasize speed over memory conservation. § Is CPLEX memory usage reasonable? § 1 GB per million constraints of an LP - Exceptions: large aspect ratios, very dense constraints § 1 GB per million (constraints + quadratic variables) of a QP - Exception: dense Q matrix lacking sparse factorization § At least that much for MIP, MIQP 27 Copyright © ILOG 2008

Memory Tuning: CPLEX Considerations § Memory conservation techniques § General § Turn memory emphasis on (slight extra run time) § Use different algorithm (LP and QP) § As a last resort, turn presolve off (potential big increase in run time) § LP § Set presolve dual indicator on 28 Copyright © ILOG 2008

Memory Tuning: CPLEX Considerations § Memory conservation techniques (continued) § MIP search tree often dominates memory usage - Performance tuning to reduce node count. - Setting node file parameter to 3 efficiently writes nodes to disk. - Setting variable select parameter to 3 (strong branching). Fewer nodes required to make the same amount of progress. 29 Copyright © ILOG 2008

Memory Tuning: Other Tactics § Buy more memory (but only after accurately estimating requirements) § Specialized algorithms § Example: Column generation for models with large aspect ratios. 30 Copyright © ILOG 2008

Memory Tuning: Other Tactics § Review of model development process § Model reformulation § Reduce problem dimensions § Reduce “difficulty” while increasing problem size. § MIP reformulations for column generation. § Increase in problem size in exchange for sparser problem data 31 Copyright © ILOG 2008

Numerical Tuning: Ill conditioning § Sensitivity of computed solutions to perturbations in problem data § Condition number of a matrix § CPLEX solves systems of equations of the form § Exact solution is 32 Copyright © ILOG 2008

Numerical Tuning: Ill conditioning § Condition number of a matrix (continued) § To measure effects of perturbations, consider § Exact solution is § Combine with exact solution of x: 33 Copyright © ILOG 2008

Numerical Tuning: Ill conditioning § Condition number of a matrix (ctd). § Using Cauchy - Schwarz inequality § This gives max. absolute change in x relative to change in b. Now consider relative change by using Cauchy - Schwarz inequality in original system: 34 Copyright © ILOG 2008

Numerical Tuning: Ill conditioning § Condition number of a matrix (continued) § Combine and

Numerical Tuning: Ill conditioning § Ill conditioning can occur under perfect arithmetic § But, finite precision can perturb the exact system of equations we wish to solve, resulting in significant changes to the computed solution 36 Copyright © ILOG 2008

Numerical Tuning: Ill conditioning § Sources of perturbations due to finite precision § Rounding of data § Near singular matrices § Avoid computing data in single precision § Round off error in the simplex method computations, particularly the basis factorization. § Round off error due to scaling of large data values § Floating point calculations involving values of greatly different orders of magnitude § Avoid mixtures of very large and small coefficients 37 Copyright © ILOG 2008

Numerical Tuning: Ill conditioning § Rounding of data Maximize x 1 + x 2 c 1: 1/3 x 1 + 2/3 x 2= 1 c 2: x 1 + 2 x 2 = 3 Solution: x 1 = 3. 0 Maximize x 1 + x 2 c 1: . 333333 x 1 +. 666667 x 2 = 1 c 2: x 1 + 2 x 2 = 3 Solution: x 1 = x 2 = 1. 0, cond(B) = 6. 66667 e+06 Maximize x 1 + x 2 c 1: . 33333 x 1 +. 66667 x 2 = 1 c 2: x 1 + 2 x 2 = 3 Solution: x 1 = 3. 0, cond(B) = 6. 333333 38 Copyright © ILOG 2008

Numerical Tuning: Ill conditioning § Avoid rounding whenever possible Exact formulation: Maximize x 1 + x 2 c 1: 1/3 x 1 + 2/3 x 2= 1 c 2: x 1 + 2 x 2 = 3 Imprecisely rounded: Maximize x 1 + x 2 c 1: . 333333 x 1 +. 666667 x 2 = 1 c 2: x 1 + 2 x 2 = 3 Scale to integral value whenever possible: Maximize x 1 + x 2 c 1: x 1 + 2 x 2 = 3 c 2: x 1 + 2 x 2 = 3 39 Copyright © ILOG 2008

Numerical Tuning: Ill conditioning § Rounding of data (continued) § Changes in right hand side can have similar effect Maximize x 1 + x 2 c 1: . 333333 x 1 +. 666667 x 2 = 1 c 2: x 1 + 2 x 2 = 3 Solution: x 1 = x 2 = 1. 0 Maximize x 1 + x 2 c 1: . 333333 x 1 +. 666667 x 2 =. 999999 c 2: x 1 + 2 x 2 = 3 Solution: x 1 = 3. 0 40 Copyright © ILOG 2008

Numerical Tuning: Numerical Instability § Large values in problem data § Bounds can magnify

Symptoms of Numerical Instability § Sample iteration log Iter: 6389 Objective = 13137. 039899 Iter: 6390 Objective = 13137. 039899 Iter: 6391 Objective = 13726. 011591 Markowitz threshold set to 0. 3. Iter: 6456 Scaled infeas = 300615. 030682. . . Iter: 6752 Scaled infeas = 0. 000002 Iter: 6754 Objective = -23870. 812630 42 Copyright © ILOG 2008

Numerical Tuning Tactics § General § Turn numerical emphasis on § Rounding of data § Compute data as accurately as possible § Round off error in simplex method § Set Markowitz tolerance to. 90 (or higher if necessary) § Large values in problem data § Reduce large coefficients if possible § Set scaling parameter to 1 (aggressive scaling) 43 Copyright © ILOG 2008

CPLEX Performance Tuning Summary § CPLEX solves most problems with no trouble § Organized tactics for the outliers can yield dramatic savings in time. § Different categories of performance problems. § Use all available information § Iteration and node logs for LP and MIP § Interactive CPLEX § Simple test programs (CPLEX and your own) § OS memory management tools § Purify/Insure/Bounds Checker/Valgrind § ILOG Technical Support and Consulting 44 Copyright © ILOG 2008