Solving LP Models n Improving Search n n

Solving LP Models n Improving Search n n Special Form of Improving Search n n Part 3 Unimodal Convex feasible region Should be successful! Simplex method (now) Interior point methods (later) IE 312 1

Simple Example n n Top Brass Trophy Company Makes trophies for n football n n n soccer n n n wood base, engraved plaque, brass football on top $12 profit and uses 4’ of wood base, engraved plaque, soccer ball on top $9 profit and uses 2’ of wood Current stock n 1000 footballs, 1500 soccer balls, 1750 plaques, and 4800 feet of wood Part 3 IE 312 2

Formulation Part 3 IE 312 3

Graphical Solution 2000 1500 Optimal Solution 1000 500 Part 3 1000 1500 IE 312 2000 4

Feasible Solutions n Feasible solution is a n n n Part 3 boundary point if at least one inequality constraint that can be strict is active interior point if no such constraints are active Extreme points of convex sets do not lie within the line segment of any other points in the set IE 312 5

Example 2000 1500 1000 500 Part 3 1000 1500 IE 312 2000 6

Optimal Solutions n Every optimal solution is a boundary point n n n Part 3 We can find an improving direction whenever we are at an interior point If optimum unique the it must be an extreme point of the feasible region If optimal solution exist, an optimal extreme point exists IE 312 7

LP Standard Form n n Easier if we agree on exactly what a LP should look like Standard form n n Part 3 only equality main constraints only nonnegative variables appear at most once in left-handside and objective function all constants appear on right hand side IE 312 8

Converting to Standard n Inequality constraints n n Variables not nonnegative n n Part 3 Add nonnegative, zero-cost slack variables Add in inequalities Subtract in inequalities nonpositive - substitute with negatives unrestrictive sign (URS) - substitute difference of two nonnegative variables IE 312 9

Top Brass Model Part 3 IE 312 10

Why? n Feasible directions n n n Equality constraints n n Always active Inequality constraints n n Check only if active Keep track of active constraints May or may not be active Prefer equality constraints! Part 3 IE 312 11

Standard Notation Part 3 IE 312 12

LP Standard Form n In standard notation n In matrix notation Part 3 IE 312 13

Write in Matrix Form Part 3 IE 312 14

Extreme Points n n n Part 3 Know that an extreme point optimum exists Will search trough extreme points An extreme point is define by a set of constraints that are active simultaneously IE 312 15

Improving Search n n n Part 3 Move from one extreme point to a neighboring extreme point Extreme points are adjacent if they are defined by sets of active constraints that differ by only one element An edge is a line segment determined by a set of active constraints IE 312 16

Basic Solutions n n Part 3 Extreme points are defined by set of active nonnegativity constraints A basic solution is a solution that is obtained by fixing enough variable to be equal to zero, so that the equality constraints have a unique solution IE 312 17

Example Choose x 1, x 2, x 3, x 4 to be basic Part 3 IE 312 18

Where is the Basic Solution? 2000 1500 1000 500 Part 3 1000 1500 IE 312 2000 19

Example n Compute the basic solution for x 1 and x 2 basis: n Solve Part 3 IE 312 20

Existence of Basis Solutions n n Part 3 Remember linear algebra? A basis solution exists if and only if the columns of corresponding equality constraint form a basis (in other words, a largest possible linearly independent collection) IE 312 21

Checking n n n Part 3 The determinant of a square matrix D is A matrix is singular if its determinant = 0 and otherwise nonsingular Need to check that the matrix is nonsingular IE 312 22

Example n Part 3 Check whether basic solutions exist for IE 312 23

Basic Feasible Solutions n n Part 3 A basic feasible solution to a LP is a basic solution that satisfies all the nonnegativity contraints The basic feasible solutions correspond exactly to the extreme points of the feasible region IE 312 24

Example Problem n n Suppose we have x 3, x 4, x 5 as slack variables in the following LP: Lets plot the original problem, compute the basic solutions and check feasibility Part 3 IE 312 25

Solution Algorithm n Simplex Algorithm n n Variant of improving search Standard display: Part 3 IE 312 26

Simplex Algorithm n Starting point n n A basic feasible solution (extreme point) Direction n Follow an edge to adjacent extreme point: n n n Part 3 Increase one nonbasic variable Compute changes needed to preserve equality constraints One direction for each nonbasic variable IE 312 27

Top Brass Example Initial solution Part 3 Basic variables IE 312 28

Looking in All Directions … Can increase either one of those Part 3 Must adjust these! IE 312 29

So Many Choices. . . n Want to try to improve the objective n The reduced cost of a nonbasic variable: n Want Part 3 Defines improving direction IE 312 30

Top Brass Example n Improving x 1 gives n Improving x 2 gives n Both directions are improving directions! Part 3 IE 312 31

Where and How Far? n n n Part 3 Any improving direction will do If no component is negative Improve forever - unbounded! Otherwise, compute the minimum ratio IE 312 32

Computing Minimum Ratio Part 3 IE 312 33

Moving to New Solution Part 3 IE 312 34

Updating Basis n New basic variable n n New nonbasic variable(s) n Part 3 Nonbasic variable generating direction Basic variables fixing the step size IE 312 35

What Did We Do? 2000 1500 1000 500 Part 3 1000 1500 IE 312 2000 36

Where Will We Go? 2000 Optimum in three steps! 1500 Why is this guaranteed? 1000 500 Part 3 1000 1500 IE 312 2000 37

Simplex Algorithm (Simple) Step 0: Initialization. Choose starting feasible basis, construct basic solution x(0), and set t=0 Step 1: Simplex Directions. Construct directions Dx associated with increasing each nonbasic variable xj and compute the reduced cost cj =c ·Dx. Step 2: Optimality. If no direction is improving, then stop; otherwise choose any direction Dx(t+1) corresponding to some basic variable xp. Step 3: Step Size. If no limit on move in direction Dx(t+1) then stop; otherwise choose variable xr such that Step 4: New Point and Basis. Compute the new solution and replace xr in the basis with xp. Let t = t+1 and go to Step 1. Part 3 IE 312 38

Stopping n n n Part 3 The algorithm stop when one of two criteria is met: In Step 2 if no improving direction exists, which implies local optimum, which implied global optimum In Step 3 if no limit on improvement, which implies problem is unbounded IE 312 39

Optimization Software n Spreadsheet (e. g, MS Excel with What’s Best!) n Optimizers (e. g. , LINDO) n Combination n Modeling Language Solvers Either together (e. g. , LINGO) or separate (e. g. , GAMS with CPLEX) n LINDO and LINGO are in Room 0010 (OR Lab) n Also on disk with your book Part 3 IE 312 40

LINDO n n n Part 3 The main software that I’ll ask you to use is called LINDO Solves linear programs (LP), integer programs (IP), and quadratic programs (QP) We will look at many of its more advanced features later on, but as of yet we haven’t learned many of the concepts that we need IE 312 41

Example Part 3 IE 312 42

LINDO Program MAX 12 x 1 + 9 x 2 ST x 1 + x 2 = 1000 x 2 + x 4 = 1500 x 1 + x 2 + x 5 = 1750 4 x 1+ 2 x 2 + x 6 = 4800 x 1>=0 x 2>=0 x 3>=0 x 4>=0 x 5>=0 x 6>=0 END Part 3 IE 312 43

Part 3 IE 312 44

Part 3 IE 312 45

Output LP OPTIMUM FOUND AT STEP 4 OBJECTIVE FUNCTION VALUE 1) VARIABLE 12000. 00 VALUE REDUCED COST X 1 0. 000000 X 2 0. 000000 3. 000000 X 4 1500. 000000 X 5 750. 000000 X 6 800. 000000 X 3 Part 3 1000. 000000 IE 312 46

Output (cont. ) ROW SLACK OR SURPLUS DUAL PRICES 2) 0. 000000 12. 000000 3) 0. 000000 4) 0. 000000 5) 0. 000000 6) 1000. 000000 7) 0. 000000 8) 0. 000000 9) 1500. 000000 10) 750. 000000 11) 800. 000000 NO. ITERATIONS= Part 3 4 IE 312 47

LINDO: Basic Syntax n n n Part 3 Objective Function Syntax: Start all models with MAX or MIN Variable Names: Limited to 8 characters Constraint Name: Terminated with a parenthesis Recognized Operators (+, -, >, <, =) Order of Precedence: Parentheses not recognized IE 312 48

Syntax (cont. ) n n n Part 3 Adding Comment: Start with an exclamation mark Splitting lines in a model: Permitted in LINDO Case Sensitivity: LINDO has none Right-hand Side Syntax: Only constant values Left-hand Side Syntax: Only variables and their coefficients IE 312 49

Why Modeling Language? n n n More to learn! More ‘complicated’ to use than LINDO (at least at first glance) Advantages n n Part 3 Natural representations Similar to mathematical notation Can enter many terms simultaneously Much faster and easier to read IE 312 50

Why Solvers? n Best commercial software has modeling language and solvers separated n Advantages: n n Learn one modeling language use any solver n Part 3 Select solver that is best for your application Buy 3 rd party solvers or write your own! IE 312 51

Example Problem Part 3 IE 312 52

Problem Formulation Part 3 IE 312 53

LINDO Solution max 1. 60 x 1 + 1. 40 x 2 + 1. 90 x 3 + 1. 20 x 4 st x 1 x 1 x 2 x 3 x 4 x 5 + x 2 + x 3 + x 4 <=1200 >= 310 <= 434 >= 245 <= 343 >= 255 <= 357 >= 190 <= 266 end Part 3 IE 312 54

LINGO Solution n Capacity constraint @SUM(REGIONS(I): CASES(I)) <=1200; n Minimum/maximum cases @FOR(REGIONS(I): CASES(I) <= UBOUND; CASES(I) >= LBOUND); Part 3 IE 312 55

LINGO Solution n Objective function MAX = @SUM(REGIONS(I): PROFIT*CASES(I)); n Part 3 We also need to define REGIONS, CASES, etc, and type in the data. IE 312 56

LINGO Solution n Defining sets SETS: REGIONS / NE SE MW W/: LBOUND, UBOUND, PROFIT, CASES; ENDSETS Part 3 IE 312 57

LINGO Solution n Enter the data DATA: LBOUND = 310 245 255 190; UBOUND = 434 343 357 266; PROFIT = 1. 6 1. 4 1. 9 1. 2; ENDDATA Part 3 IE 312 58

Sensitivity Analysis n Basic Question: How does our solution change as the input parameters change? n The objective function? n n The optimal values of decision variables? n n Make different decisions! Why? n n Part 3 More/less profit or cost Only have estimates of input parameters May want to change input parameters IE 312 59

What We Know n n Qualitative Answers for All Problems Quantitative Answers for Linear Programs (LP) n n n Part 3 Dual program Same input parameters Decision variables give sensitivities Dual prices Easy to set up Theory is somewhat complicated IE 312 60

Back to Example Problem Part 3 IE 312 61

LINDO Formulation max st 1. 60 x 1 + 1. 40 x 2 + 1. 90 x 3 + 1. 20 x 4 x 1 x 1 x 2 x 3 x 4 x 5 Part 3 end + x 2 + x 3 + x 4 <=1200 >= 310 <= 434 >= 245 <= 343 >= 255 <= 357 >= 190 <= 266 IE 312 62

LINDO Solution (second half) ROW SLACK OR SURPLUS DUAL PRICES 2) 1. 600000 3) 98. 000000 0. 000000 4) 26. 000000 0. 000000 5) 0. 000000 -0. 200000 6) 98. 000000 0. 000000 7) 102. 000000 0. 000000 8) 0. 000000 0. 300000 9) 0. 000000 -0. 400000 10) Part 3 0. 000000 266. 000000 0. 000000 IE 312 63

Dual Prices n The Dual is Automatically Formed n n n Report dual prices n n Part 3 Also in LINGO Also in (all) other optimization software Gives us sensitivities to RHS parameter Know how much objective function will change When will the optimal solution change? Need to select that we want sensitivity analysis IE 312 64

LINDO Sensitivity Analysis (part) RANGES IN WHICH THE BASIS IS UNCHANGED: OBJ COEFFICIENT RANGES VARIABLE CURRENT COEF ALLOWABLE INCREASE DECREASE X 1 0. 300000 0. 200000 X 2 1. 400000 0. 200000 INFINITY X 3 1. 900000 INFINITY 0. 300000 X 4 1. 200000 0. 400000 INFINITY X 5 Part 3 1. 600000 0. 000000 INFINITY IE 312 65

Interpretation n Part 3 As long as prices for the NE region are between $1. 4 and $1. 9, we want to sell the same quantity to each region, etc. IE 312 66

Example An insurance company is introducing two new product lines: special risk insurance and mortgages. The expected profit is $5 per unit on special risk insurance and $2 per unit on mortgages. Management wishes to establish a sales target for the new product lines to maximize the expected profit. The work requirements are as follows: Part 3 IE 312 67

LINDO Formulation max 5 x 1 + 2 x 2 st 3 x 1 + 2 x 2 <= 2400 x 2 <= 800 2 x 1 <= 1200 x 1 >=0 x 2 >=0 end Part 3 IE 312 68

Graphical Solution 800 700 600 500 400 300 200 100 200 300 400 500 600 700 800 Part 3 IE 312 69

Solution VARIABLE VALUE REDUCED COST X 1 600. 000000 X 2 300. 000000 ROW SLACK OR SURPLUS DUAL PRICES 2) 1. 000000 3) 500. 000000 4) 0. 000000 1. 000000 5) 600. 000000 6) Part 3 0. 000000 300. 000000 IE 312 70

Sensitivity Analysis RANGES IN WHICH THE BASIS IS UNCHANGED: VARIABLE X 1 X 2 ROW 2 3 4 5 6 Part 3 CURRENT COEF 5. 000000 2. 000000 OBJ COEFFICIENT RANGES ALLOWABLE INCREASE DECREASE INFINITY 2. 000000 1. 333333 2. 000000 CURRENT RHS 2400. 000000 800. 000000 1200. 000000 RIGHTHAND SIDE RANGES ALLOWABLE INCREASE DECREASE 1000. 000000 600. 000000 INFINITY 500. 000000 400. 000000 666687 600. 000000 INFINITY 300. 000000 INFINITY IE 312 71

New Decisions! 800 700 600 500 Optimum Moves! 400 300 200 100 200 300 400 500 600 700 800 Part 3 IE 312 72

What-If ? Solve New Problem max 5 x 1 + 2 x 2 st 3 x 1 + 2 x 2 <= 2400 x 2 <= 290 2 x 1 <= 1200 x 1 >=0 x 2 >=0 end Part 3 IE 312 73

New Solution VARIABLE VALUE REDUCED COST X 1 600. 000000 X 2 290. 000000 ROW SLACK OR SURPLUS DUAL PRICES 2) 0. 000000 3) 0. 000000 2. 000000 4) 0. 000000 2. 500000 5) 600. 000000 6) Part 3 20. 000000 290. 000000 IE 312 74

Interior Point Methods n Simplex always stays on the boundary n Can take short cuts across the interior n Interior point methods n n More improvement in each move n Part 3 More effort in each move Much faster for large problems IE 312 75

Simple Example n Frannie’s Firewood sells up to 3 cords of firewood to two customers n n One will pay $90 per half-cord Other will pay $150 per full cord Part 3 IE 312 76

Graphical Solution 4 3 2 1 1 Part 3 2 3 IE 312 4 5 6 77

Improving Directions n n n Part 3 Which direction improves the objective function the most? The gradient Direction: IE 312 78

Most Improving Direction? Part 3 IE 312 79

Back to Example 4 3 2 1 1 Part 3 2 3 IE 312 4 5 6 80

Maintaining Feasibility n n n At the initial point all directions are feasible because it is an interior point At the new point we have to make sure that a direction Dx at x(1) satisfies Interior point algorithms begin inside and move through the interior, reaching the boundary only at an optimal solution Part 3 IE 312 81

Valid Interior Point Search? 2000 1500 1000 500 Part 3 1000 1500 IE 312 2000 82

Valid Interior Point Search? 2000 1500 1000 500 Part 3 1000 1500 IE 312 2000 83

Valid Interior Point Search? 2000 1500 1000 500 Part 3 1000 1500 IE 312 2000 84

LP Standard Form n For Simplex used the form n In Frannie’s Firewood Part 3 IE 312 85

Benefits of Standard Form? n In Simplex: n n n Here quite similar: n n n Made easy to check which variables are basic, nonbasic, etc. Needed to know which solutions are on boundary Know which are not on boundary Check that nonnegativity constraints are strict! A feasible solution to standard LP is interior point if every component is strictly positive Part 3 IE 312 86

Interior Points? Part 3 IE 312 87

Projections n n n Must satisfy main equality constraints Want direction Dx that satisfies this equation and is as nearly d as possible The projection of a vector d onto a system of equalities is the vector that satisfies the constraints and minimizes the total squared difference between the components Part 3 IE 312 88

Obtaining Projection n The projection of d onto ADx=0 is where is the projection matrix. Part 3 IE 312 89

Example: Frannie’s Firewood Part 3 IE 312 90

Example n Part 3 The cost vector is c=(90 150 0) IE 312 91

Improvement n n n Part 3 The projection matrix is design to make an improving direction feasible with minimum changes Is it still an improving direction? Yes! The projection Dx=Pc of c onto Ax=b is an improving direction at every x IE 312 92

Sample Exercise n n Part 3 Determine the direction d of most rapid improvement Project it onto the main equality constraints to get Dx Verify that the move direction Dx is feasible Verify that the move direction Dx is improving IE 312 93