Introduction to Linear Programming Romil Jain The

Introduction to Linear Programming Romil Jain

The Nutrition Problem • Each fruit contains different nutrients • Each fruit has different cost An apple a day keeps the doctor away – but apples are costly! A customer’s goal is to fulfill daily nutrition requirements at lowest cost.

The Nutrition Problem (cont’d) • Lets take a simpler case of just apples and bananas. Calories Vitamins Cost($) 2 4 3 3 5 7 Cost Function Cost = 5 x 1 + 7 x 2 Constraint Functions C 1: 2 x 1 + 4 x 2 ³ 100 C 2: 3 x 1 + 3 x 2 ³ 90 x 1, x 2 ³ 0 • Must take at least 100 units of Calories & 90 units of Vitamins for good nutrition. • A customer’s goal is to buy fruits in such a quantity that it minimizes cost but fulfills nutrition.

The Nutrition Problem (cont’d) • Matrix Representation Constraints: 2 x 1 + 4 x 2 ³ 100 3 x 1 + 3 x 2 ³ 90 Non-negativity: x 1, x 2 ³ 0 Cost function = 5 x 1 + 7 x 2 Ci Xi Real life problems may have many variables and constraints ! Mi, j Xi ³ Ni

Approach to Solution x 2 Cost Function Cost = 5 x 1 + 7 x 2 Constraint Functions C 1: 2 x 1 + 4 x 2 ³ 100 C 2: 3 x 1 + 3 x 2 ³ 90 x 1, x 2 ³ 0 (0, 0) C 2 C 1 x 1

Cost Function Cost = 5 x 1 + 7 x 2 Constraint Functions C 1: 2 x 1 + 4 x 2 ³ 100 C 2: 3 x 1 + 3 x 2 ³ 90 x 1, x 2 ³ 0 x 2 Simplex Algorithm Hill Climbing - Evaluate the vertices! Same approach for n variables with m constrains (in n-dimensional space). Maximum of m. Cn vertices C 2 (0, 0) C 1 x 1

Simplex Algorithm (cont’d) • A fast algorithm to solve Linear Programs • Invented by George Dantzig in 1947 • Computes solution to a Linear Program by evaluating vertices where constraints intersect each other.

Carbohydrates, vitamins (A, B, C, D, E), Proteins, minerals, fiber Simplex Algorithm (cont’d) Evaluate vertices which satisfy constraints! Minimize Cost : 21 x 1 - 6 x 2 – 100 x 3 Constraint Functions: 5 x 1 + 2 x 2 +31 x 3 ³ 21 1 x 1 - 4 x 2 +3 x 3 ³ 56 6 x 1 + 60 x 2 - 31 x 3 ³ 200. . . x 1, x 2, x 3 ³ 0

Linear Programs • Linear Program: An optimization problem whose constraints and cost function are linear functions • Goal: Find a solution which optimizes the cost. E. g. Maximize Cost Function : 21 x 1 - 6 x 2 – 100 x 3 - 100 x 4 Constraint Functions: 5 x 1 + 2 x 2 +31 x 3 - 20 x 4 21 1 x 1 - 4 x 2 +3 x 3 + 10 x 1 ³ 56 6 x 1 + 60 x 2 - 31 x 3 - 15 x 4 200 …. . Applied in various industrial fields: Manufacturing, Supply. Chain, Logistics, Marketing… To save money and increase profits !

Linear Programs (cont’d) • Do all Linear Programs have optimal solutions ? No ! • Three types of Linear Programs: 1. Has an optimal solution with a finite cost value: e. g. nutrition problem 2. Unbounded: e. g maximize x, x ³ 5, x ³ 0 3. Infeasible: e. g maximize x, x 3, x ³ 5 , x ³ 0

This can be proved mathematically. But how do we understand the dual linear program semantically? Duality It can be shown that, for every primal linear program, there is a dual linear program. primal linear program minimize C. x M. x ³ N dual linear program maximize NT. y MT. y CT Ci Xi Ni Yi Mi, j Mj, i Xi ³ Ni Yi Ci

Primal LP: minimize C. x Q. x ³ N Semantics of Duality A customer’s goal is to buy fruits in such a quantity that it minimizes cost but fulfills nutrition. Ci Cost of each fruit Quantity of each fruit Coefficients in each column represent the amount of nutrients in a particular food Xi Mi, j Xi ³ Ni Daily nutrition

Dual LP: maximize NT. y QT. y CT Semantics of Duality Imagine a salesman trying to sell supplements for each fruit. Ni Daily nutrition Coefficients in each row represent the amount of nutrients in a particular fruit Yi Mj, i Yi Ci Cost of each fruit But what are Yis in the dual ? Price of each nutrient!

Semantics of Duality • Primal Problem: A customer’s goal is to buy fruits in such a quantity that it minimizes cost but fulfills nutrition. • Dual Problem: A salesman goal is to set price on each nutrient, so that it maximizes profit but his supplements are cheaper than fruits. (Otherwise who will by them? !) Primal (Customer) Dual (Salesman)

References • Introduction To Algorithms, Cormen, Leiserson, Rivest, Stein • How To Think About Algorithms, Edmonds • Course Presentations, Edmonds • An Introduction To Linear Programming and the Simplex Algorithm, Reveliotis (http: //www 2. isye. gatech. edu/~spyros/LP/L P. html)

Thank You! Questions ?

Simplex y (0, 0) Recall we need to evaluate our cost function at the vertices where the constraint functions intersect each other C 2 Cost Function P = 5 x + 7 y Constraint Functions C 1: 2 x + 4 y 100 C 2: 3 x + 3 y 90 Non-Negativity: x, y ³ 0 C 1 x

Simplex (cont’d) We don’t want to deal with complex inequalities Our Equations Can be re-written as: P = 5 x + 7 y Slack Form C 1: 2 x + 4 y 100 s 1 = 100 - 2 x - 4 y C 2: 3 x + 3 y 90 s 2 = 90 - 3 x - 3 y x, y ³ 0 We introduce 2 new variables called slack variables x, y ³ 0 s 1, , s 2 ³ 0

Simplex (cont’d) Cost Function STEP 1: P = 5 x + 7 y • We want an initial point s 1 = 100 - 2 x - 4 y • Lets put x=0, y=0 s 2 = 90 - 3 x - 3 y s 1, , s 2 , x , y ³ 0 x=0, y=0 P=0 Feasible solution

Simplex (cont’d) Cost Function STEP 2: P = 5 x + 7 y • We want next point s 1 = 100 - 2 x - 4 y • Lets try to increase x. s 2 = 90 - 3 x - 3 y • s 1, , s 2 , x , y ³ 0 x can be increased maximum to 30 (s 2 becomes zero) • Rewrite equations (Pivoting) x = 30 – y – s 2/3 s 1 = 40 + 2/3 s 2 – 2 y P = 150 – 5/3 s 2 + 2 y • Now put y, s 2 = 0 x=30, y=0 P = 150 Feasible solution

Simplex (cont’d) Cost Function STEP 3: P = 150 – 5/3 s 2 + 2 y • We want next point x = 30 – y – s 2/3 • Lets try to increase y. s 1 = 40 + 2/3 s 2 – 2 y • s 1, , s 2 , x , y ³ 0 y can be increased maximum to 20 (s 1 becomes zero) • Rewrite equations (Pivoting) (We don’t increase s 2 because it will decrease P) y = 20 + 1/3 s 2 – 1/2 s 1 x = 10 – 1/2 s 1 - 2/3 s 2 ? Is this solution optimal? Or have we run into a local minimum? P = 190 - s 1 – s 2 • Note that we cannot increase s 1 & s 2 without decreasing P. So we stop ! Now put s 1, s 2 = 0 x=10, y=20 P = 190 Feasible solution