2704 BOS 867 849 Topic 10 ORD ITS

2704 BOS 867 849 Topic 10 ORD ITS 033 – Programming & Algorithms Minimum Spanning Tree Problem 1846 LAX 1391 1464 1235 JFK PVD 144 1258 184 802 SFO 337 740 621 187 BWI 1090 DFW 946 1121 MIA 2342 Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University http: //www. siit. tu. ac. th/bunyarit@siit. tu. ac. th 02 5013505 X 2005 1

ITS 033 Midterm Topic 01 - Problems & Algorithmic Problem Solving Topic 02 – Algorithm Representation & Efficiency Analysis Topic 03 - State Space of a problem Topic 04 - Brute Force Algorithm Topic 05 - Divide and Conquer Topic 06 - Decrease and Conquer Topic 07 - Dynamics Programming Topic 08 - Transform and Conquer Topic 09 - Graph Algorithms Topic 10 - Minimum Spanning Tree Topic 11 - Shortest Path Problem Topic 12 - Coping with the Limitations of Algorithms Power http: //www. siit. tu. ac. th/bunyarit/its 033. php http: //www. vcharkarn. com/vlesson/7 2

Outline n Minimum Spanning Trees ¨ Definitions ¨ A crucial fact n The Prim-Jarnik Algorithm n Kruskal's Algorithm 3

2704 BOS 867 849 Topic 10. 1 ORD ITS 033 – Programming & Algorithms Minimum Spanning Tree Problem 1846 LAX 1391 1464 1235 JFK PVD 144 1258 184 802 SFO 337 740 621 187 BWI 1090 DFW 946 1121 MIA 2342 Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University http: //www. siit. tu. ac. th/bunyarit@siit. tu. ac. th 02 5013505 X 2005 4

Concrete example Imagine: 1. You wish to connect all the computers in an office building using the least amount of cable 5

Concrete example Imagine: 2. If you want to build a railway network to all provinces in Thailand with minimum cost, i. e. , minimum total length of the rail way. 6

7

Imagine: 3. Or a network of superhighway to connect all the cities of Bangkok with minimum cost 8

Concrete example Imagine: 4. We want to connect electricity to all of the houses in the whole village with minimum total of cable length 9

Minimum Spanning Tree Problem Each vertex in a graph G represents a computer, a province or a house Each edge represents the amount of cable, length of a railway needed to connect all vertices That‘s a weighted graph problem !! A problem of finding A Minimum Spanning Tree 10

Minimum Spanning Tree (MST) • Given a connected, undirected graph, a spanning tree of that graph is a subgraph which is a tree and connects all the vertices together. A single graph can have many different spanning trees. • We can assign a weight to a spanning tree by computing the sum of the weights of the edges in that spanning tree. • A minimum spanning tree or minimum weight spanning tree is then a spanning tree with weight less than or equal to the weight of every other spanning tree. ORD 1 DEN 9 PIT 6 7 3 DCA STL 4 8 DFW 10 5 2 ATL 11

We are interested in: Finding a tree T that contains all the vertices of a graph G (spanning tree) and has the least total weight over all. such trees is minimum-spanning tree (MST) 12

Minimum Spanning Tree 13

Properties Possible multiplicity There may be several minimum spanning trees of the same weight; in particular, if all weights are the same, every spanning tree is minimum. 14

Properties Uniqueness If each edge has a distinct weight then there will only be one, unique minimum spanning tree. Minimum-cost subgraph If the weights are non-negative, then a minimum spanning tree is in fact the minimum-cost subgraph connecting all vertices, since subgraphs containing cycles necessarily have more total weight. Cycle property For any cycle C in the graph, if the weight of an edge e of C is larger than the weights of other edges of C, then this edge cannot belong to a MST. 15

Cycle Property: Let T be a minimum spanning tree of a weighted graph G ¨ Let e be an edge of G that is not in T and C let be the cycle formed by e with T ¨ For every edge f of C, weight(f) weight(e) ¨ Proof: ¨ By contradiction ¨ If weight(f) > weight(e) we can get a spanning tree of smaller weight by replacing e with f 8 f 2 6 4 C 9 3 e 8 7 7 Replacing f with e yields a better spanning tree f 2 6 8 4 C 9 3 8 e 7 7 16

Partition Property U f Partition Property: Consider a partition of the vertices of G into subsets U and V ¨ Let e be an edge of minimum weight across the partition ¨ There is a minimum spanning tree of G containing edge e V 7 4 ¨ Proof: ¨ Let T be an MST of G ¨ If T does not contain e, consider the cycle C formed by e with T and let f be an edge of C across the partition ¨ By the cycle property, weight(f) weight(e) ¨ Thus, weight(f) = weight(e) ¨ We obtain another MST by replacing f with e 9 5 2 8 8 3 e 7 Replacing f with e yields another MST U 2 f V 7 4 9 5 8 8 e 3 7 17

2704 BOS 867 849 Topic 10. 2 ORD ITS 033 – Programming & Algorithms Prim’s Algorithm 1846 LAX 1391 1464 1235 JFK PVD 144 1258 184 802 SFO 337 740 621 187 BWI 1090 DFW 946 1121 MIA 2342 Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University http: //www. siit. tu. ac. th/bunyarit@siit. tu. ac. th 02 5013505 X 2005 18

Try to use exhaustive search ? n If we were to try an exhaustive-search approach to constructing a minimum spanning tree, we would face two serious obstacles. n 1) the number of spanning trees grows exponentially with the graph size (at least for dense graphs). n 2) generating all spanning trees for a given graph is not easy; 19

Greedy Approach n The greedy approach suggests constructing a solution through a sequence of steps, each expanding a partially constructed solution obtained so far, until a complete solution to the problem is reached. On each step—and this is the central point of this technique—the choice made must be n feasible, i. e. , it has to satisfy the problem’s constraints. locally optimal, i. e. , it has to be the best local choice among all feasible choices available on that step. irrevocable, i. e. , once made, it cannot be changed on subsequent steps of the algorithm. n n 20 Greedy Technique

Prim’s Algorithm n Prim’s algorithm constructs a minimum spanning tree through a sequence of expanding subtrees. n The initial subtree in such a sequence consists of a single vertex selected arbitrarily from the set V of the graph’s vertices. n On each iteration, we expand the current tree in the greedy manner by simply attaching to it the nearest vertex not in that tree. n The algorithm stops after all the graph’s vertices have been included in the tree being constructed. 21

Prim’s Algorithm 22

Prim’s Algorithm n The nature of Prim’s algorithm makes it necessary to provide each vertex not in the current tree with the information about the shortest edge connecting the vertex to a tree vertex. n We can provide such information by attaching two labels to a vertex: n (1) The name of the nearest tree vertex and the length (the weight) of the corresponding edge. n (2) Vertices that are not adjacent to any of the tree vertices canbe given the ∞ label indicating their “infinite” distance to the tree vertices and a null label for the name of the nearest tree vertex. 23

Prim’s Algorithm n We pick an arbitrary vertex s and we grow the MST as a cloud of vertices, starting from s We store with each vertex v a label d(v) = the smallest weight of an edge connecting v to a vertex in the cloud At each step: n We add to the cloud the vertex u outside the cloud with the smallest distance label n We update the labels of the vertices adjacent to u 24

Prim‘s Algorithm 1. All vertices are marked as not visited 2. Any vertex v you like is chosen as starting vertex and is marked as visited (define a cluster C) 3. 4. The smallest- weighted edge e = (v, u), which connects one vertex v inside the cluster C with another vertex u outside of C, is chosen and is added to the MST. The process is repeated until a spanning tree is formed 25

Example Step 1 2 2 7 B 8 C 5 8 0 A D 7 4 9 F 8 E 3 7 26

Example Step 2 2 2 B 5 C 5 8 0 A D 7 7 7 4 9 F 8 E 3 7 27

Example Step 3 2 2 7 B 5 C 5 8 0 A D 7 7 4 9 F 8 E 3 7 28

Example Step 4 2 2 B 5 C 5 8 0 A D 7 7 7 4 9 F 8 E 4 3 7 29

Example (contd. ) 2 2 B 5 C 5 8 0 A D 7 7 7 4 9 F 8 E 4 3 3 Step 5 30

Example (contd. ) Step 6 2 2 7 B 5 C 5 8 0 A D 7 7 4 9 F 8 E 4 3 3 31

Prim‘s Algorithm 5 A 4 B 6 2 C 2 3 D 1 3 E 2 4 F 32

Prim‘s Algorithm 5 A 4 B 6 2 C 2 3 D 1 3 E 2 4 F 33

Prim‘s Algorithm 5 A 4 B 6 2 C 2 3 D 1 3 E 2 4 F 34

Prim‘s Algorithm A B 2 2 C 3 D 1 3 E 2 4 F 35

Prim‘s Algorithm A B 2 2 C 1 3 E 3 D 2 F 36

Prim‘s Algorithm A B 2 2 C 1 3 E 3 D 2 F 37

Prim‘s Algorithm A B 2 2 C D 1 3 E 2 F 38

Prim‘s Algorithm A B 2 2 C D 1 3 E 2 F 39

Prim‘s Algorithm minimum- spanning tree A B 2 2 C D 1 3 E 2 F 40

Prim’s Algorithm Analysis n n 41 Efficiency of Prim’s algorithm depends on the data structures chosen for the graph and for the priority queue of the set V-VT whose vertex priorities are the distances to the nearest tree vertices. For example, if a graph is represented by its weight matrix and the priority queue is implemented as an unordered array, the algorithm’s running time will be in θ(|V |2). Greedy Technique

Prim’s Algorithm n A min-heap is a complete binary tree in which every element is less than or equal to its children. n Deletion of the smallest element from and insertion of a new element into a min-heap of size n are O(log n) operations, and so is the operation of changing an element’s priority n If a graph is represented by its adjacency linked lists and the priority queue is implemented as a min-heap, the running time of the algorithm is in O(|E| log |V |). n This is because the algorithm performs |V| - 1 deletions of the smallest element and makes |E| verifications and, possibly, changes of an element’s priority in a min-heap of size not greater than |V |. Each of these operations, as noted earlier, is a O(log |V |) operation. Hence, the running time of this implementation of Prim’s algorithm is in (|V| - 1+ |E|)O(log |V |) = O(|E| log |V |) 42 Greedy Technique

Prim’ Minimum Spanning Tree – Exercise 43

2704 BOS 867 849 Topic 10. 3 ORD ITS 033 – Programming & Algorithms Kruskal’s Algorithm 1846 LAX 1391 1464 1235 JFK PVD 144 1258 184 802 SFO 337 740 621 187 BWI 1090 DFW 946 1121 MIA 2342 Asst. Prof. Dr. Bunyarit Uyyanonvara IT Program, Image and Vision Computing Lab. School of Information, Computer and Communication Technology (ICT) Sirindhorn International Institute of Technology (SIIT) Thammasat University http: //www. siit. tu. ac. th/bunyarit@siit. tu. ac. th 02 5013505 X 2005 44

Minimum Spanning Tree Problem • Kruskal's algorithm is an algorithm in graph theory that finds a minimum spanning tree for a connected weighted graph. • If the graph is not connected, then it finds a minimum spanning forest (a minimum spanning tree for each connected component). • Kruskal's algorithm is also an example of a greedy algorithm. 45

Minimum Spanning Tree Problem It works as follows: 1. Each vertex is in its own cluster 2. Take the edge e with the smallest weight - if e connects two vertices in different clusters, then e is added to the MST and the two clusters, which are connected by e, are merged into a single cluster - if e connects two vertices, which are already in the same cluster, ignore it 3. Continue until n-1 edges were selected 46

9. 2 Kruskal’s Algorithm 47 Greedy Technique

Kruskal’s Algorithm Analysis n On each of its iterations, Kruskal’s algorithm has to check whether the addition of the next edge to the edges already selected would create a cycle. n A new cycle is created if and only if the new edge connects two vertices already connected by a path, i. e. , if and only if the two vertices belong to the same connected component n Each connected component of a subgraph generated by Kruskal’s algorithm is a tree because it has no cycles. 48 Greedy Technique

Kruskal’s Algorithm Analysis New edge connecting two vertices may (a) or may not (b) create a cycle 49 Greedy Technique

Kruskal‘s Algorithm 5 A 4 B 6 2 C 2 3 D 1 3 E 2 4 F 50

Kruskal‘s Algorithm 5 A 4 B 6 2 C 2 3 D 1 3 E 2 4 F 51

Kruskal‘s Algorithm 5 A 4 B 6 2 C 2 3 D 1 3 E 2 4 F 52

Kruskal‘s Algorithm 5 A 4 B 6 2 C 2 3 D 1 3 E 2 4 F 53

Kruskal‘s Algorithm 5 A 4 B 6 2 C 2 3 D 1 3 E 2 4 F 54

Kruskal‘s Algorithm 5 A 4 B 6 2 C 2 3 D 1 3 E cycle!! 2 4 F 55

Kruskal‘s Algorithm 5 A 4 B 6 2 C 2 3 D 1 3 E 2 4 F 56

Kruskal‘s Algorithm 5 A 4 B 6 2 C 2 3 D 1 3 E 2 4 F 57

Kruskal‘s Algorithm minimum- spanning tree A B 2 2 C D 1 3 E 2 F 58

Kruskal Example 2704 BOS 867 849 ORD 740 621 1846 337 LAX 1391 1464 1235 187 JFK 144 1258 184 802 SFO PVD BWI 1090 DFW 946 1121 MIA 2342 59

Example 60

Example 61

Example 62

Example 63

Example 64

Example 65

Example 66

Example 67

Example 68

Example 69

Example 70

Example 71

Example 2704 BOS 867 849 ORD 740 621 1846 337 LAX 1391 1464 1235 187 JFK 144 1258 184 802 SFO PVD BWI 1090 DFW 946 1121 MIA 2342 72

Kruskal’s Algorithm Analysis n The check whether two vertices belong to the same tree is crucial in determining running time of Kruskal’s algorithm n There is efficient algorithm that perform this check => union-find algorithm n With an efficient union- find algorithm, the running time of Kruskal’s algorithm will be dominated by the time needed for sorting the edge weights of a given graph. n Hence, with an efficient sorting algorithm, the time efficiency of Kruskal’s algorithm will be in O(|E| log |E|). 73 Greedy Technique

Data Structure for Kruskal Algortihm n n n The algorithm maintains a forest of trees An edge is accepted it if connects distinct trees We need a data structure that maintains a partition, i. e. , a collection of disjoint sets, with the operations: 74

Kruskal’s Minimum Spanning Tree 75

76

77

ITS 033 Midterm Topic 01 - Problems & Algorithmic Problem Solving Topic 02 – Algorithm Representation & Efficiency Analysis Topic 03 - State Space of a problem Topic 04 - Brute Force Algorithm Topic 05 - Divide and Conquer Topic 06 - Decrease and Conquer Topic 07 - Dynamics Programming Topic 08 - Transform and Conquer Topic 09 - Graph Algorithms Topic 10 - Minimum Spanning Tree Topic 11 - Shortest Path Problem Topic 12 - Coping with the Limitations of Algorithms Power http: //www. siit. tu. ac. th/bunyarit/its 033. php http: //www. vcharkarn. com/vlesson/7 78

End of Chapter 10 Thank you! 79