Data Mining Concepts and Techniques Chapter 9

Data Mining: Concepts and Techniques — Chapter 9 — Graph mining: Part I Graph Pattern Mining Jiawei Han and Micheline Kamber Department of Computer Science University of Illinois at Urbana-Champaign www. cs. uiuc. edu/~hanj © 2006 Jiawei Han and Micheline Kamber. All rights reserved. 1

18 March 2018 Mining and Searching Graphs in Graph Databases 2

Graph Mining n Graph Pattern Mining Frequent Subgraph Patterns n Impact on Graph Search I: Graph Indexing n n Impact on Graph Search II: Graph Similarity Search Constrained Graph Pattern Mining n Graph Classification n Graph Clustering n Summary 3

Why Graph Mining? n Graphs are ubiquitous n n Protein structures, biological pathways/networks (Bioinformactics) n Program control flow, traffic flow, and workflow analysis n n Chemical compounds (Cheminformatics) XML databases, Web, and social network analysis Graph is a general model n n Diversity of graphs n n Trees, lattices, sequences, and items are degenerated graphs Directed vs. undirected, labeled vs. unlabeled (edges & vertices), weighted, with angles & geometry (topological vs. 2 -D/3 -D) Complexity of algorithms: many problems are of high complexity 4

from H. Jeong et al Nature 411, 41 (2001) Graph, Everywhere Aspirin Internet Yeast protein interaction network Co-author network 5

Graph Pattern Mining n Frequent subgraphs n A (sub)graph is frequent if its support (occurrence frequency) in a given dataset is no less than a minimum support threshold n Applications of graph pattern mining n Mining biochemical structures n Program control flow analysis n Mining XML structures or Web communities n Building blocks for graph classification, clustering, compression, comparison, and correlation analysis 6

Example: Frequent Subgraphs GRAPH DATASET (A) (B) (C) FREQUENT PATTERNS (MIN SUPPORT IS 2) (1) (2) 7

EXAMPLE (II) GRAPH DATASET FREQUENT PATTERNS (MIN SUPPORT IS 2) 8

Graph Mining Algorithms n Incomplete beam search – Greedy (Subdue) n Inductive logic programming (WARMR) n Graph theory-based approaches n Apriori-based approach n Pattern-growth approach 9

SUBDUE (Holder et al. KDD’ 94) n Start with single vertices n Expand best substructures with a new edge n Limit the number of best substructures n n Substructures are evaluated based on their ability to compress input graphs Using minimum description length (DL) Best substructure S in graph G minimizes: DL(S) + DL(GS) Terminate until no new substructure is discovered 10

WARMR (Dehaspe et al. KDD’ 98) n Graphs are represented by Datalog facts n atomel(C, A 1, c), bond (C, A 1, A 2, BT), atomel(C, A 2, c) : a carbon atom bound to a carbon atom with bond type BT n WARMR: the first general purpose ILP system n Level-wise search n Simulate Apriori for frequent pattern discovery 11

Frequent Subgraph Mining Approaches n n Apriori-based approach n AGM/Ac. GM: Inokuchi, et al. (PKDD’ 00) n FSG: Kuramochi and Karypis (ICDM’ 01) # n PATH : Vanetik and Gudes (ICDM’ 02, ICDM’ 04) n FFSM: Huan, et al. (ICDM’ 03) Pattern growth approach n Mo. Fa, Borgelt and Berthold (ICDM’ 02) n g. Span: Yan and Han (ICDM’ 02) n Gaston: Nijssen and Kok (KDD’ 04) 12

Properties of Graph Mining Algorithms n n n Search order n breadth vs. depth Generation of candidate subgraphs n apriori vs. pattern growth Elimination of duplicate subgraphs n passive vs. active Support calculation n embedding store or not Discover order of patterns n path tree graph 13

Apriori-Based Approach k-edge (k+1)-edge G 1 G G 2 G’ … G’’ Gn JOIN 14

Apriori-Based, Breadth-First Search n n n Methodology: breadth-search, joining two graphs AGM (Inokuchi, et al. PKDD’ 00) n generates new graphs with one more node FSG (Kuramochi and Karypis ICDM’ 01) n generates new graphs with one more edge 15

PATH (Vanetik and Gudes ICDM’ 02, ’ 04) n n Apriori-based approach Building blocks: edge-disjoint path A graph with 3 edge-disjoint paths • construct frequent paths • construct frequent graphs with 2 edge-disjoint paths • construct graphs with k+1 edge-disjoint paths from graphs with k edge-disjoint paths • repeat 16

FFSM (Huan, et al. ICDM’ 03) n n n Represent graphs using canonical adjacency matrix (CAM) Join two CAMs or extend a CAM to generate a new graph Store the embeddings of CAMs n All of the embeddings of a pattern in the database n Can derive the embeddings of newly generated CAMs 17

Pattern Growth Method (k+2)-edge (k+1)-edge G 1 k-edge G … duplicate graph G 2 … Gn … 18

Mo. Fa (Borgelt and Berthold ICDM’ 02) n Extend graphs by adding a new edge n Store embeddings of discovered frequent graphs n n Fast support calculation Also used in other later developed algorithms such as FFSM and GASTON Expensive Memory usage Local structural pruning n n 19

GSPAN (Yan and Han ICDM’ 02) Right-Most Extension Theorem: Completeness The Enumeration of Graphs using Right-most Extension is COMPLETE 20

DFS Code n Flatten a graph into a sequence using depth first search e 0: (0, 1) 0 e 1: (1, 2) 1 e 2: (2, 0) 2 3 4 e 3: (2, 3) e 4: (3, 1) e 5: (2, 4) 21

DFS Lexicographic Order n Let Z be the set of DFS codes of all graphs. Two DFS codes a and b have the relation a<=b (DFS Lexicographic Order in Z) if and only if one of the following conditions is true. Let a = (x 0, x 1, …, xn) and b = (y 0, y 1, …, yn), (i) (ii) if there exists t, 0<= t <= min(m, n), xk=yk for all k, s. t. k

DFS Code Extension n Let a be the minimum DFS code of a graph G and b be a non-minimum DFS code of G. For any DFS code d generated from b by one right-most extension, (i) (iii) d is not a minimum DFS code, min_dfs(d) cannot be extended from b, and min_dfs(d) is either less than a or can be extended from a. THEOREM [ RIGHT-EXTENSION ] The DFS code of a graph extended from a Non-minimum DFS code is NOT MINIMUM 23

GASTON (Nijssen and Kok KDD’ 04) n Extend graphs directly n Store embeddings n Separate the discovery of different types of graphs n n path tree graph Simple structures are easier to mine and duplication detection is much simpler 24

Graph Pattern Explosion Problem n If a graph is frequent, all of its subgraphs are frequent ─ the Apriori property n An n-edge frequent graph may have 2 n subgraphs n Among 422 chemical compounds which are confirmed to be active in an AIDS antiviral screen dataset, there are 1, 000 frequent graph patterns if the minimum support is 5% 25

Closed Frequent Graphs n n Motivation: Handling graph pattern explosion problem Closed frequent graph n A frequent graph G is closed if there exists no supergraph of G that carries the same support as G If some of G’s subgraphs have the same support, it is unnecessary to output these subgraphs (nonclosed graphs) Lossless compression: still ensures that the mining result is complete 26

CLOSEGRAPH (Yan & Han, KDD’ 03) A Pattern-Growth Approach (k+1)-edge G 1 k-edge G G 2 … Gn At what condition, can we stop searching their children i. e. , early termination? If G and G’ are frequent, G is a subgraph of G’. If in any part of the graph in the dataset where G occurs, G’ also occurs, then we need not grow G, since none of G’s children will be closed except those of G’. 27

Handling Tricky Exception Cases a a b a c d (graph 1) c b d (graph 2) b (pattern 1) a c d (pattern 2) 28

Experimental Result n The AIDS antiviral screen compound dataset from NCI/NIH n The dataset contains 43, 905 chemical compounds n Among these 43, 905 compounds, 423 of them belongs to CA, 1081 are of CM, and the remaining are in class CI 29

Discovered Patterns 20% 10% 5% 30

Run time per pattern (msec) Performance (1): Run Time Minimum support (in %) 31

Memory usage (GB) Performance (2): Memory Usage Minimum support (in %) 32

Number of Patterns: Frequent vs. Closed Number of patterns CA Minimum support 33

Runtime: Frequent vs. Closed Run time (sec) CA Minimum support 34

Do the Odds Beat the Curse of Complexity? n n Potentially exponential number of frequent patterns n The worst case complexty vs. the expected probability 4 n Ex. : Suppose Walmart has 10 kinds of products -4 n The chance to pick up one product 10 -40 n The chance to pick up a particular set of 10 products: 10 n What is the chance this particular set of 10 products to be frequent 103 times in 109 transactions? Have we solved the NP-hard problem of subgraph isomorphism testing? n No. But the real graphs in bio/chemistry is not so bad n A carbon has only 4 bounds and most proteins in a network have distinct labels 35

Graph Mining n Graph Pattern Mining Frequent Subgraph Patterns n Impact on Graph Search I: Graph Indexing n n Impact on Graph Search II: Graph Similarity Search Constrained Graph Pattern Mining n Graph Classification n Graph Clustering n Summary 36

Graph Search n Querying graph databases: n Given a graph database and a query graph, find all the graphs containing this query graph database 37

Scalability Issue n Sequential scan n Disk I/Os Subgraph isomorphism testing An indexing mechanism is needed n Day. Light: Daylight. com (commercial) n Graph. Grep: Dennis Shasha, et al. PODS'02 n Grace: Srinath Srinivasa, et al. ICDE'03 38

Indexing Strategy Query graph (Q) Graph (G) If graph G contains query graph Q, G should contain any substructure of Q Substructure Remarks n Index substructures of a query graph to prune graphs that do not contain these substructures 39

Indexing Framework n Two steps in processing graph queries Step 1. Index Construction n Enumerate structures in the graph database, build an inverted index between structures and graphs Step 2. Query Processing n Enumerate structures in the query graph n Calculate the candidate graphs containing these structures n Prune the false positive answers by performing subgraph isomorphism test 40

Cost Analysis QUERY RESPONSE TIME fetch index number of candidates REMARK: make |Cq| as small as possible 41

Path-based Approach GRAPH DATABASE (a) (b) (c) PATHS 0 -length: C, O, N, S 1 -length: C-C, C-O, C-N, C-S, N-N, S-O 2 -length: C-C-C, C-O-C, C-N-C, . . . 3 -length: . . . Built an inverted index between paths and graphs 42

Path-based Approach (cont. ) QUERY GRAPH 0 -edge: SC={a, b, c}, SN={a, b, c} 1 -edge: SC-C={a, b, c}, SC-N={a, b, c} 2 -edge: SC-N-C = {a, b}, … … Intersect these sets, we obtain the candidate answers - graph (a) and graph (b) - which may contain this query graph. 43

Problems: Path-based Approach GRAPH DATABASE (a) (b) (c) QUERY GRAPH Only graph (c) contains this query graph. However, if we only index paths: C, C-C-C, C-C-C-C, we cannot prune graph (a) and (b). 44

g. Index: Indexing Graphs by Data Mining n Our methodology on graph index: n n n Identify frequent structures in the database, the frequent structures are subgraphs that appear quite often in the graph database Prune redundant frequent structures to maintain a small set of discriminative structures Create an inverted index between discriminative frequent structures and graphs in the database 45

IDEAS: Indexing with Two Constraints discriminative (~103) frequent (~105) structure (>106) 46

Why Discriminative Subgraphs? Sample database (a) n n (b) (c) All graphs contain structures: C, C-C-C Why bother indexing these redundant frequent structures? n Only index structures that provide more information than existing structures 47

Discriminative Structures n n Pinpoint the most useful frequent structures n Given a set of structures and a new structure , we measure the extra indexing power provided by , When is small enough, is a discriminative structure and should be included in the index Index discriminative frequent structures only n Reduce the index size by an order of magnitude 48

Why Frequent Structures? n n We cannot index (or even search) all of substructures Large structures will likely be indexed well by their substructures Size-increasing support threshold minimum support threshold support n size 49

Experimental Setting n The AIDS antiviral screen compound dataset from NCI/NIH, containing 43, 905 chemical compounds n Query graphs are randomly extracted from the dataset n Graph. Grep: maximum length (edges) of paths is set at 10 n g. Index: maximum size (edges) of structures is set at 10 50

# OF FEATURES Experiments: Index Size DATABASE SIZE 51

# OF CANDIDATES Experiments: Answer Set Size QUERY SIZE 52

Experiments: Incremental Maintenance Frequent structures are stable to database updating Index can be built based on a small portion of a graph database, but be used for the whole database

Alternative Graph Indexing Methods n n Graph-structure-based indexing and similarity search n Structure-based index methods, e. g. , g-Index, S-path index n Use index to search for similar graph/network structures Substructure indexing n Key problem: What substructures as indexing features? n g. Index [Yan, Yu & Han, SIGMOD’ 04]: Find frequent and discriminative subgraphs (by graph-pattern mining) n S-path [Zhao & Han, VLDB’ 10]: Use decomposed shortest paths as basic indexing features 54

Why S-Path as Indexing Features? n n Neighborhood signatures of vertices are built to maintain indexing features: Effective search space pruning ability Processing (Query Decomposition): Decompose the query graph into a set of indexed shortest paths in S-Path Query Network A global lookup table Neighborhood signature of v 3

Graph Mining n Graph Pattern Mining Frequent Subgraph Patterns n Impact on Graph Search I: Graph Indexing n n Impact on Graph Search II: Graph Similarity Search Constrained Graph Pattern Mining n Graph Classification n Graph Clustering n Summary 56

Structure Similarity Search • CHEMICAL COMPOUNDS (a) caffeine (b) diurobromine (c) viagra • QUERY GRAPH 57

Some “Straightforward” Methods n Method 1: Directly compute the similarity between the graphs in the DB and the query graph n n n Sequential scan Subgraph similarity computation Method 2: Form a set of subgraph queries from the original query graph and use the exact subgraph search n Costly: If we allow 3 edges to be missed in a 20 -edge query graph, it may generate 1, 140 subgraphs 58

Index: Precise vs. Approximate Search n Precise Search n n Use frequent patterns as indexing features Select features in the database space based on their selectivity Build the index Approximate Search n n Hard to build indices covering similar subgraphs— explosive number of subgraphs in databases Idea: (1) keep the index structure (2) select features in the query space 59

Substructure Similarity Measure n Query relaxation measure n The number of edges that can be relabeled or missed; but the position of these edges are not fixed QUERY GRAPH … 60

Substructure Similarity Measure n Feature-based similarity measure n n n Each graph is represented as a feature vector X = {x 1, x 2, …, xn} Similarity is defined by the distance of their corresponding vectors Advantages n Easy to index n Fast n Rough measure 61

Intuition: Feature-Based Similarity Search Graph (G 1) Query (Q) Ø If graph G contains the major part of a query graph Q, G should share a number of common features with Q Graph (G 2) Substructure Ø Given a relaxation ratio, calculate the maximal number of features that can be missed ! At least one of them should be contained 62

Feature-Graph Matrix graphs in database G 2 G 3 G 4 G 5 f 1 0 1 1 f 2 0 1 0 0 1 f 3 1 0 1 1 1 f 4 1 0 0 0 1 f 5 features G 1 0 0 1 1 0 Assume a query graph has 5 features and at most 2 features to miss due to the relaxation threshold 63

Edge Relaxation—Feature Misses n If we allow k edges to be relaxed, J is the maximum number of features to be hit by k edges—it becomes the maximum coverage problem n NP-complete n A greedy algorithm exists n We design a heuristic to refine the bound of feature misses 64

Query Processing Framework n Three steps in processing approximate graph queries Step 1. Index Construction n Select small structures as features in a graph database, and build the featuregraph matrix between the features and the graphs in the database 65

Framework (cont. ) Step 2. Feature Miss Estimation n Determine the indexed features belonging to the query graph n Calculate the upper bound of the number of features that can be missed for an approximate matching, denoted by J n On the query graph, not the graph database 66

Framework (cont. ) Step 3. Query Processing n n Use the feature-graph matrix to calculate the difference in the number of features between graph G and query Q, FG – FQ If FG – FQ > J, discard G. The remaining graphs constitute a candidate answer set 67

Performance Study n n Database n Chemical compounds of Anti-Aids Drug from NCI/NIH, randomly select 10, 000 compounds Query n Randomly select 30 graphs with 16 and 20 edges as query graphs n Competitive algorithms n Grafil: Graph Filter—our algorithm n Edge: use edges only n All: use all the features 68

# of candidates Comparison of the Three Algorithms edge relaxation 69

Summary: Graph Pattern Mining n Graph mining has wide applications n Frequent and closed subgraph mining methods n n Graph indexing techniques n n Frequent and discriminative subgraphs are high-quality indexing features Similarity search in graph databases n n g. Span and Close. Graph: pattern-growth depth-first search approach Indexing and feature-based matching Constraint-based graph pattern mining

References (1) n n n n T. Asai, et al. “Efficient substructure discovery from large semi-structured data”, SDM'02 C. Borgelt and M. R. Berthold, “Mining molecular fragments: Finding relevant substructures of molecules”, ICDM'02 M. Deshpande, M. Kuramochi, and G. Karypis, “Frequent Sub-structure Based Approaches for Classifying Chemical Compounds”, ICDM 2003 M. Deshpande, M. Kuramochi, and G. Karypis. “Automated approaches for classifying structures”, BIOKDD'02 L. Dehaspe, H. Toivonen, and R. King. “Finding frequent substructures in chemical compounds”, KDD'98 C. Faloutsos, K. Mc. Curley, and A. Tomkins, “Fast Discovery of 'Connection Subgraphs”, KDD'04 L. Holder, D. Cook, and S. Djoko. “Substructure discovery in the subdue system”, KDD'94 J. Huan, W. Wang, D. Bandyopadhyay, J. Snoeyink, J. Prins, and A. Tropsha. “Mining spatial motifs from protein structure graphs”, RECOMB’ 04 J. Huan, W. Wang, and J. Prins. “Efficient mining of frequent subgraph in the presence of isomorphism”, ICDM'03 H. Hu, X. Yan, Yu, J. Han and X. J. Zhou, “Mining Coherent Dense Subgraphs across Massive Biological Networks for Functional Discovery”, ISMB'05 A. Inokuchi, T. Washio, and H. Motoda. “An apriori-based algorithm for mining frequent substructures from graph data”, PKDD'00 C. James, D. Weininger, and J. Delany. “Daylight Theory Manual Daylight Version 4. 82”. Daylight Chemical Information Systems, Inc. , 2003. G. Jeh, and J. Widom, “Mining the Space of Graph Properties”, KDD'04 M. Koyuturk, A. Grama, and W. Szpankowski. “An efficient algorithm for detecting frequent subgraphs in biological networks”, Bioinformatics, 20: I 200 --I 207, 2004.

References (2) n n n n n M. Kuramochi and G. Karypis. “Frequent subgraph discovery”, ICDM'01 M. Kuramochi and G. Karypis, “GREW: A Scalable Frequent Subgraph Discovery Algorithm”, ICDM’ 04 B. Mc. Kay. Practical graph isomorphism. Congressus Numerantium, 30: 45 --87, 1981. S. Nijssen and J. Kok. A quickstart in frequent structure mining can make a difference. KDD'04 J. Prins, J. Yang, J. Huan, and W. Wang. “Spin: Mining maximal frequent subgraphs from graph databases”. KDD'04 D. Shasha, J. T. -L. Wang, and R. Giugno. “Algorithmics and applications of tree and graph searching”, PODS'02 J. R. Ullmann. “An algorithm for subgraph isomorphism”, J. ACM, 23: 31 --42, 1976. N. Vanetik, E. Gudes, and S. E. Shimony. “Computing frequent graph patterns from semistructured data”, ICDM'02 C. Wang, W. Wang, J. Pei, Y. Zhu, and B. Shi. “Scalable mining of large disk-base graph databases”, KDD'04 T. Washio and H. Motoda, “State of the art of graph-based data mining”, SIGKDD Explorations, 5: 59 -68, 2003 X. Yan and J. Han, “g. Span: Graph-Based Substructure Pattern Mining”, ICDM'02 X. Yan and J. Han, “Close. Graph: Mining Closed Frequent Graph Patterns”, KDD'03 X. Yan, P. S. Yu, and J. Han, “Graph Indexing: A Frequent Structure-based Approach”, SIGMOD'04 X. Yan, X. J. Zhou, and J. Han, “Mining Closed Relational Graphs with Connectivity Constraints”, KDD'05 X. Yan, P. S. Yu, and J. Han, “Substructure Similarity Search in Graph Databases”, SIGMOD'05 X. Yan, F. Zhu, J. Han, and P. S. Yu, “Searching Substructures with Superimposed Distance”, ICDE'06 M. J. Zaki. “Efficiently mining frequent trees in a forest”, KDD'02 P. Zhao and J. Han, “On Graph Query Optimization in Large Networks", VLDB'10

18 March 2018 Mining and Searching Graphs in Graph Databases 85