연관 규칙 탐사와 그 응용 성신여자대학교 전산학과 박

연관 규칙 탐사와 그 응용 성신여자대학교 전산학과 박 종수 jpark@cs. sungshin. ac. kr 연관규칙탐사, 박종수 1

차례 n Data Mining in the KDD Process n Association Rule의 정의 n Mining Association Rules in Transaction Databases n Algorithm Apriori & DHP n Generalized Association Rules n Cyclic Association Rules and Negative Associations. n Interestingness Measurement n Sequential Patterns and Path Traversal Patterns n 연구 방향 및 참고 Homepages 연관규칙탐사, 박종수 2

Overview of the steps constituting the KDD process Data Selection Target Data Preprocessing Preprocessed Data Transformation Transformed Data Mining Patterns Interpretation/ Evaluation Knowledge 연관규칙탐사, 박종수 3

Types of Data-Mining Problems n Prediction – – – Classification Regression Time Series n Knowledge Discovery – – – – 연관규칙탐사, 박종수 Deviation Detection Database Segmentation Clustering Association Rules Summarization Visualization Text mining 4

Association Rule Ex: the statement that 90% of transactions that purchase bread and butter also purchase milk. [Bread], [Butter] [Milk] (12. 5%, 90%) antecedent consequent 90% : confidence factor of the rule (not 100%) 12. 5%: support for the rule, the fraction of transactions in database Find all rules that have “Diet Coke” as consequent. Find all rules that have “bagels” in the antecedent. Find the “best” k rules that have “bagels” in the consequent. 연관규칙탐사, 박종수 5

연관규칙의 정의 n I : a set of literals called items. T: a set of items such that T I, transaction. l An association rule is an implication of the form X Y, where X I, Y I and X Y = ø. l X Y [support, confidence] 연관규칙탐사, 박종수 6

Transaction Databases에서 연관 규칙 탐사 n Applications: pattern association, market analysis, etc n Given · · n data of transactions each transaction has a list of items purchased Find all association rules: the presence of one set of rules items implies the presence of another set of items. - e. g. , people who purchased hammers also purchased nails n Measurement of rule strength · Confidence: X & Y Z has 90% confidence if 90% of Confidence customers who bought X and Y also bought Z. · Support: useful rules(for business decision) should have some Support minimum transaction support. 연관규칙탐사, 박종수 7

Two Steps for Association Rules n Determining “large itemsets” · · n Find all combinations of items that have transaction support above minimum support Researches have been focussed on this phase. Generating rules for each large itemset L do for each subset c of L do if (support(L) / support(L - c) minimum confidence) then confidence output the rule (L - c) c, with confidence = support(L)/support(L - c) and support = support(L); 연관규칙탐사, 박종수 8

Focus on data structures to speed up scanning the database Hash tree, Trie, Hash table, etc. Candidate Itemsets minimum support Large Itemsets Scan Database How to generate candidate itemsets Apriori method: join step + prune step minimum confidence Association Rules 연관규칙탐사, 박종수 9

C 1 Database D TID 100 200 300 400 Items ACD BCE ABCE BE Itemset {A} {B} {C} {D} {E} Scan D L 1 Sup. 2 3 3 1 3 Itemset {A} {B} {C} {E} Sup. 2 3 3 3 minimum support = 2 C 2 Itemset {A B} {A C} {A E} {B C} {B E} {C E} C 3 Itemset {B C E} 연관규칙탐사, 박종수 Scan D Itemset {A B} {A C} {A E} {B C} {B E} {C E} L 2 Sup. 1 2 3 2 Itemset {A C} {B E} {C E} C 3 Itemset Sup. {B C E} 2 L 3 Itemset {B C E} Sup. 2 2 3 2 Sup. 2 10

Algorithms for Mining Association Rules n AIS(Agrawal et al. , ACM SIGMOD, May ‘ 93) ‘ 93 n SETM(Swami et al. , IBM Tech. Rep. , Oct ‘ 93) n Apriori(Agrawal et al. , VLDB, Sept ‘ 94) Apriori n OCD(Mannila et al. , AAAI workshop on KDD, July, ‘ 94) n DHP(Park et al. , ACM SIGMOD, May ‘ 95) DHP n PARTITION(Savasere et al. , VLDB, Sept ‘ 95) n Mining Generalized Association Rules(Srikant et al. , VLDB, Sept ‘ 95) n Sampling Approach(Toivonen, VLDB, Sept ‘ 96) n DIC(dynamic itemset counting, Brin et al. , ACM SIGMOD, May ‘ 97) DIC ‘ 97 n Cyclic Association Rules( zden et al. , IEEE ICDE, Feb ‘ 98) n Negative Associations(Savasere et al. , IEEE ICDE, Feb ‘ 98) 연관규칙탐사, 박종수 11

Algorithm Apriori n n Lk: Set of Large k-itemsets Ck: Set of Candidate k-itemsets Step; C 1 L 1 C 2 L 2, . . . , Ck Lk Input File: Transaction File, Output: Large itemsets L 1 = {large 1 -itemset} for ( k=2; Lk-1 Ø; k++) do begin Ck= apriori-gen(Lk-1); forall transactions t D do begin Ct = subset(Ck, t); forall candidates c Ct do c. count++; end Lk= {c Ck| c. count minsup} end Answer = Uk Lk; 연관규칙탐사, 박종수 12

Apriori-gen(Lk-1) n Join step insert into Ck select p. item 1, p. item 2, . . . , p. itemk-1, q. itemk-1 from Lk-1 p, Lk-1 q where p. item 1= q. item 1, . . . , p. itemk-2= q. itemk-2, p. itemk-1< q. itemk-1 n Prune step forall itemsets c Ck do forall (k-1)-subsets s of c do if ( s Lk-1 ) then delete c from Ck; 연관규칙탐사, 박종수 13

Ex: Generation of Candidate Itemsets n 예: L 3로부터 C 4 를 생성하는 과정. Ê Join step L 3 = {{1, 2 , 3}, {1, 2, 4}, {1, 3, 5}, {2, 3, 4}}일때, 후보 4 -항목집합 = { {1 2 3 4}, {1 3 4 5}} · Prune step: - {1, 2, 3, 4} 의 3 -subset = {{1, 2, 3}, {1, 2, 4}, {1, 3, 4}, {2, 3, 4}} - {1, 3, 4, 5} 의 3 -subset = {{1, 3, 4}, {1, 3, 5}, {1, 4, 5}, {3, 4, 5}} + 각 {1, 4, 5}, {3, 4, 5} L 3이므로 {1, 3, 4, 5} 는 pruning!! C 4 = {{1, 2, 3, 4}} 연관규칙탐사, 박종수 14

Data Structure for Ck n n 각 레벨의 후보집합에 대해 Hash Tree 형성. 예: C 2 = {{A, B}, {A, C}, {A, T} {B, C}, {B, D}, {C, D}}의 Hash Tree C 2 Level 1 Level 2 A B B C C C D C, D 중간노드 A, B 연관규칙탐사, 박종수 A, T A, C B, D 잎노드 15

Hash Table H 2 와 후보 2 -항목집합 C 2 를 생성하는 예 (DHP) DHP 연관규칙탐사, 박종수 16

Counting support in a hash tree TID Items 100 200 300 400 ACD BCE ABCE BE {A C} {B E} {C E} {B E} Discard Keep {B C E} Discard D 3 = { <200, B C E>, <300, B C E> } C 2 count {A C} 2 {B E} 3 {C E} 2 s=2 L 2 {A C} {B E} {C E} L 2와 D 3의 예 (DHP) 연관규칙탐사, 박종수 17

Generalized Association Rules n n Finding associations between items at any level of the taxonomy. Rules: · · · People who buy clothes tend to buy shoes. ( ) People who buy outerwear tend to buy shoes. ( o ) People who buy jacket tend to buy shoes. ( ) Clothes Outerwear Jackets 연관규칙탐사, 박종수 Footwear Shirts Shoes Hiking Boots Ski Pants 18

Problem Statement n I = { i 1, i 2, …, im}: set of literals, D: set of transactions, T: a set of taxonomy, DAG(Directed Acyclic Graph) 일때, X Y [confidence, support], where X I, Y I, X Y = , and no item in Y is an ancestor of any item in X. (X, Y: any level of taxonomy T ) n Step 1. Find all sets of items whose support is greater than minimum support. 2. Generate association rules, whose confidence is greater than minimum confidence. 3. Prune all uninteresting rules from this set with respect to the Rinteresting. 연관규칙탐사, 박종수 19

Interestingness of Generalized Rules n Using new interest measure, R-interesting: Prune out 40% to 60% of the rules as “redundant “ rules. n Example: * 가정: Taxonomy: Skim milk is-a Milk, Milk Cereal ( 8% support, 70% confidence), Skim milk의 판매량 = milk판매량의 1/4 일 때, * Skim milk Cereal 에 대해, n Expectation: 2% support, 70% confidence n Actual support & confidence: 약 2% support, 70% confidence ==> redundant & uninteresting!! 연관규칙탐사, 박종수 20

Cyclic Association Rules n n n Beer and chips are sold together primarily between 6 PM and 9 PM. Association rules could also display regular hourly, daily, weekly, etc. , variation that has the appearance of cycles. An association rule X Y holds in time unit ti, – – – n if the support of X Y in D[i] exceeds Min. Sup and the confidence of X Y in D[i] exceeds Min. Conf. It has a cycle c = (l, o), a length l and an offset o. “coffee doughnuts” has a cycle (24, 7), – if the unit of time is an hour and “coffee doughnuts” holds during the interval 7 AM-8 AM everyday (I. e. , every 24 hours). 연관규칙탐사, 박종수 21

Negative Association Rules n n A rule : “ 60% of the customers who buy potato chips do not buy bottled water. ” Negative rule: X Y such that – – n (a) support(X) and support(Y) are greater than minimum support Min. Sup; and (b) the rule interest measure is greater than Min. RI. The interest measure RI of a negative association rule, X Y, – E[support(X)] is the expected support of an itemset X. 연관규칙탐사, 박종수 22

Incremental Updating, Updating Parallel and Distributed Algorithms n 데이타베이스 연관규칙 탐사를 위한 점진적 평가기법. (김의경등, 한국정보과학회 ‘ 95 가을 학술 발표 논문지) n Fast updating algorithms, FUP (Cheung et al. , IEEE ICDE, ‘ 96). · n PDM (Park et al. , ACM CIKM, ‘ 95): · n Partitioned derivation and incremental updating. Use a hashing technique(DHP-like) to identify candidate k-itemsets from the local databases. Count Distribution (Agrawal & Shafer, IEEE TKDE, Vol 8, No 6, ‘ 96): · · n An extension of the Apriori algorithm. May require a lot of messages in count exchange. FDM(Cheung et al. , IEEE TKDE, Vol 8, No 6, ‘ 96). · · Observation: If an itemset X is globally large, there exists a partition Di such that X and all its subsets are locally large at Di. Candidate set are those which are also local candidates in some component database, plus some message passing optimizations. 연관규칙탐사, 박종수 23

When is Market Basket Analysis useful? n The following three rules are examples of real rules generated from real data: – On Thursdays, grocery store consumers often purchase diapers and beer together. 4 Useful – Customers who purchases maintenance agreements are very likely to purchase large appliances. 4 Trivial – rule: high quality, actionable information. rule When a new hardware store opens, one of the most commonly sold items is toilet rings. 4 Inexplicable 연관규칙탐사, 박종수 rule 24

Interestingness Measurement for Association Rules (I) n Two popular measurements: support and confidence · n Use taxonomy information for pruning redundant rules · · · n The longer (itemset), the fewer (support). A rule is “redundant” if its support and confidence are close to their redundant expected values based on an ancestor of the rule. Example: ”milk cereal” vs. “skim milk cereal”. More effective than that based on statistical significance. Interestingness of Patterns · · If a pattern contradicts the set of hard beliefs of the user, then this user pattern is always interesting to the user. The more a pattern “affects” the belief system, the more interesting it is. 연관규칙탐사, 박종수 25

Interestingness Measurement (II) n Improvement (Interest ) – – – n How much better a rule is at predicting the result than just assuming the result in the first place. Co-occurrence than implication. Symmetric. Conviction – How far ”condition and result” deviates from independence 연관규칙탐사, 박종수 26

Range of measurement n Improvement – – n Improvement = 1: ç condition과 result의 item이 completely independent! Improvement < 1: ç worse rule! Improvement > 1: ç better rule! Conviction – – – Conviction = 1: ç condition과 result의 item이 completely unrelated. Conviction > 1: ç better rule!! Conviction = : ç completely related rule 연관규칙탐사, 박종수 27

Sequential Patterns n Examples of such a pattern: – Customers typically rent “Star Wars”, then “Empire Strikes Back”, and then “Return of the jedi”. – Note that these rentals need not to be consecutive. – 수강신청: 관광과 여가(1학기) 수도권과 주택문제(2학기) 증권시장(3학기) – 주가 변동 패턴: 삼성전자 주가 상승 LG전자 주가 상승 보해양조 주가 상승 – 구매패턴: 양복 와이셔츠 검정색 구두 ? – 의료진단에서 질병 발생 순서 패턴 – 환자 치료에서 진료 및 투약 패턴 연관규칙탐사, 박종수 28

Mining Sequential Patterns n n An itemset is a non-empty set of items. A sequence is an ordered list of itemsets. Customer Id 1 2 3 4 5 Customer Sequence <(30) (90)> <(10 20) (30) (40 60 70)> <(30 50 70)> <(30) (40 70) (90)> <(90)> Sequential Patterns with support > 25% <(30) (90)> <(30) (40 70)> 연관규칙탐사, 박종수 29

The Algorithm for Sequential Patterns by Agrawal and Srikant, 1995 ICDE n Sort Phase – n Litemset Phase – n A customer sequence is represented by a list of sets of litemsets Sequence Phase ( Apriori 알고리즘의 응용) – n litemset = an itemset with minimum support Transformation Phase – n major key: customer-id, minor key: transaction-time Candidate sequences ==> Large sequences Maximal Phase – 연관규칙탐사, 박종수 a sequence s is maximal if s is not contained in any other sequence 30

Mining Path Traversal Patterns n Understanding user access patterns in a distributed information providing environment such as WWW, Hitel, etc. – – n help improving the system design lead to better marketing decisions Capturing user access patterns – mining path traversal patterns – capturing user traveling behavior – improving the quality of such services 연관규칙탐사, 박종수 31

Traversal patterns 1 2 3 C B A 12 13 6 O 15 14 5 E 11 7 U V 4 D 8 H G 9 10 Maximal forward references {ABCD, ABEGH, ABEGW, AOU, AOV} W 1. Find large reference sequences. 2. Find maximal reference sequences. 연관규칙탐사, 박종수 32

연구 방향 n 연관 규칙 탐사 – – n Sampling approach, parallel method, distributed algorithm등 의 연구 Candidate itemsets을 효율적으로 관리하고 scanning에 효과 적인 자료구조 연구 규칙의 흥미도 또는 중요도 측정 연관 규칙의 응용으로 구체적인 적용 방법. Other patterns – pattern의 정의와 적용에 관한 문제 연구 – Similarity search – WWW에서 path traversal patterns등의 연구 연관규칙탐사, 박종수 33

Some Data Mining Systems and Homepages • Quest (IBM Almaden: Agrawal, et al. ): Agrawal – large DB-oriented association, classification, sequential patterns, similar sequences, etc. – “http: //www. almaden. ibm. com/cs/quest/” • DBMiner: (SFC: Han, et al. ): DBMiner Han – Interactive, multi-level characterization, classification, association & prediction. – “http: //db. cs. sfu. ca/DBMiner/” • KDD (GTE: Piatetsky-Shapiro, et al. ): Shapiro – multi-strategy, strong rules, statistical approaches, etc. – KD Mine: “http: //info. gte. com/~kdd/index. html” • Other Homepages for Data Mining – – – Rakesh Agrawal: “http: //www. almaden. ibm. com/cs/people/ragrawal/” Usama Fayyad: “http: //www. research. microsoft. com/~fayyad/” Heikki Mannila: “http: //www. cs. Helsinki. Fl/~mannila/” Jiawei Han: “http: //fas. sfu. ca/cs/people/Faculty/Han/” Data Mining and Knowledge Discovery Journal: Journal “http: //www. research. microsoft. com/research/datamine/”의 Editorial Board 연관규칙탐사, 박종수 34