Скачать презентацию Sequential Pattern Mining CS 685 Special Topics in
921ac408e7ba7f85d894eff69172f47c.ppt
- Количество слайдов: 23
Sequential Pattern Mining CS 685: Special Topics in Data Mining The UNIVERSITY of KENTUCKY
Sequential Pattern Mining Why sequential pattern mining? GSP algorithm Prefix. Span 2 CS 685: Special Topics in Data Mining
Sequence Database: Object A A A B B C 3 Timestamp 10 20 23 11 17 21 28 14 Events 2, 3, 5 6, 1 1 4, 5, 6 2 7, 8, 1, 2 1, 6 1, 8, 7 CS 685: Special Topics in Data Mining
Examples of Sequence Database Sequence Element (Transaction) Event (Item) Customer Purchase history of a given customer A set of items bought by a customer at time t Books, diary products, CDs, etc Web Data Browsing activity of a particular Web visitor Home page, index page, contact info, etc Event data History of events generated by a given sensor A collection of files viewed by a Web visitor after a single mouse click Events triggered by a sensor at time t Genome sequences DNA sequence of a particular species An element of the DNA sequence Bases A, T, G, C Element (Transaction) Sequence 4 E 1 E 2 E 1 E 3 E 2 E 3 E 4 Types of alarms generated by sensors Event (Item) CS 685: Special Topics in Data Mining
Formal Definition of a Sequence A sequence is an ordered list of elements (transactions) s = < e 1 e 2 e 3 … > Each element contains a collection of events (items) ei = {i 1, i 2, …, ik} Each element is attributed to a specific time or location Length of a sequence, |s|, is given by the number of elements of the sequence A k-sequence is a sequence that contains k events (items) 5 CS 685: Special Topics in Data Mining
What Is Sequential Pattern Mining? Given a set of sequences, find the complete set of frequent subsequences A sequence database SID sequence 10
Sequential Pattern Mining: Definition Given: a database of sequences a user-specified minimum support threshold, minsup Task: Find all subsequences with support ≥ minsup 7 CS 685: Special Topics in Data Mining
Sequential Pattern Mining: Challenge Given a sequence: <{a b} {c d e} {f} {g h i}> Examples of subsequences: <{a} {c d} {f} {g} >, < {c d e} >, < {b} {g} >, etc. How many k-subsequences can be extracted from a given n-sequence? <{a b} {c d e} {f} {g h i}> n = 9 k=4: _YY _ <{a} 8 Y_ {d e} _ _Y {i}> CS 685: Special Topics in Data Mining
Challenges on Sequential Pattern Mining A huge number of possible sequential patterns are hidden in databases A mining algorithm should Find the complete set of patterns satisfying the minimum support (frequency) threshold Be highly efficient, scalable, involving only a small number of database scans Be able to incorporate various kinds of userspecific constraints 9 CS 685: Special Topics in Data Mining
A Basic Property of Sequential Patterns: Apriori A basic property: Apriori (Agrawal & Sirkant’ 94) If a sequence S is not frequent Then none of the super-sequences of S is frequent E. g,
Basic Algorithm : Breadth First Search (GSP) L=1 While (Result. L != NULL) Candidate Generate Prune Test L=L+1 11 CS 685: Special Topics in Data Mining
Finding Length-1 Sequential Patterns , ,
Generating Length-2 Candidates
The Mining Process 5 th scan: 1 cand. 1 length-5 seq. pat. Cand. cannot pass sup. threshold <(bd)cba> Cand. not in DB at all 4 th scan: 8 cand. 6 length-4 seq.
Candidate Generate-and-test: Drawbacks A huge set of candidate sequences generated. Especially 2 -item candidate sequence. Multiple Scans of database needed. Inefficient for mining long sequential patterns. A long pattern grow up from short patterns The number of short patterns is exponential to the length of mined patterns. 15 18 March 2018 Data Mining: Concepts and Techniques Special Topics in Data Mining 15 CS 685:
Bottlenecks of GSP A huge set of candidates could be generated 1, 000 frequent length-1 sequences generate s huge number of length-2 candidates! Multiple scans of database in mining The length of each candidate grows by one at each database scan. Mining long sequential patterns Needs an exponential number of short candidates A length-100 sequential pattern needs 1030 candidate sequences! 16 CS 685: Special Topics in Data Mining
Pattern Growth (prefix. Span) Prefix and Suffix (Projection) ,
Mining Sequential Patterns by Prefix Projections Step 1: find length-1 sequential patterns , ,
Finding Seq. Patterns with Prefix Only need to consider projections w. r. t. -projected database: <(abc)(ac)d(cf)>, <(_d)c(bc)(ae)>, <(_b)(df)cb>, <(_f)cbc> Find all the length-2 seq. pat. Having prefix :
Completeness of Prefix. Span SDB SID 10
Efficiency of Prefix. Span No candidate sequence needs to be generated Projected databases keep shrinking Major cost of Prefix. Span: constructing projected databases Can be improved by pseudo-projections 21 CS 685: Special Topics in Data Mining
Speed-up by Pseudo-projection Major cost of Prefix. Span: projection Postfixes of sequences often appear repeatedly in recursive projected databases When (projected) database can be held in main memory, use pointers to form projections s=
Pseudo-Projection vs. Physical Projection Pseudo-projection avoids physically copying postfixes Efficient in running time and space when database can be held in main memory However, it is not efficient when database cannot fit in main memory Disk-based random accessing is very costly Suggested Approach: Integration of physical and pseudo-projection Swapping to pseudo-projection when the data set fits in memory 23 CS 685: Special Topics in Data Mining