Скачать презентацию Context-Aware Wrapping Synchronized Data Extraction Shui-Lung Chuang Kevin Скачать презентацию Context-Aware Wrapping Synchronized Data Extraction Shui-Lung Chuang Kevin

d9066909c7d27fcba5256f8432c85e26.ppt

  • Количество слайдов: 37

Context-Aware Wrapping: Synchronized Data Extraction Shui-Lung Chuang, Kevin C. Chang, Cheng. Xiang Zhai The Context-Aware Wrapping: Synchronized Data Extraction Shui-Lung Chuang, Kevin C. Chang, Cheng. Xiang Zhai The Database and Info. Systems Lab. University of Illinois at Urbana-Champaign

Structured Data on the Web Source amazon query Data in less-structured format Structured Data on the Web Source amazon query Data in less-structured format

Structured Data on the Web Source query Data in less-structured format wrapping Structured Data Structured Data on the Web Source query Data in less-structured format wrapping Structured Data amazon C Programming Lang. Effective Java PL guide ……… B. Kernighan et al J. Bloch ……… Mar 1988 Paperback Jun 2001 …………… … $44. 20 … $44. 99 … …… Wrapper [$title] by [$author] ($fmt $pdate) …. [$price]

Automatic Wrapper Construction Source query Data in less-structured format wrapping Structured Data amazon C Automatic Wrapper Construction Source query Data in less-structured format wrapping Structured Data amazon C Programming Lang. Effective Java PL guide ……… Automatic Wrapper Construction (E. g. , Road. Runner (VLDB’ 01), Ex. Alg (SIGMOD’ 03)) B. Kernighan et al J. Bloch ……… Mar 1988 Paperback Jun 2001 …………… … $44. 20 … $44. 99 … …… variant parts are data slots Wrapper Data Page(s): C Programming Lang. . by B. Kernighan et al ( Mar 1988 ) $44. 20 Effective Java PL Guide by J. Bloch ( Paperback Jun 2001 ) $44. 99 Discover Template [$title] by [$author] ($fmt $pdate) …. [$price]

Problem I: Correct Inherent Errors? Source query Data in less-structured format wrapping Structured Data Problem I: Correct Inherent Errors? Source query Data in less-structured format wrapping Structured Data amazon C Programming Lang. Effective Java PL guide ……… B. Kernighan et al J. Bloch ……… Mar 1988 Paperback Jun 2001 …………… format Automatic Wrapper Construction (E. g. , Road. Runner (VLDB’ 01), Ex. Alg (SIGMOD’ 03)) variant parts are data slots … $44. 20 … $44. 99 … …… pub-date data. Wrapper fields Data Page(s): C Programming Lang. . by B. Kernighan et al ( Mar 1988 ) $44. 20 Effective Java PL Guide by J. Bloch ( Paperback Jun 2001 ) $44. 99 Discover Template [$title] by [$author] ($fmt $pdate) …. [$price]

Putting In Integration— Isolated Wrapping Sources in a domain amazon Structured Data Wrapperam bn. Putting In Integration— Isolated Wrapping Sources in a domain amazon Structured Data Wrapperam bn. com C Programming Lang. Effective Java PL guide ……… B. Kernighan et al Mar 1988 … J. Bloch Paperback Jun 2001 … …………… … Wrapperbn The Art of Comp. Prog. Modern Operating Systems ……… D. Knuth A. Tanenbaum ……… Wrapperpw C Prog. Lang. (2 nd Edition) C++ Prog. Lang. (3 nd Edition) …… Britchie. Kernighan Biarne Stroustrup …… $44. 20 $44. 99 …… … Hardcover $16. 38 … Paperback $6. 95 … … ………… borders … … … Price: $39. 48 Price: $51. 99 ……

Problem II: Achieve Consistency? Sources in a domain amazon Structured Data Wrapperam bn. com Problem II: Achieve Consistency? Sources in a domain amazon Structured Data Wrapperam bn. com C Programming Lang. Effective Java PL guide ……… B. Kernighan et al Mar 1988 … J. Bloch Paperback Jun 2001 … …………… … Wrapperbn The Art of Comp. Prog. Modern Operating Systems ……… D. Knuth A. Tanenbaum ……… Wrapperpw C Prog. Lang. (2 nd Edition) C++ Prog. Lang. (3 nd Edition) …… Britchie. Kernighan Biarne Stroustrup …… $44. 20 $44. 99 …… … Hardcover $16. 38 … Paperback $6. 95 … … ………… borders Inconsistency … … … Price: $39. 48 Price: $51. 99 ……

Scenario 1: Can we leverage domain knowledge? n Leverage the users’ efforts C++ Programming Scenario 1: Can we leverage domain knowledge? n Leverage the users’ efforts C++ Programming Language (3 rd Ed. ) Bjarne Stroustrup Addison-Wesley ISBN: 0201539926 Pub Date: Jul 1991 List Price: $43. 95 Min Price: $8. 95 Automatic wrapping …… … Addison-Wesley ISBN: 0201539926 Pub Date: Jul 1991 ……… List Price: $43. 95 Min Price: $8. 95

Scenario 1: Can we leverage domain knowledge? n Leverage the users’ efforts C++ Programming Scenario 1: Can we leverage domain knowledge? n Leverage the users’ efforts C++ Programming Language (3 rd Ed. ) Bjarne Stroustrup Addison-Wesley ISBN: 0201539926 Pub Date: Jul 1991 List Price: $43. 95 Min Price: $8. 95 Domain Knowledge ISBN: Automatic wrapping “ISBN: ” + 10 digits Price: …… … Addison-Wesley ISBN: 0201539926 Pub Date: Jul 1991 ……… List Price: $43. 95 Min Price: $8. 95 “? Price: ” + $ ISBN: Date: Mon Year: 4 digits Mon: Jan | Feb | …. . ……… …… … Addison-Wesley Date: ISBN: Pub Date: 0201539926 Jul 1991 Price: List Price: $43. 95 Price: Min Price: $8. 95

Scenario 2: Can we leverage Existing Wrappers? n Leverage the previous efforts Existing wrapped Scenario 2: Can we leverage Existing Wrappers? n Leverage the previous efforts Existing wrapped sources: Amazon Correct Wrapperam C Programming Lang. Effective Java PL guide ……… B. Kernighan et al J. Bloch ……… Mar 1988 … Jun 2001 … ……… … Paperback ……… $44. 20 $44. 99 …… New sources to wrap: BN. com The Art of Comp. Prog. Modern Operating Systems …………… D. Knuth A. Tanenbaum …… …… Hardcover $143. 65 Paperback $45. 99 ………

Scenario 2: Can we leverage Existing Wrappers? n Leverage the previous efforts Existing wrapped Scenario 2: Can we leverage Existing Wrappers? n Leverage the previous efforts Existing wrapped sources: Amazon Correct Wrapperam C Programming Lang. Effective Java PL guide ……… B. Kernighan et al J. Bloch ……… Mar 1988 … Jun 2001 … ……… … Paperback ……… $44. 20 $44. 99 …… New sources to wrap: BN. com The Art of Comp. Prog. Modern Operating Systems …………… D. Knuth A. Tanenbaum …… …… Hardcover Paperback ……… $143. 65 $45. 99 ……

Scenario 3: Can we leverage Other Sources in integration? n Leverage the concurrent efforts Scenario 3: Can we leverage Other Sources in integration? n Leverage the concurrent efforts New sources to wrap: Amazon BN. com Borders Powells C Programming Lang. Effective Java PL guide ……… B. Kernighan et al Mar 1988 … J. Bloch Paperback Jun 2001 … …………… … The Art of Comp. Prog. Modern Operating Systems …………… C Prog. Lang. (2 nd Edition) C++ Prog. Lang. (3 nd Edition) …… The Da Vinci Code Lord of the Rigns …………… D. Knuth A. Tanenbaum …… Britchie. Kernighan Biarne Stroustrup …… Hanegraaff & Maier Tolkien …… …… … $44. 20 $44. 99 …… Hardcover $143. 65 Paperback $45. 99 ……… Price: $39. 48 Price: $51. 99 …… UK Edition 1 st Edition …… 5. 29 EUR 6. 38 EUR ……

Scenario 3: Can we leverage Other Sources in integration? n Leverage the concurrent efforts Scenario 3: Can we leverage Other Sources in integration? n Leverage the concurrent efforts New sources to wrap: Amazon BN. com Borders Powells C Programming Lang. Effective Java PL guide ……… B. Kernighan et al Mar 1988 … J. Bloch Paperback Jun 2001 … …………… … The Art of Comp. Prog. Modern Operating Systems …………… C Prog. Lang. (2 nd Edition) C++ Prog. Lang. (3 nd Edition) …… The Da Vinci Code Lord of the Rigns …………… D. Knuth A. Tanenbaum …… Britchie. Kernighan Biarne Stroustrup …… Hanegraaff & Maier Tolkien …… …… … $44. 20 $44. 99 …… Hardcover $143. 65 Paperback $45. 99 ……… Price: $39. 48 Price: $51. 99 …… UK Edition 1 st Edition …… 5. 29 EUR 6. 38 EUR ……

Scenario 3: Can we leverage Other Sources in integration? n Leverage the concurrent efforts Scenario 3: Can we leverage Other Sources in integration? n Leverage the concurrent efforts New sources to wrap: Amazon BN. com Borders Powells C Programming Lang. Effective Java PL guide ……… B. Kernighan et al Mar 1988 … J. Bloch Paperback Jun 2001 … …………… … The Art of Comp. Prog. Modern Operating Systems …………… C Prog. Lang. (2 nd Edition) C++ Prog. Lang. (3 nd Edition) …… The Da Vinci Code Lord of the Rigns …………… D. Knuth A. Tanenbaum …… Britchie. Kernighan Biarne Stroustrup …… Hanegraaff & Maier Tolkien …… …… … $44. 20 $44. 99 …… Hardcover Paperback ……… $143. 65 $45. 99 …… Price: $39. 48 Price: $51. 99 …… UK Edition 1 st Edition …… 5. 29 EUR 6. 38 EUR ……

Our Approach: Context-Aware Wrapping n Wrapping should exploit its context The Context of Another. Our Approach: Context-Aware Wrapping n Wrapping should exploit its context The Context of Another. Book. com Domain Knowledge ISBN: 10 digits Price: $ Date: Mon Year Existing wrapped sources: Amazon buy. com New sources: borders BN. com … … powells ……… ……… Context-Aware Wrapping. . . Another. Book. com Consistent Extractions

Demo! Demo!

Observation: Complementary Extraction (a) Road. Runner (VLDB’ 01) o: correct x: incorrect (b) Ex. Observation: Complementary Extraction (a) Road. Runner (VLDB’ 01) o: correct x: incorrect (b) Ex. Alg (SIGMOD’ 03) (c) Tree. Align (WWW’ 05) bn. com amazon title authors – Most sources have some fields extracted correctly – Most fields could be extracted correctly in some sources

Our Solution: Synchronization n Synchronize a source with its context The Context of Another. Our Solution: Synchronization n Synchronize a source with its context The Context of Another. Book. com Domain Knowledge Context-Aware Wrapping ISBN: 10 digits Price: $ Date: Mon Year ……… ……… Existing wrapped sources: Amazon buy. com New sources: borders BN. com … … powells . . . Another. Book. com Consistent Extractions

Our Solution: Synchronization n Synchronize a source with its context The Context of Another. Our Solution: Synchronization n Synchronize a source with its context The Context of Another. Book. com Domain Knowledge Context-Aware Wrapping ISBN: 10 digits Price: $ Date: Mon Year ……… ……… Existing wrapped sources: Amazon buy. com New sources: borders BN. com … … powells Another. Book. com Wrapper A …… Wrapper C Auto-Wrap …… . . . Auto-Wrap Initial Extractions Consistent Extractions …… ……

Our Solution: Synchronization n Synchronize a source with its context Context-Aware Wrapping ISBN: 10 Our Solution: Synchronization n Synchronize a source with its context Context-Aware Wrapping ISBN: 10 digits Price: $ Date: Mon Year Existing wrapped sources: Amazon buy. com New sources: borders BN. com … … powells Another. Book. com Wrapper A …… Wrapper C Auto-Wrap …… Synchronization The Context of Another. Book. com Domain Knowledge ……… ………. . . Auto-Wrap Initial Extractions Consistent Extractions …… ……

Our Solution: Synchronization n Synchronize a source with its context Context-Aware Wrapping ISBN: 10 Our Solution: Synchronization n Synchronize a source with its context Context-Aware Wrapping ISBN: 10 digits Price: $ Date: Mon Year Existing wrapped sources: Amazon Synchronization The Context of Another. Book. com Domain Knowledge But, How …… to Synchronize? buy. com New sources: borders BN. com Wrapper A … Wrapper C … powells Another. Book. com Auto-Wrap …… ……… ……… . . . Auto-Wrap Initial Extractions Consistent Extractions …… ……

Intuition: Wrapping as Communication q q Sources encode records as Web pages Wrappers decode Intuition: Wrapping as Communication q q Sources encode records as Web pages Wrappers decode Web pages back to records (Encoder) records n Source (Code) HTML pages (Decoder) Wrapper records What is the best possible decoding scheme?

Encoding & Decoding a Single Message 100 error 100 -10 Source HTML pages 10 Encoding & Decoding a Single Message 100 error 100 -10 Source HTML pages 10 X Decoder The Art of … . . . Hardcover $143. 65 The Art of. . … Hardcover $143. 65 records Channel Encoder 101 -10 The Art of. . … Hardcover $143. 65 Wrapper records

Insight: Turbo Codes– Multiple Messages error Message 100 -10 100 -01 Encoder 2 C Insight: Turbo Codes– Multiple Messages error Message 100 -10 100 -01 Encoder 2 C Prog … … Paperback Source 1 Source 2 100 Reinforcing 100 -11 X 00 Decoder 2 HTML pages The Art of. . … Hardcover $143. 65 records Wrapper 1 C Prog…/a> Paperback Mar 1988 $44. 20 … $44. 20 10 X Decoder The Art of … . . . Hardcover $143. 65 The Art of. . … Hardcover $143. 65 records Channel Encoder 101 -10 C… Wrapper 2 … Paperback Mar 1988 $44. 20

Context-Aware v. s. Single-Source Wrapping n Traditional single-source wrapping: Utilize the information within a Context-Aware v. s. Single-Source Wrapping n Traditional single-source wrapping: Utilize the information within a source (syntax) n Context-aware wrapping: Further utilize the information between multiple sources (content) The Art of … . . . Hardcover $143. 65 The Art of. . … Hardcover $143. 65 records C Prog … … Paperback Source 1 HTML pages The Art of. . … Hardcover $143. 65 C Prog…/a> Paperback Mar 1988 $44. 20 … $44. 20 Source 2 records Wrapper 1 C… Wrapper 2 … Paperback Mar 1988 $44. 20

Challenge 1: Is there a common message? n From our observation q n The Challenge 1: Is there a common message? n From our observation q n The fields converge to a small set domain fields Hidden domain model: ={ f 1, f 2, … } q Each fi is a statistic model for generating a field q Example: Book domain n n n title: 2 -20 randomly picked-up words authors: a few randomly picked-up non-dictionary words format: 1 -2 words from {paperback, hardcover, …} date: +<4 -digits> or mm/dd/yyyy price: $. . .

Message: Hidden Domain Model n From our observation q n Each source is a Message: Hidden Domain Model n From our observation q n Each source is a subset of domain fields Abstraction of encoding records as pages Message

Challenge 2: How to reinforce between sources? n Determine Field Correspondence Schema Matching f Challenge 2: How to reinforce between sources? n Determine Field Correspondence Schema Matching f 11 S 1: f 12 C Programming Lang. Effective Java PL guide f 13 B. Kernighan et al J. Bloch Mar 1988 … $44. 20 Paperback Jun 2001 … $44. 99 f 21 S 2: f 22 The Art of Comp. Prog. Modern Operating Systemsh D. Knuth A. Tanenbaum f 31 S 3: The Da Vinci Code Lord of the Rigns 1 f 23 …… …… Hardcover $143. 65 Paperback $45. 99 f 32 C Prog. Lang. (2 nd Edition) C++ Prog. Lang. (3 nd Edition) f 41 S 4: f 14 f 34 Britchie. Kernighan Biarne Stroustrup f 42 Hanegraaff & Maier Tolkien 2 … … Price: $39. 48 Price: $51. 99 f 44 …… …… UK Edition 1 st Edition 3 f 45 5. 29 EUR 6. 38 EUR 4

Reinforcing: Soft Schema Matching n If two sources are consistent, their matching score will Reinforcing: Soft Schema Matching n If two sources are consistent, their matching score will be high; otherwise, low. f 11 S 1: f 12 C Programming Lang. Effective Java PL guide f 13 B. Kernighan et al J. Bloch Mar 1988 … $44. 20 Paperback Jun 2001 … $44. 99 f 21 S 2: f 22 The Art of Comp. Prog. Modern Operating Systemsh D. Knuth A. Tanenbaum f 31 S 3: The Da Vinci Code Lord of the Rigns f 23 …… …… Hardcover $143. 65 Paperback $45. 99 f 32 C Prog. Lang. (2 nd Edition) C++ Prog. Lang. (3 nd Edition) f 41 S 4: f 14 f 34 Britchie. Kernighan Biarne Stroustrup f 42 Hanegraaff & Maier Tolkien 2 … … Price: $39. 48 Price: $51. 99 f 44 …… …… UK Edition 1 st Edition 3 f 45 5. 29 EUR 6. 38 EUR 4

Reinforcing: Soft Schema Matching n If two sources are consistent, their matching score will Reinforcing: Soft Schema Matching n If two sources are consistent, their matching score will be high; otherwise, low. f 11 S 1: f 12 C Programming Lang. Effective Java PL guide f 13 B. Kernighan et al J. Bloch Mar 1988 … $44. 20 Paperback Jun 2001 … $44. 99 f 21 S 2: f 22 The Art of Comp. Prog. Modern Operating Systemsh D. Knuth A. Tanenbaum f 31 S 3: The Da Vinci Code Lord of the Rigns 1* f 23 …… …… Hardcover $143. 65 Paperback $45. 99 f 32 C Prog. Lang. (2 nd Edition) C++ Prog. Lang. (3 nd Edition) f 41 S 4: f 14 2 * Y 3 * Y 4 * f 34 Britchie. Kernighan Biarne Stroustrup f 42 Hanegraaff & Maier Tolkien Y 2 * … … Price: $39. 48 Price: $51. 99 f 44 …… …… UK Edition 1 st Edition f 45 5. 29 EUR 6. 38 EUR

Implementation n A field is modeled as a graph model: Label n Value Examples: Implementation n A field is modeled as a graph model: Label n Value Examples: $35. 07 Our Price: $34. 07 Low Price: $23. 05 ISBN: 012569586161 UPC# 014633147841 Each node (L or V) is a prob. distribution over

Experiment Setup n Domains (each, 10 sources) q Albums, e. g. , allmusic. com, Experiment Setup n Domains (each, 10 sources) q Albums, e. g. , allmusic. com, etc Books, e. g. , amazon. com, bn. com, etc Cars, e. g. , cars. com q The 1 st response pages for 3 queries (~300 records in a domain) q q n Base automatic single-source wrappers q q LL: Line-Splitter (baseline) RR: Road. Runner (VLDB’ 01) EA: Ex. Alg (SIGMOD’ 03) TA: Tree. Align (WWW’ 05)

Experiment Result: 10 New Book Sources Error rate: LL: . 33 -> RR: . Experiment Result: 10 New Book Sources Error rate: LL: . 33 -> RR: . 38 -> EA: . 15 -> TA: . 35 -> q q Achieve lower inconsistency (i. e. , lower error rate) Achieve stabler performance (regardless the initial single-source wrappers) . 05. 06

Experiment Result: Incremental Setting Base Wrapper: LL q More sources do achieve better performance Experiment Result: Incremental Setting Base Wrapper: LL q More sources do achieve better performance

Conclusions n We proposed and developed q q Concept: Context-aware wrapping to support data Conclusions n We proposed and developed q q Concept: Context-aware wrapping to support data wrapping in integrating online sources Approach: Synchronized data extraction from the insight of turbo codes n Or, in a nutshell: Wrapping (Data Extraction) q Schema Matching Evaluation: Experiments show the feasibility of the proposed approach

The End Thanks!! The End Thanks!!