Introduction to Data Mining Univ -Prof Dr Peter

Introduction to Data Mining Univ. -Prof. Dr. Peter Brezany Institute of Scientific Computing Faculty for Information Science University of Vienna E-mail : brezany@par. univie. ac. at WWW: http: //www. par. univie. ac. at/~brezany http: //artemis. wszib. edu. pl/~brezany/ P. Brezany Institute of Scientific Computing – University of Vienna

Introduction • This lecture topic is about theme which has come to be known as data mining and knowledge discovery in large databases, data warehouses, and other massive information repositories. • Data mining emerged during the late 1980 s; has made great strides during the late 1990 s, and is expected to continue to flourish into the new millennium. • The implementation methods discussed are particularly oriented towards the development of scalable and efficient data mining tools. • We introduce interesting data mining techniques and systems, and discuss applications and research directions. P. Brezany Institute of Scientific Computing – University of Vienna 2

What Motivated Data Mining? Why Is It Important? • There is the wide availability of huge amounts of data and the imminent need for turning such data into useful information and knowledge. • Applications ranging from business management, production control, and market analysis, to engineering design and medical and science exploration. • Data mining can be viewed as a result of the natural evolution of information technology - including database technology, artificial intelligence, machine learning, neural networks, statistics, pattern-recognition, knowledge-based systems, high -performance computing, and data visualization. P. Brezany Institute of Scientific Computing – University of Vienna 3

Motivation Business Medicine Scientific experiments Simulations P. Brezany Data repositories (files, databases, data warehouses) Earth observations Institute of Scientific Computing – University of Vienna 4

Motivation (another view) Laboratories Satellites (microscopes, MRI/CT scanners, . . . ) Business Data Repositories Experiments (high energy physics, . . . ) P. Brezany Analysis Computer simulations Institute of Scientific Computing – University of Vienna 5

The Evolution of Database Technology Data Collection and Database Creation (1960 s and earlier) - Primitive file processing Database Management Systems (1970 s-early 1980 s) - Hierarchical, network and relational DB systems - Query languages (SQL, etc), query optimization - Transaction management, concurrency control, recovery - Data modeling tools Advanced Database Systems (mid-1980 s-present) object-oriented, object-relational, spatial, temporal, multimedia Web-based Database Systems (1990 s-present) - XML-based DB systems, - Web mining Data Warehousing and Data Mining (late 1980 s-present) - Data warehouse and OLAP technology - Data mining and knowledge discovery P. Brezany Institute of Scientific Computing – University of Vienna 6

Database Querying and Data Mining Database query languages like SQL are standardized and powerful, but for not skilled users are they too difficult. OLAP is used for interactive analysis of data stored in a data warehouse. Its applications require viewing the data from many perspectives (dimensions). OLAP Tools allow flexible multidimensional queries. Their methods are query-centric. The user can selelect and query any subset of dimensions for processing and perform aggregations along the dimensions. Data mining goes far beyond OLAP summarization style analytical processing by incorporating more Data Warehouse advanced analysis techniques. Query languages like SQL P. Brezany OLAP Tools Institute of Scientific Computing – University of Vienna Data Mining Tools 7

We Are Data Rich, But Information Poor P. Brezany Institute of Scientific Computing – University of Vienna 8

So, What Is Data Mining? Data mining – searching for knowledge (interesting patterns) in your data. P. Brezany Institute of Scientific Computing – University of Vienna 9

Data Mining As a Step in the Process of Knowledge Discovery • Many people treat data mining as a synonym for the term Knowledge Discovery in Databases, or KDD. • Alternative view: data mining as an step in KDD: – 1, Data cleaning (to remove noise and inconsistent data) – 2. Data integration (where multiple data sources may be combined) – 3. Data selection (where data relevant to the analysis task are retrieved from the database) – 4. Data transformation (where data are transformed or consolidated into forms appropriate for mining by performing summary or aggregation operations, for instance) – 5. Data mining (an essential process where intelligent methods are applied in order to extract patterns) – 6. Pattern evaluation (to identify the truly interesting patterns representing knowledge based on some interestingness measures) – 7. Knowledge presentation to the user P. Brezany Institute of Scientific Computing – University of Vienna 10

Data Mining in Knowledge Discovery P. Brezany Institute of Scientific Computing – University of Vienna 11

Architecture of a Data Mining System Graphical user interface Pattern evaluation Knowledge base Data mining engine Database or data warehouse server Data cleaning, data integration Database P. Brezany Filtering Data warehouse Institute of Scientific Computing – University of Vienna 12

Architecture of a Data Mining System (2) Database, data warehouse, or other information repository: One or a set of databases, data warehouses, spreadsheets, etc. Database or data warehouse server: responsible for fetching the relevant data, based on the user’s data mining request. Knowledge base: domain knowledge that is used to guide the search, or evaluate the interestingness of resulting patterns. Such knowledge can include concept hierarchies, used to organize attribute values into different levels of abstraction. Data mining engine: essential to the data mining system; ideally consists of a set of functional modules for tasks such as association, classification, cluster analysis, and evolution and deviation analysis. 13 P. Brezany Institute of Scientific Computing – University of Vienna

Architecture of a Data Mining System (3) Pattern evaluation module: This component typically employs interestingness measures and interacts with the data mining so as to focus the search towards interesting patterns. It may use interestingness thresholds to filter out discovered patterns. Graphical user interface: This module communicates between users and the data mining system allowing the user • to specify a data mining query or task • provide information to help focus the search • perform exploratory data mining based on the intermediate data mining results • browse database and data warehouse schemas or data structures • evaluate mined patterns • visualize the patterns in different forms. P. Brezany Institute of Scientific Computing – University of Vienna 14

Data Mining vs. Other Disciplines From a data warehouse perspective, data mining can be viewed as an advance stage of on-line analytical processing (OLAP). However, data mining goes far beyond OLAP. There may be many “data mining systems” on the market not all of them can perform true data mining. Data mining integrates techniques from multiple disciplines: database technology, statistics, machine learning, high-performance computing, neural networks, pattern recognition, visualization. P. Brezany Institute of Scientific Computing – University of Vienna 15

Data Mining - On What Kind of Data? • • Relational Databases Data Warehouses Transactional Databases Advanced Database Systems and Advanced Database Applications – – – P. Brezany Object-oriented databases Object-relational databases Spatial databases Text databases and multimedia databases The World Wide Web. . . Institute of Scientific Computing – University of Vienna 16

Relational Database - An Example • A database system, also called a database management system (DBMS), consists of a collection of interrelated data, known as a database, and a set of software programs to manage and access the data. • A relational database is a collection of tables, each of which is assigned a unique name. Each table consists of a set of attributes (columns or fields) and usually stores a large set of tuples (records or rows). Each tuple represents an object identified by a unique key. • Relational data can be accessed by database queries written in a relational query language, such as SQL. • Using data mining, one can search for trends or data patterns in relational databases. P. Brezany Institute of Scientific Computing – University of Vienna 17

Relational Databases - Example The All. Electronics company is described by the following table: customer, item, employee, and branch. Fragments of these tables are shown on the next slide; the attribute that represents the key or composite key component is underlined. • The relation customer consists of a set of attributes, including a unique customer identity number (cust_ID), and so on. • Tables can also be used to represent the relationships between or among multiple relational tables. E. g. , these include purchases (customer purchases items, creating a sales transaction that is handled by an employee), items_sold (lists the items sold in the given transaction), and works_at (employee works at a branch of All. Electronics). P. Brezany Institute of Scientific Computing – University of Vienna 18

Fragments of Relations from All. Electronics DB P. Brezany Institute of Scientific Computing – University of Vienna 19

Data Warehouses A data warehouse is a repository of information collected from multiple sources, stored under a unified schema, and which usually resides at a single site. Data warehouses are constructed via a process of data cleaning, data transformation, data integration, data loading and periodic data refreshing. Figure on the next slide shows the basic architecture of a data warehouse for All. Electronics. In order to facilitate decision making, the data in a data warehouse are organized around major subjects, such as customer, item, supplier, and activity. The data are stored from a historical perspective and are typically summarized. P. Brezany Institute of Scientific Computing – University of Vienna 20

Architecture of a Data Warehouse Client Data source in Ch. Data source in NY Clean Transform Integrate Load Data warehouse Client Data source in T. Data source in Vancouver P. Brezany Query and analysis tools Remarks: Ch - Chicago, NY - New York, T - Toronto Institute of Scientific Computing – University of Vienna 21

Modeling a Data Warehouse A data warehouse is usually modeled by a multidimensional database structure, where each dimension corresponds to an attribute in the schema, each cell stores the value of some aggregate measure, such as count or sales_amount. The actual physical structure of a data warehouse may be a relational data store or a multidimensional data cube. It provides a multidimensional view of data and allows the precomputation and fast accessing of summarized data. Example: A data cube for summarized sales data of All. Electronics is presented in the next slide. P. Brezany Institute of Scientific Computing – University of Vienna 22

A Multidimensional Data Cube P. Brezany Institute of Scientific Computing – University of Vienna 23

Modeling a Data Warehouse (2) Data warehouse vs. Data mart: A data warehouse collects information about subjects and span an entire organization, and thus its scope is enterprise wide. A data mart is a department-wide. Data warehouse systems are well suited for On-Line Analytical processing, or OLAP operations allow the presentation of data at different levels of abstractions. Examples of OLAP operations include drill-down and roll-up, which allow the user to view the data at different degrees of summarization as illustrated in the previous slide. P. Brezany Institute of Scientific Computing – University of Vienna 24

Illustration of Some Other OLAP Operations P. Brezany Institute of Scientific Computing – University of Vienna 25

P. Brezany Institute of Scientific Computing – University of Vienna

Transactional Databases A transactional database consists of a file where each record represents a transaction. A transaction includes a unique transaction identity number (trans_id), and a list of the items making up the transaction (such as items purchased in a store). The transactional database may have additional tables associated with it, which contain other information regarding the sale, such as the date of the transaction, the custommer ID number, the ID number of the sales person, etc. Example: Transactions can be stored in a table, with one record per transaction. A fragment of a transactional database for All. Electronics is shown in the next slide. P. Brezany Institute of Scientific Computing – University of Vienna 27

Transactional Databases (2) Trans_id list of item_Ids T 100. . . I 1, I 3, I 8, I 16. . . The transactional database is usually either stored in a flat file in a format similar to that of the above table, or unfolded into a standard relation in a format similar to that of the items_sold table in slide no. 42. A regular data retrieval system is not able to answer queries like “Which items sold well together? ” P. Brezany Institute of Scientific Computing – University of Vienna 28

Advanced Database Systems and Database Applications Relational DB systems have been widely used in business applications. The new database applications include handling • spatial data (e. g. maps) • engineering design data (e. g. , the design of buildings or integrated circuits) • hypertext and multimedia data (text, image, video, audio data) • time-related data (e. g. stock exchange data) • World Wide Web (a huge, widely distributed information repository made available by the Internet) P. Brezany Institute of Scientific Computing – University of Vienna 29

Data Mining Functionalities - What Kinds of Patterns Can be Minded? • Data mining functionalities are used to specify the kind of patterns that can be found in data mining tasks. • Data mining tasks can be classified into 2 categories: – Descriptive - they characterize the general properties of the data in the database. – Prescriptive - they perform inference on the current data in order to make predictions. • In some cases, users may have no idea which kinds of patterns may be interesting => searching for several different kinds of patterns in parallel. • Data mining systems should be able to discover patterns at various granularities (abstraction levels). • Specifying hints to guide or focus the search. P. Brezany Institute of Scientific Computing – University of Vienna 30

Association Analysis • Association analysis is the discovery of association rules showing attribute-value conditions that occur frequently in a given set of data. • The association rule X => Y is interpreted as “database tuples that satisfy the conditions in X are also likely to satisfy the conditions in Y. ” • Example A data mining system may find in All. Electronics: age(X, “ 20. . 29”) and income(X, “ 20 K. . 29 K”) => buys(X, ”CD player”) [support = 2%, confidence = 60%] • X is a variable representing a customer. The rule indicates that of the customers under study, 2% are 20 to 29 years of age with an income of 20 K to 29 K and have purchased a CD player. There is a 60% probability that a customer in this age and incomegroup will purchase a CD player. P. Brezany Institute of Scientific Computing – University of Vienna 31

Association Analysis (Cont. ) • We would like to determine which items are frequently purchased together within the same transactions. E. g. , contains(T, “computer”) => contains(T, “software”) [support = 1%, confidence = 50%] • Explanation: if a transaction, T, contains “computer”, there is a 50% chance that it contains “software” as well, and 1% of all of the transactions contain both. • This rule involves a single attribute or predicate (i. e. contains) => single-dimensional association rule. It can be written simpy as “computer => software {1%, 50%]” Remark: On the last slide, we have: multi-dimensional assoc. rule. P. Brezany Institute of Scientific Computing – University of Vienna 32

Classification and Prediction • Classification is the process of finding a set of models (or functions) that describe and distinguish data classes or concepts, for the purpose of being able to use the model to predict the class of objects whose class label is unknown. The derived model is based on the analysis of a training data (i. e. , data objects whose class label is known), • “How is the derived model presented? ” – Classification (IF-THEN) rules – Mathematical formulae – Decision tree - it is a flow-chart-like tree structure, where each node denotes a test on an attribute value, each branch represents an outcome of the test, and the tree leaves represent classes or class distributions. – Neural networks - a collection of neuron-like processing units with weighted connections between the units. P. Brezany Institute of Scientific Computing – University of Vienna 33

Classification and Prediction (Cont. ) • Prediction - in many applications, users may wish to predict some missing or unavailable data values rather then class labels. The predicted values are usually numerical data. • Classification and prediction may need to be preceded by relevance analysis, which attempts to identify attributes that do not contribute to the classification or prediction process. These attributes can then be excluded. P. Brezany Institute of Scientific Computing – University of Vienna 34

Cluster Analysis • Clustering analyzes data objects without consulting a known class label. • Clustering can be used to generate such labels. • The objects are clustered or grouped based on the principle of maximizing the intraclass similarity and minimizing the interclass similarity. • Each cluster can be viewed as a class of objects, from which rules can be derived. • Example Cluster analysis can be performed on All. Electronics customer data in order to identify homogeneous subpopulations of customers. These clusters may represent individual target groups for marketing. (Figure on the next slide shows a 2 -D plot of customers with respect to customer locations in a city). P. Brezany Institute of Scientific Computing – University of Vienna 35

Cluster Analysis - Example A 2 -D plot of customer data with respect to customer locations in a city, showing 3 data clusters. Each cluster „center“ is marked with a „+“. P. Brezany Institute of Scientific Computing – University of Vienna 36

Outlier Analysis • A database may contain data objects that do not comply with the general behaviour or model of the data. These data objects are outliers, • Most data mining methods discard outliers as noise or exceptions. • In some applications such as fraud detection, the rare events can be more interesting than the more regularly occuring ones, • Example Outlier analysis may uncover fraudulent usage of credit cards by detecting purchases of extreemly large amounts for a given account number in comparison to regular charges incurred by the same account. P. Brezany Institute of Scientific Computing – University of Vienna 37

Evolution Analysis • It describes and models regularities or trends for objects whose behavior changes over time. • It includes time-series data analysis. • Example Suppose that we have the major stock market (time-series) data of the last several years available from the New York Stock Exchange and we would like to invest in shares of high-tech industrial companies. A data mining study of stock exchange data may identify stock evolution regularities for overall stocks and for the stocks of particular companies, Such regularities may help predict future trends in stock market prices. P. Brezany Institute of Scientific Computing – University of Vienna 38

Are All of the Patterns Interesting? • A data mining system has the potential to generate thousands or even millions of patterns, or rules. • Only a small fraction of the patterns potentially generated would actually be of interest to any given user. • Questions: What makes a pattern interesting? Can a data mining system generate all of the interesting patterns? Can a data mining system generate only interesting patterns? • A pattern is interesting if – – P. Brezany (1) it is easily understood by humans, (2) valid on new or test data with some degree of certainty. (3) potentially useful , and (4) novel Institute of Scientific Computing – University of Vienna 39

Interestingness of Patterns (Cont. ) • A pattern is also interesting if it validates a hypothesis that the user sought to confirm. • An interesting pattern represents knowledge. • Objective measures of pattern interestingness these are based on the structure of discovered patterns and the statistics underlying them. – An objective measure for association rules X => Y is rule support, representing the percentage of transactions from a transaction base that the given rule satisfies. This is taken to be the probability P(X U Y), where X U Y indicates that a transaction contains both X and Y, that is, the union of item sets X and Y. – Another objective measure for association rules is confidence, which assesses the degree of certainty of the detected association. This is taken to be the conditional probability P(X | Y), that is, the probability that a transaction containing X also contains Y. P. Brezany Institute of Scientific Computing – University of Vienna 40

Interestingness of Patterns (Cont. ) • Each interestingness measure is associated with a threshold, which can be controlled by the user. – For example, rules that do not satisfy a confidence threshold of, say, 50% can be considered uninteresting. Rules below the threshold likely reflect noise, exceptions, or minority cases and are probably of less value. • Objective measures are insufficient unless combined with subjective measures which reflect the needs and interests of a particular user. • Many patterns that are interesting by objective standards may represent common knowledge and, therefore, are actually uninteresting. P. Brezany Institute of Scientific Computing – University of Vienna 41

Interestingness of Patterns (Cont. ) • Subjective interestingness measures are based on user beliefs in the data. These measures find patterns interesting if they are unexpected or offer strategic information on which the user can act. • Patterns that are expected can be interesting if they confirm a hypothesis that the user wished to validate. P. Brezany Institute of Scientific Computing – University of Vienna 42

Completeness And Optimization of Data Mining Algorithm • Can a data mining system generate all of the interesting patterns? - this question refers to the completeness of a data mining algorithm. • It is often unrealistic and inefficient to generate all of the possible patterns. Instead, user-provided constraints and interestingness measures should be used to focus the search. • Can a data mining system generate only interesting patterns? This is an optimization problem in data mining - a challenging issue. P. Brezany Institute of Scientific Computing – University of Vienna 43