Introduction to Data Mining Dr. Hany Saleeb
Why Data Mining? — Potential Applications z Direct Marketing y identify which prospects should be included in a mailing list z Market segmentation y identify common characteristics of customers who buy same products z Market Basket Analysis y Identify what products are likely to be bought together z Insurance Claims Analysis y discover patterns of fraudulent transactions y compare current transactions against those patterns
What Is Data Mining? z Combination of AI and statistical analysis to discover information that is “hidden” in the data y associations (e. g. linking purchase of pizza with beer) y sequences (e. g. tying events together: marriage and purchase of furniture) y classifications (e. g. recognizing patterns such as the attributes of employees that are most likely to quit) y forecasting (e. g. predicting buying habits of customers based on past patterns) Expert systems or small ML/statistical programs
What can data mining do? z Classification – Classify credit applicants as low, medium, high risk – Classify insurance claims as normal, suspicious z Estimation – Estimate the probability of a direct mailing response – Estimate the lifetime value of a customer z Prediction – Predict which customers will leave within six months – Predict the size of the balance that will be transferred by a credit card prospect
What can data mining do? (cont’d) z Association – Find out items customers are likely to buy together – Find out what books to recommend to Amazon. com users z Clustering – Difference from classification: classes are unknown!
Market Analysis and Management z Where are the data sources for analysis? y Credit card transactions, loyalty cards, discount coupons, customer complaint calls, plus (public) lifestyle studies z Target marketing y Find clusters of “model” customers who share the same characteristics: interest, income level, spending habits, etc. z Determine customer purchasing patterns over time y Conversion of single to a joint bank account: marriage, etc. z Cross-market analysis y Associations/co-relations between product sales y Prediction based on the association information
Data Mining: Confluence of Multiple Disciplines Database Technology Machine Learning Information Science Statistics Data Mining Visualization Other Disciplines
Data Mining: On What Kind of Data? z z Relational databases Data warehouses Transactional databases Advanced DB and information repositories y Object-oriented and object-relational databases y Spatial databases y Time-series data and temporal data y Text databases and multimedia databases y Heterogeneous and legacy databases y WWW
Data Mining Process Learning Collecting relevant data Model building Understanding of business Problem identification Business strategy and evaluation Action
Requirements/challenges in Data Mining z. User interface z. Mining methodology z. Performance z. Data source z. Social and Security
Requirements/challenges in Data Mining(2) z. User interface - Data Visualization x. Understandability and interpretation of results x. Information representation and rendering x. Screen real-estate - Interactivity x. Manipulation of mined knowledge xfocus and refine mining tasks x. Focus and refine mining results
Requirements/challenges in Data Mining(3) z. Mining Methodology y. Mining different kinds of knowledge in databases y. Interactive mining of knowledge at multiple levels of abstraction y. Incorporation of background knowledge y. Query languages y. Expression and visualization of results y. Handling noise and incomplete data y. Pattern evaluation
Requirements/challenges in Data Mining (4) z. Performance y. Efficiency and scalability of data mining algorithms x. Linear algorithms needed y. Parallel and distributed methods x. Incremental methods x. Divide and conquer?
Requirements/challenges in Data Mining(5) z. Data Source y. Diversity of data types x. Handling complex types of data x. Mining information from heterogenous data bases or information repositories x. Can we expect a DM algorithm to do well on all types of data ? y. Data glut x. Are we collecting the right data for the right answer? x. Distinguish between important and unimportant data
Requirements/challenges in Data Mining(6) z. Social and Security -Social Impact x. Private and sensitive data is gathered and mined without individual’s knowledge and/or consent x. Appropriate use and distribution of discovered knowledge - Regulations Need for privacy and DM policies
Data Mining Tools
Summary z. The benefits of knowing one’s business is critical; technologies are coming together to support data mining. z. Data mining is the process and result of knowledge production, knowledge discovery and knowledge management.
Скачать презентацию Introduction to Data Mining Dr Hany Saleeb
34f274ea2a94444ed031a926831a32d0.ppt