数据挖掘应用赵卫东博士复旦大学客户关系管理 CRM 顾客生命周期

数据挖掘应用赵卫东博士复旦大学

客户关系管理（CRM）

顾客生命周期盈利收入寿命支出获取消费者保持消费者寿命消费者分析和恢复

顾客数据

数据挖掘在CRM中的应用

Customer identification n CRM begins with customer identification. This phase involves targeting the population who are most likely to become customers or most profitable to the company. It also involves analyzing customers who are being lost to the competition and how they can be won back. Elements for customer identification include target customer analysis and customer segmentation.

Customer attraction n Organizations can direct effort and resources into attracting the target customer segments. Direct marketing is a promotion process which motivates customers to place orders through various channels. direct mail or coupon

目标营销

Customer retention n n Central concern for CRM. Customer satisfaction is the essential condition for retaining customers. Elements of customer retention include one-to-one marketing, loyalty programs and complaints management. One-to-one marketing refers to personalized marketing campaigns which are supported by analyzing, detecting and predicting changes in customer behaviors. Loyalty programs involve campaigns or supporting activities which aim at maintaining a long term relationship with customers. Churn analysis, credit scoring, service quality or satisfaction form part of loyalty programs.

客户流失分析

Customer development n n n Elements of customer development include customer lifetime value analysis, up/cross selling and market basket analysis. Customer lifetime value analysis is defined as the prediction of the total net income a company can expect from a customer. Up/Cross selling refers to promotion activities which aim at augmenting the number of associated or closely related services that a customer uses within a firm. Market basket analysis aims at maximizing the customer transaction intensity and value by revealing regularities in the purchase behaviour of customers.

SPSS通讯行业分析专题

SPSS Modeler通讯行业分析模型

Personalized recommendation systems

Personalized recommendation n Personalization is defined as “the ability to provide content and services tailored to individuals based on knowledge about their preferences and behavior” or “the use of technology and customer information to tailor electronic commerce interactions between a business and each individual customer” Internet recommendation systems (Internet recommender systems) in electronic commerce is to reduce irrelevant content and provide users with more pertinent information or product. A recommendation system is a computer-based system that uses profiles built from past usage behavior to provide relevant recommendations.

Information filtering and recommendation n n rule-based filtering, content-based filtering, and collaborative filtering. Rule-based filtering uses pre-specified if-then rules to select relevant information for recommendation. Content-based filtering uses keywords or other productrelated attributes to make recommendations. Collaborative filtering uses preferences of similar users in the same reference group as a basis for recommendation.

Typical personalization process n n n understanding customers through profile building delivering personalized offering based on the knowledge about the product and the customer measuring personalization impact

Inadequate information in IR n n One possible solution for overcoming the problem is to expand the query by adding more semantic information to better describe the concepts. Relevance feedbacks and knowledge structure are used to add appropriate terms to expand the queries. Relevance feedbacks are information on the items selected by the user from the output of previous queries.

Spreading Activation Model n In the Spreading Activation (SA) Model, concepts are expanded based on the semantics in the process of identifying customer profile and matching items and the model has been applied to expand queries.

A personalized knowledge recommendation system n n A semantic-expansion approach to build the user profile by analyzing documents previously read by the person. The semantic-expansion approach that integrates semantic information for spreading expansion and content-based filtering for document recommendation.

A sample semantic-expansion network

Experimental results n An empirical study using master theses in the National Central library in Taiwan shows that the semantic-expansion approach outperforms the traditional keyword approach in catching user interests.

构件库管理

自适应构件检索 n n n 构件检索是构件库研究中的重要问题，有效的构件检索机制能够降低构件复用成本。构件的复用者并不是构件的设计者或构件库的管理员，在检索构件时对构件库的描述理解不充分，导致难以给出完整和精确的检索需求。用户选择构件的结果反映其真实需求，如果能够从用户的检索行为以及用户对检索结果的反馈中推断出用户的非精确检索条件与用户实际需要的精确检索条件之间内在联系的模式，就可以提高系统的查准率。

基于关联挖掘的自适应构件检索 n 把关联规则挖掘方法引入构件检索，从用户检索行为以及反馈中挖掘出非精确检索条件与精确检索结果之间的关联规则，从而调整检索机制，提高构件检索的查准率。

实例 {windows} {windows , SQL Server} {Linux , Mysql} {金融, SQL Server} {windows , 金融, SQL Server}

供应链管理

零部件供应商选择 n n 如何选择供应商不仅决定了产品的质量和成本, 也决定了产品的销售价格、维护费用和用户满意程度。选择供应商一般以满足时间约束的条件下最小化物流成本为目标，没有考虑零部件故障率与不同地域环境之间的相关性。

基于关联规则的零部件供应商选择 n n 使用关联规则挖掘算法，从产品维修记录中，寻找不同供应商提供的产品零部件及其组合在不同地域的频繁故障模式。在生成供应商选择和配送方案过程中，利用这些频繁故障模式，选择合适的零部件供应商组合，达到物流成本与产品维护成本的联合优化。

人力资源管理

人力资源管理 n n 人力资源在高科技公司中的地位相当重要。人力招聘直接影响公司员的素质，但传统的人力资源管理方法已经不适应高科技公司的需要。高科技行业知识不断变化，作不易定界，跨职能任务较多，作过程趋于多元化。这些因素都对员素质提出了更高的要求，依靠传统方法获知竞聘者是否能够胜任作变得比较困难。

采用决策树挖掘出人员选拔规则 CHAID

Decision tree for predicting job performance

Improving education

Improving teaching and learning n n Instructors can have trouble identifying their real difficulties in learning. Based on the students’ testing records, the system works to identify and find those problems, and then comes up with its suggestions for designing new teaching strategies. Assist teachers to identify students’ specific difficulties and weaknesses in learning. Helps the student to find out his or her weak points in learning and offers improvement recommendations.

ESL recommender teaching and learning

Right/wrong answer statistical table For every student, the system creates a right/wrong answer statistical table: a wrong answer is represented by 1 and a right answer by 0.

Summary table of students’ wrong answers The right/wrong answer statistical tables for respective students are integrated in a summary table of students’ wrong answers, and the sum values in the table are then ranked in descending order so as to show the descending degrees of weaknesses the students have collectively.

Hierarchical clustering algorithm is then applied to data collected to segment the students into a certain number of clusters, or categories, each of which includes students sharing the same or similar characteristics.

All students’ right/wrong answer statistical tables

Clustering analysis n A clustering analysis is made of the data in All students’ right/wrong answer statistical tables. It is evident that the students whose numbers are enclosed in the following separate parentheses belong to different clusters respectively: (9, 15, 6, 17, 13, 19, 14, 5); (22, 23, 4, 3, 21, 11, 24, 20, 7, 1); (12, 18, 2, 8, 25, 10, 16).

搜索引擎优化

搜索引擎优化 n n n They are usually not search engines by themselves. The clustering engine uses one or more traditional search engines to gather a number of results; then, it does a form of postprocessing on these results in order to cluster them into meaningful groups. The post-processing step analyzes snippets, i. e. , short document abstracts returned by the search engine, usually containing words around query term occurrences.

E-Commerce Recommender Systems

Background n n E-commerce has allowed businesses to provide consumers with more choices. Increasing choice, however, has also brought about information overload. E-commerce stores are applying mass customization principles to their presentation in on-line stores. One way to achieve mass customization in e-commerce is the use of recommender systems.

What is E-commerce Recommender Systems? n n Recommender systems are used by e-commerce sites to suggest products to their customers and to provide consumers with information to help them decide which products to purchase. In a sense, recommender systems enable the creation of a new store personally designed for each consumer(one-to-one marketing). Tool for database marketing and CRM

The Structure of Recommender Systems • A typical e-commerce recommender application includes the functional I/O, the recommendation method. Recommendation Method • Targeted customer inputs • Community inputs • Outputs

Targeted Customer Inputs n n explicit navigation inputs are intentionally made by the customer with the purpose of informing the recommender application of his or her preferences— keywords search，registration etc. Implicit inputs：specific item or items that the customer is currently viewing or those items in the customer's shopping cart(purchase history).

Community Inputs n n n community purchase history best-seller lists text comments

Output Recommendations n a set of suggestions：ordered list or unordered lists Ratings meta-rating: rating the comments themselves text comments item-to-item correlation user-to-user correlation n Top-N n Email marketing n n n

Delivery and Presentation n Push methods reach a customer who is not currently interacting with the system for example, by sending e-mail, recommendations for related products. Pull methods notify customers that personalized information is available but display this information only when the customer explicitly requests it. Other types of visualization.

Recommendation Methods n n n Statistical summaries of community opinion withincommunity popularity measures and aggregate or summary ratings Association analysis Content-based recommendations: The user will be recommended items similar to the ones the user preferred in the past; . Collaborative recommendations: The user will be recommended items that people with similar tastes and preferences liked in the past; Hybrid approaches: These methods combine collaborative and content-based methods. Examples?

Techniques for Recommendation n n Many techniques from data mining can be adapted to the scalability problem for recommender systems: nearestneighbor, classifiers(rule induction, neural networks, and Bayesian networks), clustering, association Web usage mining and more general commerce-related data mining may reveal techniques for exploiting complex behavioral data.

Effectiveness of Recommendations n n n Developing good metrics to measure the effectiveness of recommendations. The performance evaluation of recommendation algorithms is usually done in terms of coverage and accuracy metrics. Coverage measures the percentage of items for which a recommender system is capable of making predictions; Accuracy measures can be either statistical or decision-support.

E-commerce Recommender Applications n n n E-recommender systems enhance Ecommerce sales (2 %-8 %) in the following ways: Converting Browsers into Buyers Increasing Cross-sell Building Credibility through Community Inviting customers back Give the type of feedback needed for marketing professionals

Issues and Challenges(1) n Scalability and real-time performance: a large Web site must produce each recommendation within a few tens of milliseconds while serving hundreds or thousands of consumers simultaneously. n n n the extreme sparsity of ratings in recommender systems Limited Content Analysis Overspecialization New User Problem not enough data is also a challenge for recommender systems Intrusiveness

Issues and Challenges(2) n Incorporating Rich Data: explicit ratings and simple behavioral data such as purchases, click stream and many other types of data n Hybrid recommendation

Privacy Issues n n E-commerce sites may learn a great deal about customers without the customers' awareness or consent. Privacy policies Technological approaches for automating enforcement of privacy policies: Anonymizing techniques, the Platform for Privacy Preferences (P 3 P) Other issues…

Background n n E-commerce has allowed businesses to provide consumers with more choices. Increasing choice, however, has also brought about information overload. E-commerce stores are applying mass customization principles to their presentation in on-line stores. One way to achieve mass customization in e-commerce is the use of recommender systems.

Recommender systems n Recommender systems form a specific type of information filtering technique that attempts to predict and present items (movies, music, books, news, images, web pages) a user may be interested in.

What is E-commerce Recommender Systems? n n Recommender systems are used by e-commerce sites to suggest products to their customers and to provide consumers with information to help them decide which products to purchase. In a sense, recommender systems enable the creation of a new store personally designed for each consumer(one-toone marketing). Tool for database marketing and CRM

The Structure of Recommender Systems • A typical e-commerce recommender application includes the functional I/O, the recommendation method. Recommendation Method • Targeted customer inputs • Community inputs • Outputs

Targeted Customer Inputs n n explicit navigation inputs are intentionally made by the customer with the purpose of informing the recommender application of his or her preferences— keywords search，registration etc. Implicit inputs：specific item or items that the customer is currently viewing or those items in the customer's shopping cart(purchase history).

Community Inputs n n n community purchase history best-seller lists text comments

Recommendations Output n a set of suggestions：ordered list or unordered lists Ratings meta-rating: rating the comments themselves text comments item-to-item correlation user-to-user correlation n Top-N n Email marketing n n n

Delivery and Presentation n Push methods reach a customer who is not currently interacting with the system for example, by sending e-mail, recommendations for related products. Pull methods notify customers that personalized information is available but display this information only when the customer explicitly requests it. Other types of visualization.

Recommendation Methods n n n Statistical summaries of community opinion withincommunity popularity measures and aggregate or summary ratings Association analysis Content-based recommendations: The user will be recommended items similar to the ones the user preferred in the past; . Collaborative recommendations: The user will be recommended items that people with similar tastes and preferences liked in the past; Hybrid approaches: These methods combine collaborative and content-based methods Social network-based social recommender system

Social recommender systems n n Social network is a very important source of information to profile users. Taking part into social relationships may cause individuals to modify their attitudes and behaviors. Individuals become interested in topics or subjects that do not necessarily match their personal preferences and tastes, but that reflect those of their social network.

Social psychology n n Joining in a network with other people exposes individuals to social dynamics which can influence their attitudes and behaviors. Individuals become interested in topics or subjects that do not necessarily match their personal pre-existing preferences and tastes, but that reflect those of the network.

Theories of social influence n n n Social Conformity: people belonging to a group usually experience a “pressure to conform”, namely, they tend to change their attitudes and behaviors to match the expectations of the other members. Social Comparison: people who are new to a certain context or are not expert of certain domain should be interested in topics which reflect the opinions of other individuals in their network. Social Facilitation: people who are interested in a certain topic, but lack strong motivation, should appreciate information showing that other people in their network share their interest.

So. NARS algorithm

Techniques for Recommendation n n Many techniques from data mining can be adapted to the scalability problem for recommender systems: nearestneighbor, classifiers(rule induction, neural networks, and Bayesian networks), clustering, association Web usage mining and more general commerce-related data mining may reveal techniques for exploiting complex behavioral data.

A recommender system for the cosmetic business n n n First of all, we can categorize the customers with the clustering algorithm. Then we use content-based method to set a basic average score for each product in each cluster. Finally, we apply the association algorithm to the transaction data assign more recommender scores to existing associated products.

A location-aware recommender system for mobile shopping environments n When receiving a service request, the on-line subsystem generates a list of possibly interesting web pages based on the customer’s interests profile, vendor data, and the instantaneous position of the customer provided by the location manager.

Other application fields n n n Recommending conference paper submission to reviewing committee members Smart recommendation for an Evolving E-learning system Recommendation of citations for research papers Knowledge recommender system ……

Effectiveness of Recommendations n n n Developing good metrics to measure the effectiveness of recommendations. The performance evaluation of recommendation algorithms is usually done in terms of coverage and accuracy metrics. Coverage measures the percentage of items for which a recommender system is capable of making predictions; Accuracy measures can be either statistical or decision-support.

E-commerce Recommender Applications n n n E-recommender systems enhance Ecommerce sales (2 %-8 %) in the following ways: Converting Browsers into Buyers Increasing Cross-sell Building Credibility through Community Inviting customers back 日用消费品 Give the type of feedback needed for marketing professionals

Issues and Challenges(1) n Scalability and real-time performance: a large Web site must produce each recommendation within a few tens of milliseconds while serving hundreds or thousands of consumers simultaneously. n n n the extreme sparsity of ratings in recommender systems Limited Content Analysis Overspecialization New User Problem not enough data is also a challenge for recommender systems Intrusiveness

Issues and Challenges(2) n Incorporating Rich Data: explicit ratings and simple behavioral data such as purchases, click stream and many other types of data n Hybrid recommendation

Privacy Issues n n E-commerce sites may learn a great deal about customers without the customers' awareness or consent. Privacy policies Technological approaches for automating enforcement of privacy policies: Anonymizing techniques, the Platform for Privacy Preferences (P 3 P) Other issues…

研讨题 n n 阅读后面参考文献，分析案例使用的数据挖掘方法以及解决的主要问题。结合自己的实践，说明所在岗位对商务智能的需求（针对软件程硕士）。

典型参考文献(1) n n n Chen-Fu Chien, Li-Fei Chen. Data mining to improve personnel selection and enhance human capital: a case study in high-technology industry. Expert Systems with Application, 2008, 34(1): 280 -290 Cristo´bal Romero, Sebastia´n Ventura, Enrique Garcı´a. Data mining in course management systems: Moodle case study and tutorial. Computers & Education 51 (2008) 368– 384 Yang, C. C. et al. , Improving scheduling of emergency physicians using data mining analysis, Expert Systems with Applications (2008), doi: 10. 1016/j. eswa. 2008. 02. 069 Jang Hee Lee, Sang Chan Park. Intelligent profitable customers segmentation system based on business intelligence tools. Expert Systems with Applications 29 (2005): 145– 152 Chih-Ming Chen, Ying-Ling Hsieh, Shih-Hsun Hsu. Mining learner profile utilizing association rule for web-based learning diagnosis. Expert Systems with Applications 33 (2007) 6– 22 Bong-Horng Vhu, Ming-Shian Tsai, Cheng-Seen Ho. Toward a hybrid data mining model for cluster retention. Knowledge-Based Systems 20 (2007) 703– 718

典型参考文献(2) n n n n Daniela Grigoria, Fabio Casatib, Malu Castellanos, et al. Business process intelligence. Computers in Industry 53 (2004) 321– 343 Dursun Delen, Christie Fuller, Charles Mc. Cann. Analysis of healthcare coverage: A data mining approach. Delen, D. et al. , Analysis of healthcare coverage: A data mining approach, Expert Systems with Applications (2007), doi: 10. 1016/j. eswa. 2007. 10. 041 Mei-Hua Hsu. Proposing an ESL recommender teaching and learning system. Expert Systems with Applications. 2008, 34（3）: 2102– 2110 Yi-Fan Wang, Ding-An Chiang, Mei-Hua Hsu, et al. A recommender system to avoid customer churn: A case study. Expert Systems with Applications, 2009, 36: 8071– 8075 倪日文, 徐晓飞, 邓胜春. 基于关联规则的零部件供应商选择优化. 计算机集成制造系统， 2004, 10(3): 317 -335 薛云皎 , 钱乐秋, 花鸣等. 一种基于关联挖掘的自适应构件检索方法. 电子学报， 2004, 32(12 A): 203 -206 Ting-Peng Liang, Yung-Fang Yang, Deng-Neng Chen, et al. A semantic-expansion approach to personalized knowledge recommendation. Decision Support Systems, 2008 , 45(3): 401 -412 Giansalvatore Mecca, Salvatore Raunich, Alessandro Pappalardo. A new algorithm for clustering search results. Data & Knowledge Engineering 62 (2007) 504– 522

典型参考文献(3) n n n n Schafer, J. B. , Konstan, J. A. , and Riedl, J. 1999. Recommender Systems in ECommerce. In ACM Conference on Electronic Commerce (EC-99), pages 158 -166. Resnick and Varian. Recommender systems. Communications of the ACM , 1997 , 40 (3) : 56 – 58 SCHAFER J B , KONSTAN J , RIEDL J. Recommender systems in e - commerce. Proceedings of the First ACM Conference on Electronic Commerce. Denver , CO , 1999. 158 - 166. BEN J , KONSTAN J A , JOHN R. E - commerce recommendation applications. University of Minnesota , 2001, pp 1 -24 Yu Li, Liu Lu. Research on personal ized recommendations in E- business. Computer Integrated Manufacturing Systems, 2004，10（10）： 1306 -1313(in Chinese) TRAN T , COHEN R. Hybrid recommender systems for elec 2 tronic commerce [ R ]. Knowledge - Based Electronic Markets , Papers from the AAAI Workshop , AAAI Technical Report WS- 00 - 04. Menlo Park , CA : AAAI Press , 78 - 83.

典型参考文献(4) n. Gediminas Adomavicius, Alexander Tuzhilin. Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions. IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING, 2005, 17(6): 734 -749 n. Gediminas Adomavicius，Alexander Tuzhilin. Using data mining methods to build customer profiles. IEEE Computer, 2001: 74 -82 n. G. -J. Houben et al. (Eds. ): UMAP 2009, LNCS 5535, pp. 223– 234, 2009 n. Yi-Fan Wang, Yu-Liang Chuang, Mei-Hua Hsu, et al. A personalized recommender system for the cosmetic business. Expert Systems with Applications 26 (2004) 427– 434 n. Wan-Shiou Yang *, Hung-Chi Cheng, Jia-Ben Dia. A location-aware recommender system for mobile shopping environments. Expert Systems with Applications 34 (2008) 437– 445 n. Chumki Basu, Haym Hirsh, William W. Cohen, et al. Technical paper recommendation: a study in combining multiple information sources. Journal of Artificial Intelligence Research, 1(2001): 231 -253 n. T. Tang and G. Mc. Calla. Smart recommendation for an evolving e-learning system: Architecture and experiment. International Journal on e-Learning, vol. 4, pp. 105 -129, 2005 n. Sean M. Mc. Nee, Istvan Albert, Dan Cosley, et al. On the recommending of citations for research papers. CSCW’ 02, 2002: 16 -20 n. Lu Zhen, George Q. Huang, Zuhua Jiang. An inner-enterprise knowledge recommender system. Expert Systems with Applications, 37(2010): 1703 -1712 n. Francesca Carmagnola, Fabiana Vernero, Pierluigi Grillo. So. NARS: a social networksbased algorithm for social recommender systems. G. -J. Houben et al. (Eds), LNCS 5535, 2009: 223 -234

数据挖掘应用 赵卫东 博士 复旦大学 客户关系管理 CRM 顾客生命周期

数据挖掘应用赵卫东博士复旦大学客户关系管理 CRM 顾客生命周期