User Modeling and Machine Learning A Survey Thierry

User Modeling and Machine Learning: A Survey Thierry Artières LIP 6 – Université Paris 6 – France MMDSS Nato - 11/09/2007 T. Artières - 1

User modeling Goals: Analyze and model user’s knowledge, goals, preferences, . . = User Model Mandatory step for human-computer interaction personalization and adaptation User model collected/built in either an intrusive or a non intrusive way Non intrusive techniques allow automatic gathering of information about the user Artificial Intelligence techniques, Data Mining, Machine Learning algorithms MMDSS Nato - 11/09/2007 2

Outline Main applications What are User models? Web Log Processing Dealing with sequences Building a generic user model from a document corpus Hypermedia Navigation Behaviour Detection and Tracking MMDSS Nato - 11/09/2007 3

Main Applications Intelligent User Interfaces Adaptive Hypermedia Educational Hypermedia Recommender Systems Website Personalization Web Analytics Personalized Information Retrieval Prediction of Next Action Desktop User Help Systems Office Human Activity help systems MMDSS Nato - 11/09/2007 4

Intelligent User Interfaces History: 1960 s: First GUI (Graphical User Interface) + mouse, Multi windows, hypertext 70 s technology driven focus on interfaces 80 s users starts becoming focus of attention 95: Intelligent agents and recommender systems appear on Internet 97: Microsoft “Intelligent” Office Assistant Interest of IUI Increasing number of applications Applications more an more complex Consequences Learn and remember lots of functions with their “how to do it” Repeat many times long sequences of actions to perform a given task MMDSS Nato - 11/09/2007 5

Example: The Lumiere project: Office’ 97 Assistant [Horwitz] Reasoning under uncertainty about the goals of a software user Bayesian networks for encoding links between goals and needs Goals = target tasks or subtasks at the focus of a user's attention Needs=information or automated actions required to achieve the goals BNs structure determined by hand MMDSS Nato - 11/09/2007 6

Adapative Hypermedia Websites, cdrom etc. Personalization based on a user model Gathered automatically or not Through user actions, forms filling etc AH builds a model of the User Example Goals (looking an information, wanting to know a little bit about…) Preferences (text/video) Interests, Knowledge, … Natural History Museum website Many visitors With various backgrounds Kids, Professors, With various goals Seeking a precise information / interested in general information on the domain / wanting to know open hours Personalization may consist in Personalizing content (summarizing) Personalizing structure (link recommendation, link hiding) MMDSS Nato - 11/09/2007 7

Main problems Traditional drawbacks of Adaptive systems are unpredictable and less transparent User does not have control anymore What to do with mistakes? In case. . who is responsible? Privacy? Usually requires ad-hoc resources Domain model Represents knowledge about the concepts of the application and of their relations E. g. Mathematical foundations Addition, subtraction, multiplication, division Addition is related (similar and pre-requisite) to subtraction, multiplication idem for division Used to perform inference in the User model A user that is known to understand subtraction should understand addition too At least Meta-data information MMDSS Nato - 11/09/2007 8

Personalization strategies Related fields Adaptive Interface Design Interface Evaluation MMDSS Nato - 11/09/2007 9

Related: Adaptive interface Main screen Touch screen Example: Adaptive multimodal interface Defining contextual buttons Defining multimodal interaction strategy Track Ball Keyboard E. g. : Use of modalities - per user - per action type MMDSS Nato - 11/09/2007 10

Educational Systems Historically precedes UM birth One of the emblematic application UM field Many systems Most hand-made: Tools to build Educational Hypermedia Example of AHA! An Hypermedia architecture [De Bra] Allows building a educational hypermedia (course etc) Domain model: choice of concepts and their relations Overlay user models Multiple attributes per concept User model update rules Concept attributes change with user navigation Adaptation rules Hiding links if concepts (prerequisite not satisfied) Content personalization by conditional inclusion of fragments Introduction of stability feature (to force respecting editor’s view): “always adapted” up to “stable” MMDSS Nato - 11/09/2007 11

Recommender Systems Goal Assist users in a decision-making process, choosing one item amongst a potentially overwhelming set of alternative products or services (when insufficient personal experience) Applications E-commerce / Digital Libraries Item (Product) recommendation Industrial Market Amazon. com, CDNow!, EBay, Moviefinder. com etc. First System around 1980 [Rich] Based on stereotypes Usually A new user starts by rating few items Recommendations are based on his ratings and on ratings by others Recommending items similar users liked Recommending items similar to what the user likes MMDSS Nato - 11/09/2007 12

Main approaches Collaborative filtering Most successful (Amazon. com, CDnow. com, Movie. Finder. com) Exploit all Users ratings to make prediction for a given user Explicit or implicit ratings (e. g. time-spent reading) Content-based filtering (text-based products) Compare representations of an item and of items that have interested a particular user Prediction models Demographic-based, utility-based + Hybrid methods Specific problems Clustering users based on incomplete data (Very) Large Number of Categories / Items Few training data Tunnel phenomenon (content-based filtering) Recommendation based on interest to the user Spam Always recommend much similar items => no discovery Crawling attack for altering the system’s recommendation behaviour wrt a single item MMDSS Nato - 11/09/2007 13

Web. Site log analysis – Web analytics solutions Web server log analysis Many existing systems Free, shareware, commercial products Webtrends, openmarket, net. Genesis, Elytics… Outputs range from Basic log analysis “Simple” statistics Number of accesses to individual files Unique visitors Times of visits To much more valuable information (integrated solutions) Categorization of the users / Clustering Measuring the effect of a marketing campaign Browse-to-buy rate Advertisement click-through rate MMDSS Nato - 11/09/2007 14

All users Size of observed population Panel users My Website Logs Group of Websites All Websites Executables Number of observed websites Forms Richness of information Log server data Panel data Anaweb project Goal: Site-centric solution that combines Poor log server: huge quantity Technological bias Rich logs for panel users: limited Statistical bias Notions of Network (selected panel users) Galaxy Set of sites in the sphere of the targeted website (sites concurents, partners, portals, search engines, …) Expected Personalize pop-up/marketing offers Behaviour analysis on concurrent sites Discovery of valuable partnerships Air company car rental MMDSS Nato - 11/09/2007 15

Personalized Information Retrieval Textual search engines e. g. search for “Java” Language / Island ? Integrate user model in existing search engines Modify search using weighted tf-idf Personalized page-rank Sort Google© results according to user model Use relevance feedback Other search engines (e. g. image, video) Weakness of search engines makes interaction acceptable for users Solutions Putting the User in the loop Already interactive search Helps the user in his search Towards user modeling integration Social media sites (Flickr, Delicious, Cite. ULike, You. Tube) Alike recommender systems without explicit ratings Use of contacts (collaborative Filtering) or metadata like tags on image (collaborative and content based Filtering) MMDSS Nato - 11/09/2007 16

Implicit User feedback for IR based on eyemovements NIPS 2005 challenge Simulation of search engine use Query 10 answers/titles 1 correct 4 relevant 5 not relevant Techniques (70% accuracy) Conditional Random Fields Grammatical Inference … MMDSS Nato - 11/09/2007 17

Example Not relevant Non Pertinent Correct Non Pertinent Not relevant Non Pertinent Relevant Pertinent Not relevant Non Pertinent Relevant MMDSS Nato - 11/09/2007 18

Other applications Intelligent routing in P 2 P networks MMDSS Nato - 11/09/2007 19

Prediction (of Next Action) More defined problems (from the machine learning point of view) Applications Shell (Unix) help systems Internet prefetching Interface design and personalization Contextual buttons Discovering macro buttons Methods Extensions of standard ML techniques: Markov models etc. Problems Most often highly semantic nominal observation sequences E. g. Unix command / page visited / category of page / action on an interface Not so easy to take into account Ways to integrate various knowledge E. g. Internet prefetching User logs Website structure Page content, category and metadata MMDSS Nato - 11/09/2007 20

Navigation Help Systems Recommending links in the whole Web Much difficult task Problem of inference for what has never been seen Inherits from Adaptive hypermedia Recommender systems Prediction systems (of next action) Examples of Two Web recommender systems Web. Watchers (Joachims 98) WEBIC (Zhu et al. 2003) MMDSS Nato - 11/09/2007 21

Web. Watcher [Joachims 98] Tour guide of the WWW: Next page suggestion Web. Watcher acts as a proxy Display Web pages embedded in Web. Watcher’s additions An Original Webpage is displayed with highlighted links and embedded in a page with Webwatcher commands Gathering Logs Learning Start page at CMU: ask brief description of user’s interests: search keywords Follow the User’s navigation on the Web All along the session replace every URL in a page by a modified page on CMU server P(Link/Current Page, User Interest) Few methods Annotate hyperlinks that are followed by user’s search keywords Annotate hyperlinks with words encountered in pages downstream of it Use Reinforcement Learning where Reward is related to TFIDF of a search keyword Link suggestion E. g. Proximity between user’s interest and annotation of hyperlinks MMDSS Nato - 11/09/2007 22

Web IC [Zu, 2003] Client-side Web recommender system Predicts the user's information need based on his browsing patterns Points the user to webpages that contain information useful to him Information Content (IC) words and pages Example: A user examine P 1 goes to P 2 (anchor Dolphins) Click back immediately to P 1 Goes to P 3 (anchor Whale) Continue following links from P 3 Detecting IC-words Classifier using browsing features #Title, #Hyperlink, #Back, Recommending pages Based on IC-words in it. [Zu, 2003] MMDSS Nato - 11/09/2007 23

Desktop User Help System Multi-Tasking According to CHI studies A user works on multiple tasks in parallel Average length of an episode devoted to a task: 12’ Around 10 tasks a day Information needed for each task fragmented across multiple application programs Few efforts to detect task switches Automatic method Use of text in the active window as low level features Hierarchical model Tasks modeled as Hidden Markov Models / Gaussian Mixtures Succession of tasks modeled as Markov Chain Semi-automatic Task. Tracer [Diettrich] helps the user define a hierarchy of tasks and associated resources Goal: detect task switch to configure computer for the current task Components Classifier for the current task, based on active window features MMDSS Nato - 11/09/2007 24

Office Activity Help Systems Recognition of office activities [Oliver, Horwitz] State of user’s activity based on video, acoustic, and computer interactions Phone conversation, Face to Face conversation, Nobody present… (98% accuracy) Hierarchical representations Two level Dynamical models (HMMs) architecture Operate on different level of temporal detail First level: Audio HMMs, Video HMMs. E. g. Audio HMMs: Speech, music, Phone ringing, keyboard typing Second level HMMs (one / activity) operating on outputs of first level + additional information (sound localization, etc) Learning and inference Layers connected via inferential results Trained independently MMDSS Nato - 11/09/2007 25

User models A user model contains information about a user/group that the system believes to be true all information useful for improving communication between the user and the system much dependent on the system and the kind of improvements one is seeking Often defined or updated based on additional models Domain model Represents knowledge about the concepts of the application and of their relations = graph of concepts (eventually typed links) Task model Mostly in Industrial interfaces E. g. maritime surveillance Finite state automata describing how the user should do a task MMDSS Nato - 11/09/2007 26

Task model Model of how to do complex actions i. e. Sequence of actions Use: Sensitive tasks and applications Helps determine / prevent the deviation of a user behaviour with the reference behaviour Helps personalizing MMDSS Nato - 11/09/2007 27

User Models Practically almost as many user model definitions as there are systems exploiting information about the user Some trends and general ideas may serve as a basis for building more complex and accurate user models Implicit (or not understandable) user models Recommender systems Set of ratings Information retrieval Most significant words that may represent the user needs MMDSS Nato - 11/09/2007 28

Educational hypermedia and tutorial systems Overlay user models represent a user knowledge and/or interest in a concept space vectors of attributes, one for each concept in the domain model. updated from user navigation logs according to the domain model (e. g. Bayesian Networks) Stereotype user models E. g. Novice, Medium, Expert Adaptive systems for any single website or hypermedia Wider domain model Implicit (e. g. current state in a behaviour Markovian model) Educational Hypermedia like Naïve approach: domain model organized as a hierarchy which is derived from the website structure Domain model provided by the hypermedia editors Overlay user models Personalized Information retrieval like Important keyword , Multimodal Interface, Office activity help systems, … MMDSS Nato - 11/09/2007 29

A general Multi-Layer User Model The dynamic user Time-varying individual Detecting and tracking the user behaviour is an on-line task Dynamic statistical models Often: pre-determined behaviour taxonomy The anonymous user All the users act similarly Clustering individuals The unique user Unique individual with his/her own knowledge, interests, preferences, Often explicitly asked OR inferred from the user actions. Usually represented in a ad-hoc way. MMDSS Nato - 11/09/2007 30

Some more details about… Web Log processing Sequential data processing Navigation behaviour detection and tracking MMDSS Nato - 11/09/2007 31

Web log preprocessing Web server logs IP address Time Request Status Size Referrer Agent Few steps Preprocessing - Log analysis - Web usage mining Terminology Web site User Collection of interlinked Web pages including a host page, residing at the same location Individual accessing files form a Web server using a browser User session Delimited set of user clicks across one or more Web servers Server session or visit Collection of user clicks to a single Web server during a user session Pageview Visual rendering of a Web page May consist of several items (frames, text, graphics, scripts) Clickstream A sequential series of pageviews requests MMDSS Nato - 11/09/2007 32

Web log preprocessing 1: Data preparation Existing noise Browser and proxy caching Existing techniques to overcome not always efficient User registration, cookies, techniques preventing browsers cache Cleaning logs Many implicit requests Presence of images Removing non relevant entries with a dictionary of suffixes (. jpg etc) Possible errors Removing depends on what one wants to do/discover in logs MMDSS Nato - 11/09/2007 33

Web log preprocessing 2: User identification Much heuristic Use the agent (browser) Although (IP, agent) do not always define uniquely a user A user may access from different computers Many users may access from the same computer Consider the time between two requests from the same (IP, agent) Consecutive accesses from the same host within a given time period same user Rebuild sessions based on Server logs Referrer field if available Website Structure if available Web log preprocessing 3: Session identification Long period of time: Use timeout (30’) MMDSS Nato - 11/09/2007 34

Many other kind of logs in UM applications Raw data Web logs (Server logs, Client logs) Application logs Low level signals (speech, gestures, eye-tracking) multimodal logs Sequence of Actions Preprocessed data Raw data Feature vectors sequence of feature vectors set of sequences of feature vectors etc. MMDSS Nato - 11/09/2007 35

Standard Web Usage Mining technique: Association rules Co-occurrences of items in a sequence X Y, with associated Support and Confidence. Goals Detecting frequent items Example from [Cooley 1999] IBM analysis of the server log of the official 1996 Olympics web site 45% of visitors who accessed a page about Indoor Volleyball also accessed a page on Handball. 59. 7% of visitors who accessed a page about Badminton also accessed a page about Table Tennis Percentage are confidences With sequentiality = sequential pattern discovery 9. 81% of the site visitors accessed the Atlanto home apge followed by the Sneakpeek main page 0. 42% of the site visitors accessed the Sports main page followed by the Schedules main page Percentages are supports MMDSS Nato - 11/09/2007 36

Dealing with sequences Tasks Prediction Clustering Prediction Many Markov models Action=State Markov Chain [Davison] Predicting sequences of user actions / Unix commands People tend to repeat themselves Accuracy around 40%, top-5: 70% Adapted on-line MC with exponential decay Markov models [Zukerman] Presending pages: Prediction of next page and next access time Time-Space Markov State includes previous page + referrer, or two previous pages MMDSS Nato - 11/09/2007 37

Clustering Problems Nature of observations Highly semantic observations Nominal observations Representation choice E. g. Web sessions: Node = page ; node = group of pages; node = complex function of navigation history Techniques Pattern Recognition Define an application-based distance between sequences with standard clustering E. g. DTW with file path based distance between pages Eventually represent as a fixed dimension feature vector ML techniques Model-driven (e. g. HMM based) Learn a mixture models: Learn a Probabilistic Grammar + interpretation, e. g. identify user’s preferred trails MMDSS Nato - 11/09/2007 38

Model driven approaches (HMM, MC, MS-HMM) Problems Evaluation Topology design Nature of observations Solutions Topology learning in a restricted model space Integrating a priori knowledge about symbols similarities E. g. smoothing pdf in HMM estimation MMDSS Nato - 11/09/2007 39

Defining similarities between observations Sim(P 1, P 2)=sim. Status(P 1, P 2)xsim. Type(P 1, P 2)xsim. Topic(P 1, P 2) STATUS 1=>competitors 2=>partners 3=>clients 4=>other sites 5=>unknown status/status 1 2 3 4 5 1 1 0. 01 0. 2 2 0. 01 1 1 0. 2 3 0. 01 1 1 0. 2 4 0. 2 1 0. 2 5 0. 2 1 typ es 1 2 3 4 5 6 7 8 9 10 11 1 S 9 S 7 S 8 S 7 S 7 S 9 S 7 2 S 9 S 1 S 9 S 9 S 9 3 S 7 S 9 S 1 S 5 S 9 S 5 4=>Administration 4 S 8 S 9 S 5 S 1 S 4 S 9 S 9 S 2 TYPES 1=>Portal 2=>e-commerce 3=>Content 5=>Services 5 S 7 S 9 S 5 S 4 S 1 S 9 S 7 S 9 S 4 6=>Média 6 S 7 S 9 S 1 S 9 S 9 S 5 S 9 7=>Services 2 7 S 9 S 5 S 4 S 1 S 9 S 7 S 9 S 4 8=>Search engine 8 S 9 S 9 S 1 S 9 S 2 S 9 9 S 9 S 5 S 9 S 7 S 5 S 7 S 9 S 1 S 9 10 S 9 S 9 S 2 S 9 S 1 S 9 11 S 7 S 9 S 5 S 2 S 4 S 9 MMDSS Nato - 11/09/2007 S 4 S 9 S 9 S 1 9=>FAI 10=>Annuaire 11=>Institutionnal 40

User navigation behaviour detection and tracking Classification / Clustering of Users Navigation Help Systems Detection and Tracking of User Behaviour Statistical sequence models Available Databases Discovering Micro behaviours Help strategy Link recommendation Pre. Processing Rich Logs (Spy agent) Representation level One feature vector per page or per 30 seconds period Features Reading / Activity features Ressources accessed features Path features Semantic Features Content features Dynamic features (Dfeatures) MMDSS Nato - 11/09/2007 41

Reading and Activity Features Time spent on a page or on parts of the page (via Scrolling) Actions (mouse moves, scroll, print etc) Ressources accessed features Video or other animation accessed via a link Images, text, etc, drawings. Normalization features Curent page size, #out links, #images Simple Path features Percentages of moves Backward/Forward/Bookmark #Visits for a given page #Forward since the last Backward, # Backward / #moves Pathiness Ringiness Loopiness Spikiness MMDSS Nato - 11/09/2007 42

Links features Level (Foreground / Background) Explicit / Implicit Functionality (Orientation/ Information/…) Nature (Structural / Semantic / ) Media (Icone / Sound / Image / Video / Text /) Content features Focus on concepts associated to last pages visited without sequence matter Distance between concepts associated to last pages visited with sequentiality

Behaviour categorization Supervised study Generic behaviour from [Canter] Wandering Browsing Scanning Exploring Searching Labelled database collection Ask users to fill forms by browsing the hypermedia Finding answers induce navigation behaviours Find a particular information (e. g. date, name)? What is your favorite image in this particular part of the hypermedia? MMDSS Nato - 11/09/2007 44

Problem of navigation behaviour taxonomies Too generic behaviours No help Needs for a dictionary of micro-behaviours Example: Learning micro-behaviour with left-right HMM components Related to hierarchical HMM learning Rewriting of original sessions as sequences of micro-behaviours MMDSS Nato - 11/09/2007 45

Post learning interpretation of micro-behaviours The user takes time reading TOC then goes quickly to the searched information The user does not loose time with TOC and rather spends time on few content pages Search Micro-behaviour Disordered browsing microbehaviour MMDSS Nato - 11/09/2007 46

MMDSS Nato - 11/09/2007 47

References [Brusilovsky, 1996], P. Brusilovsky, Methods and Techniques of Adaptive Hypermedia, User Modeling and User Adapted Interaction, 1996, Vol. 6, n° 2 -3, pp. 87 -129. [Canter 1995], Canter, Rivers, Storrs, 1995, Characterizing user navigation through complex data structures, Behavior and information technology, vol 4. [Cooley 1999], R. Cooley, B. Mobasher, J. Srivastava, Data Preparation for Mining World Wide Web Browsing Patterns, Knowledge and Information Systems, 1999. [Davison 1998], B. Davison, H. Hirsh, Predicting Sequences of User Actions, AAAI/ICML Workshop on Predicting the Future: AI Approaches to Time-Seris Analysis, 1998. [De Bra, 2003], P. De Bra, B. Berden, B. De Lange, B. Rousseau, Aha! The Adaptive Hypermedia Architecture, HT, 2003. [Eirinaki 2003], M. Eirinaki, M. Vazirgiannis, Web Mining for Web Personalization, ACM Transactions on Internet Technology, Vol. 3, N 0 1, February 203, pp 1 -27. [Horwitz] E. Horvitz, J. Breese, D. Heckerman, D. Hovel, K. Rommelse, The Lumiere Project: Bayesian User Modeling for Inferring the Goals and Needs of Software Users, Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, July 1998. [Joachims, 1998], T. Joachims, D. Freitag, T. Mitchell, Web. Watcher: A Tour Guide for the World Wide Web, IJCAI, 1998. [Oliver and Horwitz, 2005], N. Oliver and E. Horwitz, A comparison of HMMs and Dynamic Bayesian Networks for Recognizing Office Activities, User Modeling, 2005. [Shen, 2007], J. Shen, L. Li, T. G. Diettrich, Real-Time Detection of Task Switches of Desktop Users, IJCAI, 2007 [Shen, 2005], X. Shen, B. Tan, C. Zhai, Implicit User Modeling for Personalized Search, CIKM, 2005. [Slaney, 2003], M. Slaney, J. Subrahmonia, P. Maglio, Modeling Multitasking Users, User Modeling, 2003 [Zukerman 1999], I. Zukerman, D. Albrecht, A. Nicholson, Predicting Users’ Requests on the WWW, User Modeling, 1999. [Zu, 2003], Zhu T. , Greiner R. , Haübl G. , Learning a model of a web user’s interests, User Modeling 2003, pp 65 -75. MMDSS Nato - 11/09/2007 48