Link Analysis Adapted from Lectures by Prabhakar Raghavan

Link Analysis Adapted from Lectures by Prabhakar Raghavan (Yahoo, Stanford), Christopher Manning (Stanford), and Raymond Mooney (UT, Austin) Prasad L 21 Link. Analysis 1

Evolution of Search Engines n n n 1 st Generation : Retrieved documents that matched keyword-based queries based on boolean model. 2 nd Generation : Incorporated contentspecific relevance ranking based on vector space model (TF-IDF), to deal with high recall. 3 rd Generation: Incorporated contentindependent source ranking, to overcome spamming, and to exploit “collective web wisdom”. Prasad L 21 Link. Analysis 2

(cont’d) n n 3 rd Generation: Tried to glean relative semantic emphasis of various words based on syntactic features such as fonts, span of query term hits, etc. to enhance the efficacy of VSM. Future search engines will incorporate context, profile, and past query history associated with a user, to personalize search, and apply additional reasoning and heuristics to improve satisfaction of information need. Prasad L 21 Link. Analysis 3

Meta-Search Engines n Search engine that passes query to several other search engines and integrates results. n n n Submit queries to host sites. Parse resulting HTML pages to extract search results. Integrate multiple rankings into a “consensus” ranking. Present integrated results to user. Examples: Metacrawler, Savvy. Search , Dogpile 4

HTML Structure & Feature Weighting n Weight tokens under particular HTML tags more heavily: n n <TITLE> tokens (Google seems to like title matches) <H 1>, <H 2>… tokens <META> keyword tokens Parse page into conceptual sections (e. g. navigation links vs. page content) and weight tokens differently based on section. 5

Bibliometrics: Citation Analysis n n n Many standard documents include bibliographies (or references), explicit citations to other previously published documents. Using citations as links, standard corpora can be viewed as a graph. The structure of this graph, independent of the content, can provide interesting information about the similarity of documents and the structure of 6 information.

Bibliographic Coupling: Similarity Measure n n Measure of similarity of documents introduced by Kessler in 1963. The bibliographic coupling of two documents A and B is the number of documents cited by both A and B. Size of the intersection of their bibliographies. Maybe want to normalize by size of bibliographies? A B 7

Co-Citation : Similarity Measure n n n An alternate citation-based measure of similarity introduced by Small in 1973. Number of documents that cite both A and B. Maybe want to normalize by total number of documents citing either A or B ? A B 8

Impact Factor (of a journal) n n n Developed by Garfield in 1972 to measure the importance (quality, influence) of scientific journals. Measure of how often papers in the journal are cited by other scientists. Computed and published annually by the Institute for Scientific Information (ISI). The impact factor of a journal J in year Y is the average number of citations (from indexed documents published in year Y) to a paper published in J in year Y 1 or Y 2. Does not account for the quality of the citing article. 9

Authorities n n Authorities are pages that are recognized as providing significant, trustworthy, and useful information on a topic. In-degree (number of pointers to a page) is one simple measure of authority. However in-degree treats all links as equal. Should links from pages that are themselves authoritative count for more? 10

Hubs n Hubs are index pages that provide lots of useful links to relevant content pages (topic authorities). n Hub pages for IR are included in the course home page: n http: //www. cs. utexas. edu/users/mo oney/ir-course 11

Hubs and Authorities n Together they tend to form a bipartite graph: Hubs Authorities 12

Today’s lecture: Basics of 3 rd Generation Search Engines n Role of anchor text n Link analysis for ranking n Pagerank and variants (Page and Brin) n n Prasad BONUS: Illustrates how to solve an important problem by adapting topology/statistics of large dataset for mathematically sound analysis and a practical scalable algorithm HITS (Klienberg) L 21 Link. Analysis 13

The Web as a Directed Graph Page A Anchor hyperlink Page B Assumption 1: A hyperlink between pages denotes author perceived relevance (quality signal) Assumption 2: The anchor text of the hyperlink describes the target page (textual context) Prasad L 21 Link. Analysis 14

Anchor Text WWW Worm - Mc. Bryan [Mcbr 94] n For ibm how to distinguish between: n n n IBM’s home page (mostly graphical) IBM’s copyright page (high term freq. for ‘ibm’) Rival’s spam page (arbitrarily high term freq. ) “ibm” A million pieces of anchor text with “ibm” send a strong signal Prasad “ibm. com” “IBM home page” www. ibm. com L 21 Link. Analysis 15

Indexing anchor text n When indexing a document D, include anchor text from links pointing to D. Armonk, NY-based computer giant IBM announced today www. ibm. com Joe’s computer hardware links Compaq HP IBM Big Blue today announced record profits for the quarter 16

Motivation and Introduction to Page. Rank n n Why is Page Importance Rating important? n New challenges for information retrieval on the World Wide Web. n Huge number of web pages: 150 million by 1998 1000 billion by 2008 n Diversity of web pages: different topics, different quality, etc. What is Page. Rank? n A method for rating the importance of web pages objectively and mechanically using the link structure of the web.

Initial Page. Rank Idea n Can view it as a process of Page. Rank “flowing” from pages to the pages they cite. . 1 . 08 . 05. 03 . 09 . 08 . 03 22

Sample Stable Fixpoint 0. 4 0. 2 0. 4 23

Pagerank scoring : More details n Imagine a browser doing a random walk on web pages: 1/3 n n n 1/3 Start at a random page 1/3 At each step, go out of the current page along one of the links on that page, with equal probability “In the steady state” each page has a longterm visit rate - use this as the page’s score. Prasad L 21 Link. Analysis 24

Not quite enough n The web is full of dead-ends. n n Random walk can get stuck in dead-ends. Makes no sense to talk about long-term visit rates. ? ? Prasad L 21 Link. Analysis 25

Teleporting n n At a dead end, jump to a random web page. At any non-dead end, with probability 10%, jump to a random web page. With remaining probability (90%), go out on a random link. n 10% - a parameter. n Prasad L 21 Link. Analysis 26

Result of teleporting Now cannot get stuck locally. n There is a long-term rate at which any page is visited (not obvious, will show this). n How do we compute this visit rate? n Prasad L 21 Link. Analysis 27

Markov chains n n n A Markov chain consists of n states, plus an n n transition probability matrix P. At each step, we are in exactly one of these states. For 1 i, j n, the matrix entry Pij tells us the probability of j being the next state, given we are currently in state i. Pii>0 is OK. Prasad i Pij j 28

Markov chains n n n Clearly, for all i, Markov chains are abstractions of random walks. Exercise: represent the teleporting random walk from 3 slides ago as a Markov chain. For this case: Prasad 29

Ergodic Markov chains n A Markov chain is ergodic if n n Prasad you have a path from any state to any other. For any start state, after a finite transient time T 0, the probability of being in any state at a fixed time T>T 0 is nonzero. Not ergodic (even/ odd). 30

Ergodic Markov chains n n n For any ergodic Markov chain, there is a unique long-term visit rate for each state. n Steady-state probability distribution. Over a long time-period, we visit each state in proportion to this rate. It doesn’t matter where we start. Prasad L 21 Link. Analysis 31

Probability vectors n n A probability (row) vector x = (x 1, … xn) tells us where the walk is, at any point. E. g. , (000… 1… 000) means we’re in state i. 1 i n More generally, the vector x = (x 1, … xn) means the walk is in state i with probability xi. Prasad 32

Change in probability vector n n n If the probability vector is x = (x 1, … xn) at this step, what is it at the next step? Recall that row i of the transition prob. matrix P tells us where we go next from state i. So from x, our next state is distributed as x. P. Prasad L 21 Link. Analysis 33

Steady state example n The steady state looks like a vector of probabilities a = (a 1, … an): n ai is the probability of being in state i. 1/4 1 3/4 2 3/4 1/4 For this example, a 1=1/4 and a 2=3/4. Prasad L 21 Link. Analysis 34

How do we compute this vector? n n n Let a = (a 1, … an) denote the row vector of steady-state probabilities. If current state is described by a, then the next step is distributed as a. P. But a is the steady state, so a=a. P. n So a is the (left) eigenvector for P. n Corresponds to the “principal” eigenvector of P with the largest eigenvalue. n n Prasad Transition probability matrices always have largest eigenvalue 1 (and others are smaller). Principal eigenvector can be computed iteratively from any initial vector. 35

One way of computing a n n n Recall, regardless of where we start, we eventually reach the steady state a. Start with any distribution (say x=(10… 0)). After one step, we’re at x. P; after two steps at x. P 2 , then x. P 3 and so on. “Eventually” means for “large” k, x. Pk = a. Algorithm: multiply x by increasing powers of P until the product looks stable. Prasad L 21 Link. Analysis 36

Link Structure of the Web n 150 million web pages 1. 7 billion links Backlinks and Forward links: ØA and B are C’s backlinks ØC is A and B’s forward link Intuitively, a webpage is important if it has a lot of backlinks. What if a webpage has only one link off www. yahoo. com?

A Simple Version of Page. Rank n n u: a web page Bu: the set of u’s backlinks Nv: the number of forward links of page v c: the normalization factor to make ||R||L 1 = 1 (||R||L 1= |R 1 + … + Rn|)

An example of Simplified Page. Rank (Transposed version) Page. Rank Calculation: first iteration

An example of Simplified Page. Rank (Transposed version) Page. Rank Calculation: second iteration

An example of Simplified Page. Rank (Transposed version) Convergence after some iterations

A Problem with Simplified Page. Rank A loop: During each iteration, the loop accumulates rank but never distributes rank to other pages!

An example of the Problem

Random Walks in Graphs n The Random Surfer Model n n The simplified model: the standing probability distribution of a random walk on the graph of the web. simply keeps clicking successive links at random The Modified Model n The modified model: the “random surfer” simply keeps clicking successive links at random, but periodically “gets bored” and jumps to a random page based on the distribution of E

Modified Version of Page. Rank E(u): a distribution of ranks of web pages that “users” jump to when they “gets bored” after successive links at random.

An example of Modified Page. Rank 48

Pagerank summary n Preprocessing: n n Given graph of links, build matrix P. From it compute a. The entry ai is a number between 0 and 1: the pagerank of page i. Query processing: n n n Prasad Retrieve pages meeting query. Rank them by their pagerank. Order is query-independent. L 21 Link. Analysis 49

The reality n Pagerank is used in google, but with so are many other clever heuristics. Prasad L 21 Link. Analysis 50

Pagerank: Issues and Variants n n How realistic is the random surfer model? n What if we modeled the back button? [Fagi 00] n Surfer behavior sharply skewed towards short paths [Hube 98] n Search engines, bookmarks & directories make jumps non-random. Biased Surfer Models n Weight link traversal probabilities based on match with topic/query (skewed selection) n Bias jumps to pages on topic (e. g. , based on personal bookmarks & categories of interest) Prasad 51

Non-uniform Teleportation Sports Prasad Teleport with 10% probability to a Sports page L 21 Link. Analysis 55

Hubs and Authorities Alternative to Pagerank Prasad L 21 Link. Analysis 60

Hyperlink-Induced Topic Search (HITS) - Klei 98 n In response to a query, instead of an ordered list of pages each meeting the query, find two sets of inter-related pages: n Hub pages are good lists of links on a subject. n n e. g. , “Bob’s list of cancer-related links. ” Authority pages occur recurrently on good hubs for the subject. Best suited for “broad topic” queries rather than for page-finding queries. Gets at a broader slice of common opinion. Prasad L 21 Link. Analysis 61

Hubs and Authorities n n n Thus, a good hub page for a topic points to many authoritative pages for that topic. A good authority page for a topic is pointed to by many good hubs for that topic. Circular definition - will turn this into an iterative computation. Prasad L 21 Link. Analysis 62

The hope Authorities Hubs Prasad Long distance telephone companies 63

High-level scheme Extract from the web a base set of pages that could be good hubs or authorities. n From these, identify a small set of top hub and authority pages; n ® Prasad iterative algorithm. L 21 Link. Analysis 64

Base set n n n Given text query (say browser use a text ), index to get all pages containing browser. n Call this the root set of pages. Add in any page that either n points to a page in the root set, or n is pointed to by a page in the root set. Call this the base set. Prasad L 21 Link. Analysis 65

Visualization Root set Base set Prasad L 21 Link. Analysis 66

Assembling the base set [Klei 98] n n n Root set typically 200 -1000 nodes. Base set may have up to 5000 nodes. How do you find the base set nodes? n n n Prasad Follow out-links by parsing root set pages. Get in-links (and out-links) from a connectivity server. (Actually, suffices to text-index strings of the form href=“ URL” to get in-links to URL. ) L 21 Link. Analysis 67

Distilling hubs and authorities n n Compute, for each page x in the base set, a hub score h(x) and an authority score a(x). Initialize: for all x, h(x) 1; a(x) 1; Iteratively update all h(x), a(x); Key After iterations n n Prasad output pages with highest h() scores as top hubs highest a() scores as top authorities. L 21 Link. Analysis 68

Illustrated Update Rules 1 4 2 3 5 h 4 = a 5 + a 6 + a 7 4 6 7 69 a 4 = h 1 + h 2 + h 3

Iterative update n Repeat the following updates, for all x: y’s x y’s Prasad L 21 Link. Analysis x 70

Scaling n n To prevent the h() and a() values from getting too big, can normalize after each iteration. Scaling factor doesn’t really matter: n Prasad we only care about the relative values of the scores. L 21 Link. Analysis 71

Old Results n Authorities for query: “Java” n n n Authorities for query “search engine” n n n java. sun. comp. lang. java FAQ Yahoo. com Excite. com Lycos. com Altavista. com Authorities for query “Gates” n n Microsoft. com roadahead. com 72

Similar Page Results from Hubs n Given “honda. com” n n n n toyota. com ford. com bmwusa. com saturncars. com nissanmotors. com audi. com volvocars. com 73

Japan Elementary Schools Hubs n n n n n schools LINK Page-13 “ú–{‚ÌŠw Z a‰„ Zƒz ƒy ¬Šw [ƒ [ƒW 100 Schools Home Pages (English) K-12 from Japan 10/. . . rnet and Education ) http: //www. . . iglobe. ne. jp/~IKESAN ‚l‚f‚j Z‚U”N‚P‘g • ¨Œê ¬Šw ÒŠ—’¬—§ ÒŠ—“Œ Z ¬Šw Koulutus ja oppilaitokset TOYODA HOMEPAGE Education Cay's Homepage(Japanese) –y“ì Z‚Ìƒz ƒy ¬Šw [ƒ [ƒW UNIVERSITY ‰J—³ Z DRAGON 97 -TOP ¬Šw ‰ª Z‚T”N‚P‘gƒz ƒy ¬Šw [ƒ [ƒW ¶µ°é¼ Á© ¥á¥Ë¥å¡¼ Prasad Authorities n n n n n L 21 Link. Analysis The American School in Japan The Link Page ‰ª —§ˆä“c Zƒz ƒy è s ¬Šw [ƒ [ƒW Kids' Space ˆÀ —§ˆÀ • ” Z é s é ¬Šw ¼ ‹{ é‹³ˆç‘åŠw • ¬Šw ‘® Z KEIMEI GAKUEN Home Page ( Japanese ) Shiranuma Home Page fuzoku-es. fukui-u. ac. jp welcome to Miasa E&J school _“Þ E‰¡ • l ìŒ§ s—§’† Z‚Ìƒy ì ¬Šw ¼ http: //www. . . p/~m_maru/index. html fukui haruyama-es Home. Page Torisu primary school goo Yakumo Elementary, Hokkaido, Japan FUZOKU Home Page Kamishibun Elementary School. . . 76

Things to note n n Pulled together good pages regardless of language of page content. Use link analysis only after base set assembled n n Prasad iterative scoring is query-independent. Iterative computation after text index retrieval - significant overhead. L 21 Link. Analysis 77

Proof of convergence n n n adjacency matrix A: n n each of the n pages in the base set has a row and column in the matrix. Entry Aij = 1 if page i links to page j, else = 0. 1 3 Prasad L 21 Link. Analysis 3 0 1 1 1 3 2 2 1 1 0 0 78

Hub/authority vectors n n View the hub scores h() and the authority scores a() as vectors with n components. Recall the iterative updates Prasad 79

Rewrite in matrix form n n h=Aa. a=Ath. Recall At is the transpose of A. Substituting, h=AAth and a=At. Aa. Thus, h is an eigenvector of AAt and a is an eigenvector of At. A. Further, our algorithm is known for computing eigenvectors: the power iteration method. Prasad 80 Guaranteed to converge.

Issues n Topic Drift n Off-topic pages can cause off-topic “authorities” to be returned n n E. g. , the neighborhood graph can be about a “super topic” Mutually Reinforcing Affiliates n Affiliated pages/sites can boost each others’ scores n Prasad Linkage between affiliated pages is not a useful signal L 21 Link. Analysis 81