Nonnegative Matrix Factorization via Rank-one Downdate Nonnegative

Nonnegative Matrix Factorization via Rank-one Downdate

Nonnegative Matrix Factorization

2 by 1965 560 by 2 -2. 19 -0. 02 20 by 28 -3. 19 1. 02 2 by 1

Singular Value Decomposition (SVD)

History

History

History

History

History (Algorithms)

History (Algorithms)

First observation

Power method • Computes the leading singular vectors/value (or eigenvector/value) of a matrix 1 2 while not converged 3 4 5 6 end

Example • Computes the leading singular vectors/value (or eigenvector/value) of a matrix 2 while not converged 3 4 5 6 end

Naive approach to NMF using this observation 1 2 3 4 5 6 for all end for set Without step 5, this will simply compute the SVD (Jordan's algorithm, Camille Jordan 1874. )

Second observation

Modified power iteration 1 2 while not converged 3 4 5 6 7 8 end

Modified power iteration: Demo A= Rank-1 submatrix

Modified power iteration: Demo v: 0. 14 0. 07 0. 64 0. 41 0. 55 Rank-1 submatrix

Modified power iteration: Demo v: 0. 0 0. 64 0. 41 0. 55 Rank-1 submatrix

Modified power iteration: Demo v: 0. 0 0. 64 0. 41 0. 55 Rank-1 submatrix

Modified power iteration: Demo v: u: 0. 16 0. 21 0. 22 0. 44 0. 74 0. 20 0. 64 0. 41 0. 55 Rank-1 submatrix

Modified power iteration: Demo v: u: 0. 0 0. 44 0. 74 0. 20 0. 64 0. 41 0. 55 Rank-1 submatrix

Modified power iteration: Demo v: u: 0. 0 0. 44 0. 74 0. 20 0. 64 0. 41 0. 55 Rank-1 submatrix

Modified power iteration: Demo v: u: 0. 0 0. 44 0. 74 0. 20 0. 60 0. 28 0. 59 Rank-1 submatrix

Modified power iteration: Demo v: u: 0. 0 0. 44 0. 74 0. 20 0. 60 0. 28 0. 59 Rank-1 submatrix

Modified power iteration: Demo v: u: 0. 0 0. 44 0. 74 0. 20 0. 60 0. 28 0. 59 Rank-1 submatrix Rank-1 Zero-out! submatrix

Modified power iteration: Demo Anew = Rank-1 submatrix

Rank-one Downdata (R 1 D) 1 2 3 4 5 end for

Objective function

Approx. Rank. One. Submatrix(A)

Rank-one Downdata (R 1 D)

Rank-one Downdata (R 1 D)

A simple model for text

Generating a corpus in the model

Theorem about text

LSI

R 1 D

Experimental results • Information retrieval task – TDT corpus (pilot study, v. 1. 3, 1997): news articles Identified topics (first few columns of ): Topic: president, clinton, house, white Topic: bosnian, serbs, bosnia, serb, nato, sarajevo, air, bihac Topic: haiti, military, aristide, haitian, troops, port, invasion, … Topic: simpson, defense, judge, case, jury, trial, angeles, los, court, … Topic: bill, today, senate, republicans, house, congress, republican, …

Experimental results • For topic: “OJ Simpson trial” (simpson, defense, judge, case, jury, trial) Document 1 Document 2 Document 3 Document 4 Allen Simpson Judge Gloves Kaelin Judge Defense Case Police Defence Statements Prosecution Defense Marcus Opening Defense Chicago Court Jury Roger Testimony Witness Legal Put Night Testify Yesterday Fit Knapsack Nicole Prosecution Today Cnn gloves Marc Problems Blood los jury Witness

Theorem about images

Experimental results

LSI

NMF-DIV

R 1 D

LSI

NMF_DIV

NMF_SC

R 1 D