Subjectivity and Sentiment Analysis from Words to Discourse

Subjectivity and Sentiment Analysis: from Words to Discourse Jan Wiebe Computer Science Department Intelligent Systems Program University of Pittsburgh I 2 R Singapore 2009

Burgeoning Field Quite a large problem space Several terms reflecting varying goals and models – – – – – Sentiment Analysis Opinion Mining Opinion Extraction Subjectivity Analysis Appraisal Analysis Affect Sensing Emotion Detection Identifying Perspective Etc.

What is Subjectivity? The linguistic expression of somebody’s opinions, sentiments, emotions, evaluations, beliefs, speculations (private states) Private state: state that is not open to objective observation or verification Quirk, Greenbaum, Leech, Svartvik (1985). Note that this particular use of subjectivity is adapted from literary theory E. G. Banfield 1982, Fludernik 1993; Wiebe Ph. D Dissertation 1990.

Examples of Subjective Expressions References to private states – She was enthusiastic about the plan – He was boiling with anger References to speech or writing events expressing private states – Leaders rounding condemned his verbal assault on Israel Expressive subjective elements – That would lead to disastrous consequences – What a freak show

Manually (human) Annotated News Data Wilson Ph. D Dissertation 2008 I think people are happy because Chavez has fallen direct subjective span: think source: <writer, I> attitude: attitude span: think type: positive arguing intensity: medium target: target span: people are happy because Chavez has fallen direct subjective span: are happy source: <writer, I, People> attitude: attitude span: are happy type: pos sentiment intensity: medium target: inferred attitude span: are happy because Chavez has fallen type: neg sentiment intensity: medium target: target span: Chavez has fallen target span: Chavez MPQA corpus: http: //www. cs. pitt. edu/mpqa

Subjectivity and Sentiment Analysis Automatic extraction of subjectivity (opinions) expressed in text or dialog (newspapers, blogs, conversations, etc) – Sentiment analysis: specifically looking for postiive and negative sentiments

Why? Subjectivity analysis systems can provide useful input to several kinds of end applications

Why? Opinion Question Answering – Answer Questions about Opinions Q: What is the international reaction to the reelection of Robert Mugabe as President of Zimbabwe? Stoyanov, Cardie, Wiebe EMNLP 05 Somasundaran, Wilson, Wiebe, Stoyanov ICWSM 07

Why? Information Extraction (AAAI Filter out false hits for Information Extraction systems “The Parliament exploded into fury against the government when word leaked out…” Riloff, Wiebe, Phillips AAAI 05

Why? Recognizing Stances in Debates Pro-Firefox – Firefox is more respectful of W 3 C internet standards while µsoft sucks by trying to force us to use their own standards to keep their monopoly. Pro-IE – IE is much easier to use. It also is more visually pleasing. It is much more secure as well.

Why? Product Review Mining • Determine if the given product/movie review is Negative review positive or negative • “… was billed as a suspense thriller along the lines of Hitchcock. . . the problem here is that writing has failed some very capable actors. . ” • “The last half of the film is very well done. Another thing that carries this film are the superb performances. . . is a very entertaining and suspenseful film. . . ” Positive review

And Several Others… Tracking sentiments toward topics over time: Is anger ratcheting up or cooling down? Prediction (election outcomes, market trends): Will Clinton or Obama win? Meeting summarization: What were the main opinions expressed? Etcetera!

Focus Our focus is linguistic disambiguation; how should language be interpreted? – Is it subjective in the first place? If so, is it positive or negative? What is it about? Etc. Subjective language is highly ambiguous

Interpretation Lexicon of keywords out of context continuum NLP methods/resources building toward full interpretations Full contextual Interpretation of words in text or dialogue “The dream” Today: several tasks along the continuum

Interpretation Lexicon of keywords out of context Brilliant Difference Hate Interest Love … continuum Full contextual Interpretation of words in text or dialogue

Subjectivity Lexicons Most approaches to subjectivity and sentiment analysis exploit subjectivity lexicons. – Lists of keywords that have been gathered together because they have subjective uses

Automatically Identifying Subjective Words Much work in this area E. g. Hatzivassiloglou & Mc. Keown ACL 97; Wiebe AAAI 00; Turney ACL 02; Kamps & Marx 2002; Wiebe, Riloff, Wilson Co. NLL 03; Kim & Hovy 2005; Esuli & Sebastiani 2005; Subjectivity Lexicon: http: //www. cs. pitt. edu/mpqa Entries from several sources (our work and others’)

However… Consider the keyword “Interest”. It is in the subjectivity lexicon. But, what about “interest rate”, for example?

Dictionary Definitions senses Interest, involvement -- (a sense of concern with and curiosity about someone or something; "an interest in music") Interest -- (a fixed charge for borrowing money; usually a percentage of the amount borrowed; "how much interest do you pay on your mortgage? ")

Dictionary Definitions senses S Interest, involvement -- (a sense of concern with and curiosity about someone or something; "an interest in music") O Interest -- (a fixed charge for borrowing money; usually a percentage of the amount borrowed; "how much interest do you pay on your mortgage? ")

Senses Even in subjectivity lexicons, many senses of the keywords are objective ~50% in our study! Thus, many appearances of keywords in texts are false hits

Senses His alarm grew as the election returns came in. He set his alarm for 7 am. His trust grew as the candidate spoke. His trust grew as interest rates increased.

Word. Net Miller 1995; Fellbaum 1998

Examples “There are many differences between African and Asian elephants. ” “… dividing by the absolute value of the difference from the mean…” “Their differences only grew as they spent more time together …” “Her support really made a difference in my life” “The difference after subtracting X from Y…”

Subjectivity Sense Labeling Automatically classifying senses as subjective or objective Wiebe & Mihalcea ACL 06 Gyamfi, Wiebe, Mihalcea, Akkaya NAACL 09 See also: Esuli & Sebastiani EACL 06, ACL 07 Andreevskaia & Bergler EACL 06, LREC 06 Su & Markert Coling 08, NAACL 09

Word. Net Senses

Word. Net

Word. Net If this sense is subjective, then maybe these senses of brainy and smart-as-a-whip are as well

Word. Net glosses

Word. Net Examples Glosses and examples contain clues as to the subjectivity of a sense

Word. Net Relations

Hierarchical Structure

Using Hierarchical Structure The higher the IC of the LCS, the more specific it is, and the more similar the seed and target sense are Seed Sense s LCS c Being similar to a subjective seed More likely the target is subjective Information content of the lowest common subsumer Sim(t, s) = -log(p(c)) (Resnik 1995) Target Sense t

Using Hierarchical Structure LCS Target sense Seed sense

Using Hierarchical Structure LCS voice#1 (objective)

Sense Subjectivity LCS Feature LCS c 4 LCS c 3 LCS c 2 LCS c 1 Seed Sense s 4 Seed Sense s 3 Seed Sense s 2 Seed Sense s 1 Target Sense t

Domains Several researchers have noted that subjectivity may be domain specific Word. Net Domains (Gliozzo et al. 2005) assigns a domain label to each synset

Domains Over 80% of the subjective seed senses are in 6 domains (rest are in 35) – Factotum “other” [201] garishness#2, racism#1 – Psychological features [98] horror#1, satisfaction#1 – Person [68] meanie#1, Francophobe#1 – Law [61] swindler#1, two-timer#1 – Psychology [20] ecstasy#1, indignity#1 – Sociology [20] vandalism#1, odium#1

Sense Subjectivity LCS Feature Saves computation LCS c 4 LCS c 3 The score is the feature value for t LCS c 2 LCS c 1 Seed Sense s 4 Seed Sense s 3 Seed Sense s 2 Seed Sense s 1 Domain D Target Sense t

Using Hierarchical Structure Gyamfi, Wiebe, Mihalcea, Akkaya NAACL 09 Hierarchical information is combined with other Word. Net-Based knowledge to classify senses as Subjective or Objective

Interpretation Lexicon of keywords out of context Brilliant sense#1 S sense#2 S … Difference sense#1 O sense#2 O sense#3 S sense#4 S sense#5 O … continuum Full contextual Interpretation of words in text or dialog Now we will leave the lexicon and look at disambiguation in the context of a text or conversation

Contextual Subjectivity Analysis S O? “He spins a riveting plot which grabs and holds the reader’s interest…” Subjectivity Sentence Classifier Do the sentences contain subjectivity? S O? “The notes do not pay interest. ” E. g. Riloff & Wiebe EMNLP 03 Yu & Hatzivassiloglou EMNLP 03

Contextual Subjectivity Analysis S O? “He spins a riveting plot which grabs and holds the reader’s interest…” Subjectivity Phrase Classifier Is a phrase containing a keyword subjective? S O? “The notes do not pay interest. ” Wilson, Wiebe, Hoffmann EMNLP 05

Contextual Subjectivity Analysis S O? Pos, Neg, “There are many differences between African and Asian elephants. ” Neutral? Sentiment Phrase Classifier SPos, Neg, O? Neutral? Is a phrase containing a keyword positive, Negative, or neutral? We’ll return to this, topic after next. But first… “Their differences only grew as they spent more time together …” Wilson, Wiebe, Hoffmann EMNLP 05

Interpretation Lexicon of keywords out of context Brilliant sense#1 S sense#2 S … Difference sense#1 O sense#2 O sense#3 S sense#4 S sense#5 O … continuum Contextual Subjectivity analysis Full contextual Interpretation of words in text or dialog Exploiting sense labels to improve the contextual classifiers

Subjectivity Tagging using WSD S O? “He spins a riveting plot which grabs and holds the reader’s interest…” Sense 4 S Sense 4 “a sense of Subjectivity Classifier concern with and curiosity about someone or something” O Sense 1 “a fixed charge WSD System for borrowing money” S O? Sense 1 “The notes do not pay interest. ”

Subjectivity Tagging using WSD S O “He spins a riveting plot which grabs and holds the reader’s interest…” Sense 4 S Sense 4 “a sense of Subjectivity Classifier concern with and curiosity about someone or something” O Sense 1 “a fixed charge WSD System for borrowing money” S O Sense 1 “The notes do not pay interest. ”

Examples “There are many differences between African and Asian elephants. ” Sense#1 O “… dividing by the absolute value of the difference from the mean…” Sense#2 O Is it one of these? “Their differences only grew as they spent more time together …” Sense#3 S “Her support really made a difference in my life” Sense#4 S “The difference after subtracting X from Y…” Sense#5 O

Examples “There are many differences between African and Asian elephants. ” Sense#1 O “… dividing by the absolute value of the difference from the mean…” Sense#2 O “Their differences only grew as they spent more time together …” Sense#3 S Or one of these? “Her support really made a difference in my life” Sense#4 S “The difference after subtracting X from Y…” Sense#5 O

Subjectivity Tagging using Subjectivity WSD S O? “There are many differences between African and Asian elephants. ” Sense O {1, 2, 5} Subjectivity Classifier S O? Difference sense#1 O sense#2 O sense#3 S sense#4 S sense#5 O SWSD System Sense S {3, 4} “Their differences only grew as they spent more time together …”

Subjectivity Tagging using Subjectivity WSD S O “There are many differences between African and Asian elephants. ” Sense O {1, 2, 5} Subjectivity Classifier S O Difference sense#1 O sense#2 O sense#3 S sense#4 S sense#5 O SWSD System Sense S {3, 4} “Their differences only grew as they spent more time together …”

SWSD Akkaya, Wiebe, Mihalcea EMNLP 09 SWSD Performance is well above baseline and the performance of full WSD – SWSD is a feasible variant of WSD – Subjectivity provides a natural course-grained sense grouping

SWSD in Subjectivity Tagging SWSD exploited to improve performance of subjectivity analysis systems Both S/O and Pos/Neg/Neutral classifiers

Sentiment Analysis using SWSD Pos, Neg, Neutral? Sentiment Classifier Pos, Neg, Neutral? “There are many differences between African and Asian elephants. ” Sense O {1, 2, 5} Difference sense#1 O sense#2 O sense#3 S sense#4 S sense#5 O SWSD System Sense S {3, 4} “Their differences only grew as they spent more time together …”

Interpretation Lexicon of keywords out of context Brilliant sense#1 S sense#2 S … Difference sense#1 O sense#2 O sense#3 S sense#4 S sense#5 O … continuum SWSD Contextual Sentiment Analysis Full contextual Interpretation of words in text or dialog Rest of the talk: contextual processing not bound to word senses Return to contextual sentiment classification

Sentiment Analysis Wilson, Wiebe, Hoffman EMNLP 05, Computational Linguistics 2009 Automatically identifying positive and negative emotions, evaluations, and stances – Our approach: classify expressions containing a keyword as positive, negative, both, or neutral

Phrase-Level Sentiment Analysis See also, E. G. Yi, Nasukawa, Bunescu, Niblack ICDM 03; Polanyi & Zaenen AAAI-SS 04; Popescu & Etzioni EMNLP 05; Suzuki, Takamura, Okumura CICLing 06; Moilanen & Pulman RANLP 07; Choi & Cardie EMNLP 08

Prior versus Contextual Polarity Many subjectivity lexicons contain polarity information Prior polarity: out of context, positive, negative, or neutral A word may appear in a phrase that expresses a different polarity in context Contextual polarity

MPQA (Human) Polarity Annotations Judge the contextual polarity of the sentiment that is ultimately being conveyed in the context of the text or conversation

Contextual Interpretation They have not succeeded, and will never succeed, in breaking the will of this valiant people.

Contextual Polarity is Complex They have not succeeded, and will never succeed, in breaking the will of this valiant people.

Approach Step 1: Neutral or Polar? Step 2: Are the polar instances Positive or Negative? Combine a variety of evidence

Evidence Modifications and Conjunctions – Cheers to Timothy Whitfield for the wonderfully horrid visuals pos wonderfully horrid – Disdain and wrath mod Hatzivassiloglou & Mc. Keown ACL 97 Subjectivity disdain (neg) and wrath(neg) of the surrounding context; syntactic role in the sentence; etc.

Polarity Influencers Negation – Local not good – Longer-distance dependencies » Does not look very good (proposition) » No politically prudent Israeli could support either of them (subject) – Phrases with negations may intensify instead » Not only good, but amazing!

Polarity Influencers Contextual Valence Shifters Polanyi & Zaenan 2004 – General polarity shifter » Pose little threat » Contains little truth – Negative polarity shifters » Lack of understanding – Positive polarity shifters » Abate the damage

Approach Step 1: Neutral or Polar? Step 2: Are the polar instances Positive or Negative? Combine a variety of evidence Still much to do in the area of recognizing contextual polarity

Interpretation Lexicon of keywords out of context Brilliant sense#1 S sense#2 S … Difference sense#1 O sense#2 O sense#3 S sense#4 S sense#5 O … SWSD continuum Contextual Discourse Sentiment Analysis Full contextual Interpretation of words in text or dialog

Discourse-Level Opinion Interpretation I: Opinion Frames Somasundaran, Wiebe, Ruppenhofer COLING 08; Somasundaran, Namata, Wiebe, Getoor EMNLP 09 Interpretations involving multiple sentences within the discourse Opinion Frames are composed of 2 opinions and the relation between their targets (what they are opinions of) Larger structures emerge from interdependent frames Data: task-oriented dialogues

Discourse-Level Opinion Interpretation I like the LCD feature We must implement the LCD “Like” and “must” are clear positive clues

Discourse-Level Opinion Interpretation I like the LCD feature We must implement the LCD targets: what the opinion is about

Discourse-Level Opinion Interpretation I like the LCD feature We must implement the LCD The LCD is traditional

Discourse-Level Opinion Interpretation I like the LCD feature We must implement the LCD The LCD is traditional Joint Interpretation of opinions in the discourse If a coherent discourse expressing one overall opinion “traditional” is also positive

Discourse-Level Opinion Interpretation • Shapes should be curved, so round shapes. Nothing square-like. • . . . So we shouldn’t have too square corners and that kind of thing.

Discourse–level interaction between opinions Direct opinion • Shapes should be curved, so round shapes. Nothing square-like. Direct opinion • . . . So we shouldn’t have too square corners and that kind of thing.

Discourse–level interaction between opinions Direct opinion • Shapes should be curved, so round shapes. Nothing square-like. Direct opinion • . . . So we shouldn’t have too square corners and that kind of thing. What were the opinions regarding the curved shape? Will the curved shape be accepted?

Discourse–level interaction between opinions Direct opinion Opinions towards mutually exclusive option (alternative) • Shapes should be curved, so round shapes. Nothing square-like. • . . . So we shouldn’t have too square corners and that kind of thing. Opinions towards mutually exclusive option (alternative)

Discourse–level interaction between opinions Direct opinion Opinions towards mutually exclusive option (alternative) • Shapes should be curved, so round shapes. Nothing square-like. • . . . So we shouldn’t have too square corners and that kind of thing. Opinions towards mutually exclusive option (alternative) Opinions towards the square shapes reveal additional information about the speaker’s opinion of the curved shape

Discourse-level analysis for Recognizing stances in debates Somasundaran & Wiebe ACL-IJCNLP 09 Use our understanding of discourse-level opinion relations to recognize stances in debates Slides, part 2