Скачать презентацию On the Role of Prosody in Structuring Discourse Скачать презентацию On the Role of Prosody in Structuring Discourse

b329d49704e8f7f716f430c9d08c61d2.ppt

  • Количество слайдов: 41

On the Role of Prosody in Structuring Discourse October 5, 2005 - Berlin, Germany On the Role of Prosody in Structuring Discourse October 5, 2005 - Berlin, Germany “Downstepped contours in the given/new distinction” Agustín Gravano Spoken Language Processing Group Columbia University, New York

Participants in this project Columbia University (New York) Julia Hirschberg Stefan Benus Agustín Gravano Participants in this project Columbia University (New York) Julia Hirschberg Stefan Benus Agustín Gravano Northwestern University (Chicago) Gregory Ward Elisa Sneed 2 Agustín Gravano - Columbia University

1. Introduction a) b) c) d) e) To. BI Discourse structure (Grosz & Sidner 1. Introduction a) b) c) d) e) To. BI Discourse structure (Grosz & Sidner ’ 86) Information status (Prince ’ 92) Meaning of intonational contours The downstepped contours 2. Boston Directions Corpus a) Description of the corpus b) Downstep and discourse structure c) Downstep and information status 3. Games Project a) Description of the corpus b) Ongoing and future research 3 Agustín Gravano - Columbia University

1. Introduction a) b) c) d) e) To. BI Discourse structure (Grosz & Sidner 1. Introduction a) b) c) d) e) To. BI Discourse structure (Grosz & Sidner ’ 86) Information status (Prince ’ 92) Meaning of intonational contours The downstepped contours 2. Boston Directions Corpus a) Description of the corpus b) Downstep and discourse structure c) Downstep and information status 3. Games Project a) Description of the corpus b) Ongoing and future research 4 Agustín Gravano - Columbia University

To(nes and)B(reak)I(ndices) • Prosody annotation convention. • Two tones: H and L, which may To(nes and)B(reak)I(ndices) • Prosody annotation convention. • Two tones: H and L, which may be combined (e. g. H+L) • Devised originally for Standard American English, but To. BI standards also proposed for Japanese, German, Italian, Spanish, British, Australian English, . . • 4 tiers: – orthographic tier: words – break-index tier: degrees of junction – tonal tier: pitch accents, phrase accents, boundary tones – miscellaneous tier: disfluencies, non-speech sounds, etc. 5 Agustín Gravano - Columbia University

Discourse Structure (G&S ’ 86) • Series of discourse segments, defined in terms of Discourse Structure (G&S ’ 86) • Series of discourse segments, defined in terms of the speaker’s intentions: the discourse segment purpose (DSP). • Let a, b: DSP, – a satisfaction-precedes b iff a must first be achieved in order for b to succeed; – a dominates b iff fulfilling b partly fulfills a. Barbara Grosz & Candace Sidner, 1986. “Attention, intentions, and the structure of discourse. ” Computational Linguistics 12(3): 175 -204. 6 Agustín Gravano - Columbia University

Information Status (Prince ’ 92) Discourse { Hearer { • Given • New • Information Status (Prince ’ 92) Discourse { Hearer { • Given • New • Given • Inferrable • New Ellen Prince, 1992. “The ZPG letter: Subjects, definiteness, and informationstatus. ” In Discourse Description: Diverse Analyses of a Fund Raising Text, S. Thompson & W. Mann (eds. ), 295 -325, Philadelphia: John Benjamins B. V. Agustín Gravano - Columbia University

Multiple “ meanings” of intonational contours • “Declarative” contours (H* L- L%) – Statements Multiple “ meanings” of intonational contours • “Declarative” contours (H* L- L%) – Statements – Wh-questions • Rise-fall-rise contours (L*+H L- H%) – Uncertainty – Incredulity • H* Downstepped contours (H* (!H*)+ L- (L%|H%)? ) – Topic beginnings or endings? – “Given” information? 8 Agustín Gravano - Columbia University

Example: H* !H* !H* L-H% 9 Agustín Gravano - Columbia University Example: H* !H* !H* L-H% 9 Agustín Gravano - Columbia University

Understanding the multiple uses of contours is useful and interesting • In most TTS Understanding the multiple uses of contours is useful and interesting • In most TTS systems – ‘Standard’ declarative (H* L- L%) contour over-used – ‘Given’ information deaccented too often • The H* (!H*)+ L- (L%|H%)? contours might be used instead, if they are appropriate 10 Agustín Gravano - Columbia University

H* (!H*)+ L- (L%|H%)? in Standard American English • Topic structure markers (Pierrehumbert & H* (!H*)+ L- (L%|H%)? in Standard American English • Topic structure markers (Pierrehumbert & Hirschberg ’ 90) – Beginning and ending of topics – Professorial tone • Givenness (Hirschberg & Pierrehumbert ’ 86, Ladd ’ 96, Dahan et al ’ 02) – “This material should already be familiar to you. ” – Alternates with deaccenting – when? 11 Agustín Gravano - Columbia University

1. Introduction a) b) c) d) e) To. BI Discourse structure (Grosz & Sidner 1. Introduction a) b) c) d) e) To. BI Discourse structure (Grosz & Sidner ’ 86) Information status (Prince ’ 92) Meaning of intonational contours The downstepped contours 2. Boston Directions Corpus a) Description of the corpus b) Downstep and discourse structure c) Downstep and information status 3. Games Project a) Description of the corpus b) Ongoing and future research 12 Agustín Gravano - Columbia University

1. Introduction a) b) c) d) e) To. Bi Discourse structure (Grosz & Sidner 1. Introduction a) b) c) d) e) To. Bi Discourse structure (Grosz & Sidner ’ 86) Information status (Prince ’ 92) Meaning of intonational contours The downstepped contours 2. Boston Directions Corpus a) Description of the corpus b) Downstep and discourse structure c) Downstep and information status 3. Games Project a) Description of the corpus b) Ongoing and future research 13 Agustín Gravano - Columbia University

Boston Directions Corpus • 4 speakers • 9 increasingly complex direction-giving tasks • Spontaneous Boston Directions Corpus • 4 speakers • 9 increasingly complex direction-giving tasks • Spontaneous speech transcribed and speakers returned and read • ~67 m spon; ~50 m read 14

Boston Directions Corpus first enter the Harvard Square T stop and buy a token Boston Directions Corpus first enter the Harvard Square T stop and buy a token then proceed to get on the inbound um Red Line uh subway and take the subway from Harvard Square to Central Square and then to Kendall Square then get off the T 15 Agustín Gravano - Columbia University

BDC - Discourse Structure first enter the Harvard Square T stop and buy a BDC - Discourse Structure first enter the Harvard Square T stop and buy a token then proceed to get on the inbound um Red Line uh subway and take the subway from Harvard Square to Central Square and then to Kendall Square then get off the T 16 Agustín Gravano - Columbia University

BDC - Information Status first enter the Harvard Square T stop and buy a BDC - Information Status first enter the Harvard Square T stop and buy a token then proceed to get on the inbound um Red Line uh subway and take the subway from Harvard Square to Central Square and then to Kendall Square then get off the T Discourse Given 17 Agustín Gravano - Columbia University

BDC - Information Status first enter the Harvard Square T stop and buy a BDC - Information Status first enter the Harvard Square T stop and buy a token then proceed to get on the inbound um Red Line uh subway and take the subway from Harvard Square to Central Square and then to Kendall Square then get off the T Hearer Given Hearer Inferrable 18 Agustín Gravano - Columbia University

BDC - DS Contours first enter the Harvard Square T stop and buy a BDC - DS Contours first enter the Harvard Square T stop and buy a token then proceed to get on the inbound um Red Line uh subway and take the subway from Harvard Square to Central Square and then to Kendall Square then get off the T 19 Agustín Gravano - Columbia University

Downstep and Discourse Structure • Distribution of use of DS contours for signaling discourse Downstep and Discourse Structure • Distribution of use of DS contours for signaling discourse structure? • How frequently is discourse structure conveyed using DS contours? • Does this differ by speaking style (read vs. spontaneous speech)? • Is there notable speaker variation in either of these? 20 Agustín Gravano - Columbia University

Use of DS contours for discourse position Spontaneous: Contour Seg Beg Seg Final Total Use of DS contours for discourse position Spontaneous: Contour Seg Beg Seg Final Total H* (!H*)+ L- (L%, H%)? 88 (18%) 196 (40%) 488 Seg Beg Seg Final Total H* (!H*)+ L- (L%, H%)? 131 (29%) 195 (43%) 451 Read: Contour 21 Agustín Gravano - Columbia University

Discourse position conveyed using DS contours Spontaneous: Contour Seg Beg H* (!H*)+ L- (L%, Discourse position conveyed using DS contours Spontaneous: Contour Seg Beg H* (!H*)+ L- (L%, H%)? 88 (11%) Total Seg Final 196 (28%) 825 (100%) 693 (100%) Read: Contour Seg Beg H* (!H*)+ L- (L%, H%)? 131 (18%) Total Seg Final 195 (31%) 721 (100%) 635 (100%) 22 Agustín Gravano - Columbia University

Speaker variability • We found high variability (both in spontaneous and read speech) in: Speaker variability • We found high variability (both in spontaneous and read speech) in: – Overall use of DS contours – Distribution of use of DS contours – Frequency with which discourse structure is conveyed using DS contours • Only exception: – Speakers employ ~40% or more of their DS contours over Segment Final phrases. 23 Agustín Gravano - Columbia University

Downstep and Information Status • Are DS contours used over given information, alternating with Downstep and Information Status • Are DS contours used over given information, alternating with a deaccenting strategy? • If so, when do speakers choose one strategy over another? • Information status in the BDC data: – at the NP level (both discourse g/n and hearer g/i/n status), – at the word level (discourse g/n status for individual lexical items). • Smaller corpus: only spontaneous data labeled. 24 Agustín Gravano - Columbia University

Downstep and Information Status Hearer Given All deacc Some accent DS 52 (5%) Total Downstep and Information Status Hearer Given All deacc Some accent DS 52 (5%) Total 6 (2%) Hearer New 3 (2%) Discourse Given New 46 (8%) 15 (2%) 416 (39%) 200 (49%) 58 (45%) 261 (44%) 413 (44%) Other DS 48 (5%) Other Hearer Inferrable 25 (6%) 12 (9%) 32 (5%) 53 (6%) 540 (51%) 175 (43%) 57 (44%) 257 (43%) 469 (49%) 1056 (100%) 406 (100%) 130 (100%) 596 (100%) 950 (100%) Spontaneous productions only. 25 Agustín Gravano - Columbia University

Downstep and Information Status Hearer Given All deacc Some accent DS 45 (8%) 3 Downstep and Information Status Hearer Given All deacc Some accent DS 45 (8%) 3 (4%) Hearer New 0 (0%) Discourse Given New 44 (8%) 4 (4%) 260 (45%) 38 (54%) 3 (33%) 251 (45%) 50 (52%) Other DS 28 (5%) Other Total Hearer Inferrable 2 (3%) 2 (22%) 28 (5%) 4 (4%) 244 (42%) 27 (39%) 4 (44%) 237 (42%) 38 (40%) 577 (100%) 9 (100%) 70 (100%) 560 (100%) 96 (100%) Spon - Only NPs for which all lexical elements are Given. 26 Agustín Gravano - Columbia University

Downstep and Information Status • DS contours clearly dominate Hearer. Inferrables. • DS contours Downstep and Information Status • DS contours clearly dominate Hearer. Inferrables. • DS contours are commonly used over Given information. • Little evidence from this study that information status is a major predictor of the use of DS contours: equally likely to be used over New NPs. 27 Agustín Gravano - Columbia University

1. Introduction a) b) c) d) e) To. BI Discourse structure (Grosz & Sidner 1. Introduction a) b) c) d) e) To. BI Discourse structure (Grosz & Sidner ’ 86) Information status (Prince ’ 92) Meaning of intonational contours The downstepped contours 2. Boston Directions Corpus a) Description of the corpus b) Downstep and discourse structure c) Downstep and information status 3. Games Project a) Description of the corpus b) Ongoing and future research 28 Agustín Gravano - Columbia University

1. Introduction a) b) c) d) e) To. BI Discourse structure (Grosz & Sidner 1. Introduction a) b) c) d) e) To. BI Discourse structure (Grosz & Sidner ’ 86) Information status (Prince ’ 92) Meaning of intonational contours The downstepped contours 2. Boston Directions Corpus a) Description of the corpus b) Downstep and discourse structure c) Downstep and information status 3. Games Project a) Description of the corpus b) Ongoing and future research 29 Agustín Gravano - Columbia University

Games Project - Goal • Elicit a corpus of spontaneous dialogue containing: – given Games Project - Goal • Elicit a corpus of spontaneous dialogue containing: – given and new NPs – topic segmentation data 30 Agustín Gravano - Columbia University

Games Project - Design Session: – – – 3 collaborative computer games. 2 players, Games Project - Design Session: – – – 3 collaborative computer games. 2 players, each with an electronic game board. Unrestricted speech. No visual contact between subjects. Subjects were paid a fixed amount of money, plus a bonus based on their performance. – Each subject participated in 2 sessions with different partners and on different days. 31 Agustín Gravano - Columbia University

Game # 1 PLAYER 1 “DESCRIBER” PLAYER 2 “SEARCHER” Agustín Gravano - Columbia University Game # 1 PLAYER 1 “DESCRIBER” PLAYER 2 “SEARCHER” Agustín Gravano - Columbia University

Game # 2 PLAYER 1 “DESCRIBER” PLAYER 2 “SEARCHER” Agustín Gravano - Columbia University Game # 2 PLAYER 1 “DESCRIBER” PLAYER 2 “SEARCHER” Agustín Gravano - Columbia University

Game # 3 PLAYER 1 “DESCRIBER” PLAYER 2 “SEARCHER” Game # 3 PLAYER 1 “DESCRIBER” PLAYER 2 “SEARCHER”

Games Project - Design • Study the relation between the choice of intonational contours Games Project - Design • Study the relation between the choice of intonational contours and: – – – givenness status of NPs syntactic position of NPs complexity of NPs proportion of given lexical elements in new NPs discourse structure 35 Agustín Gravano - Columbia University

Games Project - Design • How? – Games 1 & 2: • Cards have Games Project - Design • How? – Games 1 & 2: • Cards have increasingly more features, increasing the complexity of NPs • Some features appear more frequently, becoming “given”. • Features appear in different sizes. – Game 3: • Subject blinking/target image. • Objects images surrounding the target image. • Pretests 36 Agustín Gravano - Columbia University

Games Project - Corpus: – Recorded in a sound-proof booth at Columbia’s Speech Lab Games Project - Corpus: – Recorded in a sound-proof booth at Columbia’s Speech Lab in October 2004. – 12 sessions. – ~20 hours of spontaneous speech. – Fluent dialogues, each game with very different characteristics. – All dialogues have already been transcribed. – Currently doing To. BI labeling. 37 Agustín Gravano - Columbia University

Games Project - Studies • Ongoing studies – Discourse Markers (okay, mm-hm, yeah, etc. Games Project - Studies • Ongoing studies – Discourse Markers (okay, mm-hm, yeah, etc. ) – Turn-taking – Laughter • Future studies – Use of the downstepped contour with respect to discourse structure and info status. – Evolution of the description of lexical entities. – Disfluencies (false repairs, self-repairs, etc. ) –… 38 Agustín Gravano - Columbia University

1. Introduction a) b) c) d) e) To. BI Discourse structure (Grosz & Sidner 1. Introduction a) b) c) d) e) To. BI Discourse structure (Grosz & Sidner ’ 86) Information status (Prince ’ 92) Meaning of intonational contours The downstepped contours 2. Boston Directions Corpus a) Description of the corpus b) Downstep and discourse structure c) Downstep and information status 3. Games Project a) Description of the corpus b) Ongoing and future research 39 Agustín Gravano - Columbia University

1. Introduction a) b) c) d) e) To. BI Discourse structure (Grosz & Sidner 1. Introduction a) b) c) d) e) To. BI Discourse structure (Grosz & Sidner ’ 86) Information status (Prince ’ 92) Meaning of intonational contours The downstepped contours 2. Boston Directions Corpus a) Description of the corpus b) Downstep and discourse structure c) Downstep and information status 3. Games Project a) Description of the corpus b) Ongoing and future research 40 Agustín Gravano - Columbia University

On the Role of Prosody in Structuring Discourse October 5, 2005 - Berlin, Germany On the Role of Prosody in Structuring Discourse October 5, 2005 - Berlin, Germany “Downstepped contours in the given/new distinction” Agustín Gravano Spoken Language Processing Group Columbia University, New York