Speech Data Corpus for Verbal Intelligence Estimation Information Technology – Dialogue Systems Ulm University (Germany) http: //www. dialogue-systems. de Kseniya Zablotskaya, Steffen Walter, Wolfgang Minker
Page 2 www. dialogue-systems. de | LREC 2010 | May 2010 Outline -Introduction -Improvement of Spoken Language Dialogue Systems -Verbal Intelligence Estimation -Monologues Collection -Hamburg Wechsler Intelligence Test -Dialogues Collection -Transcription Standards -Result Table for Each Candidate -Participants -Conclusions and Future Work
Page 3 www. dialogue-systems. de | LREC 2010 | May 2010 Introduction Different ways to describe the same event: - “Excuse me, could you tell me the way to the railway station? ” - “Hey you, show me where the railway station is. ” What our words can say about us: Emotion Gender Age Analysis of speech Social class Personality Verbal Intelligence …
Page 4 www. dialogue-systems. de | LREC 2010 | May 2010 Improvement of Spoken Language Dialogue Systems - estimation of cognitive processes of the user - adaptation to the user Cognitive processes of the user Spoken language dialogue system Acoustic front-end Speech recognition Linguistic analysis Dialogue management Speech synthesis Text generation Adaptation to the user Application
Page 5 www. dialogue-systems. de | LREC 2010 | May 2010 Verbal Intelligence Estimation Dialogues Monologues VI=120 VI=80 VI=105 VI=120 VI=105 VI=80 Transcribed speech Feature extraction Model / Classifier VIMod Evaluation VI-test VI %
Page 6 www. dialogue-systems. de | LREC 2010 | May 2010 Monologues Collection Two short films (Galileo): Craziest hotels in the world: -necessary to memorize the names -necessary to memorize the order Experiment on how long people could stay awake -possible to describe the film without certain details -descriptions are informative
Page 7 www. dialogue-systems. de | LREC 2010 | May 2010 Dialogues Collection Duration: at least 10 minutes Topic: the education and the school system in Germany - interesting - participants know a lot about it - participants have different opinions Dialogues Answering Questions Discussions Contra-Discussions
Page 8 www. dialogue-systems. de | LREC 2010 | May 2010 Hamburg Wechsler Intelligence Test Information (25 questions) -measures general knowledge -questions from a particular culture For example: Who is president of Russia?
Page 9 www. dialogue-systems. de | LREC 2010 | May 2010 Hamburg Wechsler Intelligence Test Comprehension (10 questions) - social awareness - common-sense For example: What would you do if you lost your way in a forest?
Page 10 www. dialogue-systems. de | LREC 2010 | May 2010 Hamburg Wechsler Intelligence Test Digital Span -forward -backward - auditory short memory - concentration - attention For example: Please listen to the fallowing digits and repeat them: 57246
Page 11 www. dialogue-systems. de | LREC 2010 | May 2010 Hamburg Wechsler Intelligence Test Arithmetic (10 questions) -mental alertness -attention and concentration while manipulation mental mathematical problems For example: Seven envelopes cost twenty five cents. How many envelopes can you buy if you have one dollar?
Page 12 www. dialogue-systems. de | LREC 2010 | May 2010 Hamburg Wechsler Intelligence Test Similarities in Dissimilar Objects (12 questions) - abstract reasoning - power of conceptualization For example: Please find a similarity among a dog and a lion?
Page 13 www. dialogue-systems. de | LREC 2010 | May 2010 Hamburg Wechsler Intelligence Test Vocabulary (42 questions) - comprehension of meanings - relation between the expressive words For example: What does the word “zebra” mean?
Page 15 www. dialogue-systems. de | LREC 2010 | May 2010 Transcription Standards All monologues and dialogues were transcribed according to the standards by Mergenthaler. The punctuation marks in transcripts are used to show rhythmical and syntactical speech interruptions: “? ” – interrogative word intonation and rising tone “. ” – completed thoughts and falling tone “, ” – short pauses in the speech, but with a continuation of the main idea “; ” – interrupted thoughts Example: “no no. or yes? you say; understand, ”
Page 16 www. dialogue-systems. de | LREC 2010 | May 2010 Result Table for Each Candidate - candidate’s points for each verbal task and the verbal IQ. - verbal IQ is measured according to the special tables of the HAWIE. Sub-Test Points Information 18 out of 25 Comprehension 19 out of 20 Digit Span 15 out of 17 Arithmetic 13 out of 14 Similarities in dissimilar Objects 24 out of 24 Vocabulary 73 out of 84 Verbal IQ 122
Page 17 www. dialogue-systems. de | LREC 2010 | May 2010 Participants 56 candidates: men - 27, women - 29 Age: 16 – 75 Language: German 71 monologues (3 hours 30 minutes) 30 dialogues (6 hours 30 minutes)
Page 18 www. dialogue-systems. de | LREC 2010 | May 2010 Conclusions and future directions Speech data corpus: - 56 candidates; - 10 hours of audio data; Approaches which can be applied to the collected data: - Word usage, abstracts, emotion words; - Analysis at different linguistic levels: morphology, lexicology, syntax, semantics, and discourse; - Linguistic styles; - Content words; - Degree of speakers’ immersion in monologues and dialogues; - “Good story” criteria; - Status in a conversation; - Levels of agreement; -Kelly’s repertory grids; We are going to find more candidates and to continue these recordings.
Page 19 www. dialogue-systems. de | LREC 2010 | May 2010 Thanks for your attention!
Скачать презентацию Speech Data Corpus for Verbal Intelligence Estimation Information
a27341021b194233f3999f7d9565cdff.ppt