Generating Multimedia Presentations Stefano Bocconi Visual material taken

Information Systems Multimedia and Human Computer Interaction Group We are currently working on four aspects: modeling of domain-dependent and domain-independent discourse to steer the presentation generation process, investigate dependencies of the user and domain models in the generation process, investigate characteristics of media types for presenting information to the user. investigate to what extent graphic design knowledge can be included in the generation process Fh. G Bonn, 26. 02. 2004

The Genre of Documentaries Video Documentary A documentary purports to present factual information about the world The viewer is led to think that the facts are filmed as they happened Fh. G Bonn, 26. 02. 2004

Categorical Documentaries Video Documentary Conveys information in a simple fashion over a specific category (or more) Examples are scientific documentaries (national geography) The category and its subcategories provide a scheme for the narrative Fh. G Bonn, 26. 02. 2004

Rhetorical Documentaries Video Documentary The goal is to persuade the audience to adopt an opinion Arguments from the source Subject-centered arguments Viewer-centered arguments The River, Pare Lorentz (1936) Fh. G Bonn, 26. 02. 2004

Interview with America Video Documentary Subject: the opinion of American people after 9 -11 Shooting: from 27 -10 -2001 to 01 -11 -2001 in Stamford (CT), New York (NY), Boston (MA) and Cleveland (OH) Editing: approximately 3 months http: //www. interviewwithamerica. com/ Fh. G Bonn, 26. 02. 2004

The problems Video Documentary Only one final version: what do we show? Who is pro and who is against? What are the numbers? Biased versus objective tradeoff: a choice must be made Fh. G Bonn, 26. 02. 2004

Providing Access to the Source Open Documentary Annotate the material: What question was asked Who answered (social categories) Where Possible query: show me all the answers to question “what do you think of the war in Afghanistan” given by black men Fh. G Bonn, 26. 02. 2004

Rhetorical Tools Open Documentary According to Aristotle, rhetoric is "the ability, in each particular case, to see the available means of persuasion” Three main forms of rhetoric: Ethos, Logos, and Pathos Fh. G Bonn, 26. 02. 2004

Ethos Open Documentary Ethos is appeal based on the character of the speaker. An ethos-driven document relies on the reputation of the author Ethos is used in what Bordwell calls Arguments from the source Fh. G Bonn, 26. 02. 2004

Logos Open Documentary Logos is appeal based on logic or reason. Documents distributed by companies or corporations are logos-driven. Scholarly documents are also often logos -driven Logos is used in what Bordwell calls Subject-centered arguments Fh. G Bonn, 26. 02. 2004

Pathos Open Documentary Pathos is appeal based on emotion. Advertisements tend to be pathosdriven Emotions have a functional role as constraints on our decision making (Damasio) Ethos is used in what Bordwell calls Viewer-centered arguments Fh. G Bonn, 26. 02. 2004

Example Open Documentary “I am never a fan of military action, in the big picture I don’t think it is ever a good thing, but I think there are circumstances in which I certainly can’t think of a more effective way to counter this sort of thing, I suppose there is a point in which certain people play by certain rules and you have to go to their level, I do not think there is any way to resolve this conflict diplomatically” Subject is Controversial The position is Pro or Against Fh. G Bonn, 26. 02. 2004

Using the Logos Open Documentary P R O Concession: I do not like A, but B A = war in Afghanistan (military action) B = only solution reasoning A is the only solution because A G A I N S T Fh. G Bonn, 26. 02. 2004 B 0) A is cruel B 1) the attack was cruel B 2) the response has to be like the attack

Using the Logos Open Documentary Different rhetorical figures, in general enthymemes: War is the best thing to do because it will solve the problem Unique : only A is possible Antithesis : only B and A possible and B not a solution Optimal : A is the best solution Unconditional : A is true Evidence : (A if B) and B is true The reasoning can be explicitly expressed in the interview or belong to the “common sense” Fh. G Bonn, 26. 02. 2004

Using the Pathos Open Documentary P R O A G A I N S T Fh. G Bonn, 26. 02. 2004 How and what emotions can we use to persuade? Cognitive structure of emotions (Ortony) The expressions for happiness, sadness, disgust, surprise, anger and fear are culture-, gender- and age independent (Ekman & Friesen)

Strategies Open Documentary Rhetori c Pathos Replace Fh. G Bonn, 26. 02. 2004 We saw some “local” strategies More high level strategies are possible, like increasing / decreasing or hyperbole Logos Augment To form a longer presentations, higher level strategies are needed

Issues Advantages Reuse the reasoning of the humans, no need for a very complex system like Cyc Disadvantages Extensive annotation is needed Need for a knowledge base Validation User studies? Mimicking the reality Fh. G Bonn, 26. 02. 2004

Previous Work “Media Streams is a system for annotating, retrieving, repurposing, and automatically assembling digital video. It uses a streambased, semantic representation of video content with an iconic visual language interface of hierarchically structured, composable, and searchable primitives. ” AUTEUR: The Application of Video Semantics and Theme Representation for Automated Film Editing Fh. G Bonn, 26. 02. 2004

Related Work `Terminal Time is a history "engine: " a machine which combines historical events, ideological rhetoric, familiar forms of TV documentary, consumer polls and artificial intelligence algorithms to create hybrid cinematic experiences for mass audiences that are different every single time’ Fh. G Bonn, 26. 02. 2004

References Damasio, Antonio - Descartes' Error: Emotion, Reason, and the Human Brain, Avon Books, 1994. Bordwell, D. (1989). Making Meaning - Inference and Rhetoric in the Interpretation of Cinema. Cambridge, Massachusetts: Harward University Press. Davis, M. Media Streams: Representing Video for Retrieval and Repurposing, Massachusetts Institute of Technology, Cambridge, Massachusetts, 1995. Nack, F. AUTEUR: The Application of Video Semantics and Theme Representation for Automated Film Editing, Lancaster University, 1996. Ortony, A. , Clore, G. L. and Collins, A. - The Cognitive Structure of Emotions. Cambridge University Press, 1988. Ekman, P. and Friesen, W. V. , Facial Action Coding System. Consulting Psychologists Press Inc. , 1978. Terminal Time http: //www. terminaltime. com/. Fh. G Bonn, 26. 02. 2004

Discussion Fh. G Bonn, 26. 02. 2004