Automatic Generation of Animated Presentations Gur Wizenfeld Advanced

Automatic Generation of Animated Presentations Gur Wizenfeld Advanced Software Tools Seminar – June 2004 Gur Wizenfeld - Advanced Software Tools Seminar 2004

Agenda n Introduction generation to animated presentations – Stating the problem – Applications n Solution Considerations n Approaches – Hierarchical task decomposition (Karp & Feiner) Gur Wizenfeld - Advanced Software Tools Seminar 2004

Methodology n Purpose – – n Introduce the problem and its complexity Introduce and evaluate various solution approaches Main Resources – – – Steven Drucker, Intelligent Camera Control for Graphical Environments [3]. Peter Karp and Steven Feiner, Issues in the Automated Generation of Animated Presentations [5]. Peter Karp and Steven Feiner, Automated Presentation Planning Using Task Decomposition with Heuristic Reasoning [6]. D. A. Friedman and Y. A. Feldman, “Knowledge-Based Cinematography and its Applications, ” Proc. 16 th European Conf. Artificial Intelligence (ECAI 2004), August 2004 [4]. David Christianson et al. , Declarative Camera Control for Automatic Cinematography [2]. Maneesh Agrawala, “Designing Effective Step-By-Step Assembly Instructions”, SIGGRAPH 2003 [1]. Gur Wizenfeld - Advanced Software Tools Seminar 2004

Introduction to Animated Presentations Generation Gur Wizenfeld - Advanced Software Tools Seminar 2004

Stating the Problem n Given a script, describing a dynamic virtual 3 D environment, generate an animated presentation of the script’s relevant information. Gur Wizenfeld - Advanced Software Tools Seminar 2004

Example: A Soccer Game Versatile content n Versatile style n – Slow motion – Multiple camera angles – Textual annotations Dynamic n Parallel occurrences n Real-time (unpredictable) n Gur Wizenfeld - Advanced Software Tools Seminar 2004

Stating the Problem (Detailed) n Environment – Spatial organization of 3 D objects – 3 D representation of objects (e. g. , polygonal mesh) – Dynamic – Objects may be capable of modifying the environment. Gur Wizenfeld - Advanced Software Tools Seminar 2004

Stating the Problem (Detailed) n Script – Written in a formal language – Describes all modifications to the environment over time – Possibly annotated – Possibly unpredictable (e. g. , in interactive applications) Gur Wizenfeld - Advanced Software Tools Seminar 2004

Stating the Problem (Detailed) n Presentation – A description of every frame of the animated presentation • A combination of viewports • The contents of each viewport is generated by rendering the environment from a virtual camera’s point of view. – Specifying camera parameters for each viewport in each frame is sufficient. – Possible extensions: • Audio • Textual annotations • Lighting Gur Wizenfeld - Advanced Software Tools Seminar 2004 14: 46

Applications: 3 D Visualization n Virtual environment walkthroughs – Architecture – Virtual malls n Scientific Visualization – Large multidimensional datasets exploration • Medical imaging • Weather simulation Gur Wizenfeld - Advanced Software Tools Seminar 2004

Applications: Entertainment Computer animated movies n Games – Highly interactive Unpredictable script – Rich 3 D environment – Fast Paced – MUDs n • Parallel & distributed • Raises the need for automated summaries Gur Wizenfeld - Advanced Software Tools Seminar 2004

Applications: Design & Planning CAD n Manufacturing process planning n Animation generation n Gur Wizenfeld - Advanced Software Tools Seminar 2004

Solution Considerations Gur Wizenfeld - Advanced Software Tools Seminar 2004

Solution Considerations: User Task n Exploration – Orientation – Navigation – Search/Wander n Planning n Control & Teleoperation Gur Wizenfeld - Advanced Software Tools Seminar 2004

Solution Considerations: Degree of Interaction n Frequency of interaction – Continuous vs. autonomous n Style of interaction – Direct control of camera vs. symbolic specification Gur Wizenfeld - Advanced Software Tools Seminar 2004

Solution Considerations: Supporting Cognitive Capabilities n Presentation viewing and understanding is based on complex underlying perceptive and cognitive mechanisms. – – – n Motion from still images 3 D spatial relations from 2 D projections Recognition Attention Memory (Story understanding) A comprehensive cognitive model would help, but is difficult to construct. Gur Wizenfeld - Advanced Software Tools Seminar 2004

Solution Considerations: Collective Cinematic Experience n Collective human experience of cinematic expression has formed a cinematic language. – Grammar – Rules – Idioms • An Idiom is a stereotypical way to capture some specific action as a series of shots. – Evolving genres Gur Wizenfeld - Advanced Software Tools Seminar 2004 ãTrevor Goring www. famousframes. com

Solution Considerations: Continuity Avoid “jump cuts” (i. e. , cuts between which the camera rotates about the object by less than 30. ) n Do not interrupt a steady, flowing action with a cut (or use time overlapping shots. ) n Maintain the relative screen position of an object across cuts. n – Avoid “line crossing” in a cut (or use an intermediate shot. ) Gur Wizenfeld - Advanced Software Tools Seminar 2004

Example: Avoid Line Crossing Gur Wizenfeld - Advanced Software Tools Seminar 2004 ãAndrew Burward-Hoy www. famousframes. com

Presentation Hierarchy Gur Wizenfeld - Advanced Software Tools Seminar 2004

Requirements Summary n Select Content – Relevance – Summarization n Map Script-time to presentation-time – Parallelism – Pacing n Determine Presentation Hierarchy – Structuring shots to scenes, etc. – Transitions – Viewports layout Gur Wizenfeld - Advanced Software Tools Seminar 2004

Requirements Summary n Camera placement – Visibility • Occlusion avoidance • Projection size – Recognition • Preferred viewing directions (“gaze vectors”) – Maximization of geometric information • Spatial relations • Distortion avoidance • Selection of geometric features n Obey cinematic conventions – Use idioms correctly – Maintain continuity – Adhere to genre rules n Take into account cognitive capabilities and limitations. Gur Wizenfeld - Advanced Software Tools Seminar 2004

Solution Map Script Understanding & Analysis Presentation Generation Presentation Rendering Presentation Gur Wizenfeld - Advanced Software Tools Seminar 2004

Hierarchical Task Decomposition Automated Presentation Planning of Animation Using Task Decomposition with Heuristic Reasoning P. Karp and S. Feiner Gur Wizenfeld - Advanced Software Tools Seminar 2004

Solution Highlights Creates a complete presentation structure (from film level to single shot level. ) n Uses communicative goals to guide presentation generation. n Formalizes and uses cinematic knowledge in a straightforward manner. n Reduces redundant computational effort. n Gur Wizenfeld - Advanced Software Tools Seminar 2004

Hierarchical Task Breakdown Example Gur Wizenfeld - Advanced Software Tools Seminar 2004

Presentation Generation Task Decomposition n Film structure hierarchy is used to guide decomposition of the presentation task. n Communicative Goals define the requirements at each level. Gur Wizenfeld - Advanced Software Tools Seminar 2004

ESPLANADE – Solution Testbed Expert System for PLANning Animation, Design and Editing n Environment: crane activation in a warehouse n Gur Wizenfeld - Advanced Software Tools Seminar 2004

Presentation Processing in Levels n Scene Structuring Techniques (idioms): – Master shot discipline – How-To structure – Causal Structure Gur Wizenfeld - Advanced Software Tools Seminar 2004

Presentation Example n Script: – Move roll-2 with the crane. – Move roll-7 with the crane. – Causal information between button pressing and crane operations. n Communicative Goals: – Show moving of roll-2. – Show moving of roll-7. – Show causal relation between pressing the “up” button and raising the crane’s hoist. Gur Wizenfeld - Advanced Software Tools Seminar 2004

Gur Wizenfeld - Advanced Software Tools Seminar 2004

Gur Wizenfeld - Advanced Software Tools Seminar 2004

Evaluation n Pros – Provides a solution for a complete presentation hierarchy. Can be generalized to support arbitrarydepth hierarchies. n Cons – Resulting presentations are rigid and predictable. – Separates communicative goals from their implementation as idioms. – Formalizes goals and idioms in a “natural” manner. Goals and idioms can be added. – Task decomposition approach reduces computational effort. – Every combination of goals requires a specific idiom. – Does not support conflicts between goals and idioms. Gur Wizenfeld - Advanced Software Tools Seminar 2004

Summary Gur Wizenfeld - Advanced Software Tools Seminar 2004

Conclusions n Use of cinematic rules is crucial. – Rules evolve flexibility and scalability – Rules often conflict contradiction handling Search space can be reduced by an appropriate abstraction. n Symbolic representation is critical for usability. n Script understanding and automatic communicative goal extraction are still a long way ahead. n Gur Wizenfeld - Advanced Software Tools Seminar 2004

References 1. 2. Background 1. [1] M. Agrawala, D. Phan, J. Heiser, J. Haymaker, J. Kilngner, P. Hanrahan and B. Tversky, “Designing Effective Step-By-Step Assembly Instructions, ” In SIGGRAPH 2003. 2. [2] D. B. Christianson, S. E. Anderson, L. He, D. H. Salesin, D. S. Weld and M. F. Cohen, “Declarative Camera Control for Automatic Cinematography, ” Proceedings of the Thirteenth National Conference on Artificial Intelligence, vol. 1, pages 148— 155, 1996. 3. [3] S. M. Drucker, “Intelligent Camera Control for Graphical Environments, ” Ph. D Thesis, MIT, 1994. 4. [4] D. A. Friedman and Y. A. Feldman, “Knowledge-Based Cinematography and its Applications, ” Proc. 16 th European Conf. Artificial Intelligence (ECAI 2004), August 5. 2004 Hierarchical Task Decomposition 1. [5] P. Karp and S. Feiner, “Issues in the automated generation of animated presentations, ” Proceedings of Graphics Interface '90, pages 39 --48, May 1990. 2. [6] P. Karp and S. Feiner, “Automated presentation planning of animation using task decomposition with heuristic reasoning, ” Proc. of Graphics Interface 93, pages 118 -127, 1993. Gur Wizenfeld - Advanced Software Tools Seminar 2004

Thank You Gur Wizenfeld - Advanced Software Tools Seminar 2004

Detours Gur Wizenfeld - Advanced Software Tools Seminar 2004

Detour: Advances in computer games Gur Wizenfeld - Advanced Software Tools Seminar 2004

Detour: Idioms n An Idiom is a stereotypical way to capture some specific action as a series of shots. (Ac. Film. Idiom name approach-idiom : parameter (Ac. Param. Approach : actor 1 : actor 2 : start : stop) (Ac. Film. Shot name shot 1 (Ac. Frag. Go. By name frag 1 : time ? start : primary-moment beginning : entry-pos center : exit-pos out-left. . . ) Gur Wizenfeld - Advanced Software Tools Seminar 2004

Detour: Idioms Gur Wizenfeld - Advanced Software Tools Seminar 2004

Detour: Rendering Crane Tilt Zoom Roll Pan Truck Dolly {X, Y, Z, Rx, Ry, Rz, fov} 7 Gur Wizenfeld - Advanced Software Tools Seminar 2004

Detour: Rendering n Transforms objects from a perspective view in the world space into the screen’s display space. Gur Wizenfeld - Advanced Software Tools Seminar 2004