Скачать презентацию Dagstuhl Seminar Semantics for the Web Dagstuhl 22 Скачать презентацию Dagstuhl Seminar Semantics for the Web Dagstuhl 22

2d4bdae5d656c7ba55acbb9761d32f98.ppt

  • Количество слайдов: 62

Dagstuhl Seminar Semantics for the Web Dagstuhl, 22 March, 2000 Generating Virtual Webpages Wolfgang Dagstuhl Seminar Semantics for the Web Dagstuhl, 22 March, 2000 Generating Virtual Webpages Wolfgang Wahlster German Research Center for Artificial Intelligence, DFKI Gmb. H Stuhlsatzenhausweg 3 66123 Saarbruecken, Germany phone: (+49 681) 302 -5252/4162 fax: (+49 681) 302 -5341 e-mail: [email protected] de WWW: http: //www. dfki. de/~wahlster © W. Wahlster, DFKI

Three Generations of Web Sites First Generation Second Generation Interactive Web Sites Static Web Three Generations of Web Sites First Generation Second Generation Interactive Web Sites Static Web Sites Third Generation Virtual Webpages Java. Scripts and Applets Fossils cast in HTML Netbots, Information Extraction, Presentation Planners Database Access and Template-based Generation User Modeling, Machine Learning, Online Layout Dynamic Web Sites Adaptive Web Sites © W. Wahlster, DFKI

What is a Virtual Web Page? Virtual Memory, Virtual Relation, Virtual Reality. . . What is a Virtual Web Page? Virtual Memory, Virtual Relation, Virtual Reality. . . A Virtual Web Page l is generated on the fly as a combination of various media objects from multiple web sites or as a transformation of a real web page. l looks like a real web page, but is not persistently stored. l integrates generated and retrieved material in a coordinated way. l can be tailored to a particular user profile and adapted to a particular interaction context. l has an underlying representation of the presentation context so that an Interface Agent can comment, point to and explain its components. © W. Wahlster, DFKI

Ai. A: Information Integration for Virtual Webpages PAN Travel Agent Andi Car Route Planner Ai. A: Information Integration for Virtual Webpages PAN Travel Agent Andi Car Route Planner Yahoo News Server Yahoo Weather Server Hotel Guide Gault Millau Restaurant Guide © W. Wahlster, DFKI

Virtual Webpage Retrieved from 5 Different Servers © W. Wahlster, DFKI Virtual Webpage Retrieved from 5 Different Servers © W. Wahlster, DFKI

The Combination of Retrieved and Generated Media Objects for Virtual Webpages Multi-Domain Problem Specs The Combination of Retrieved and Generated Media Objects for Virtual Webpages Multi-Domain Problem Specs Multiple Data Sources Information Structures l Relations, Lists l KR Terms NETBOT Retrieved Results Distributed Information Media Objects l Texts, Sounds, Videos l Pictures, Maps, Animations © W. Wahlster, DFKI

The Combination of Retrieved and Generated Media Objects for Virtual Webpages Information Structures l The Combination of Retrieved and Generated Media Objects for Virtual Webpages Information Structures l Relations, Lists l KR Terms Select Canned Media Objects Design New Media Objects l Icons, Clip Art l Frames, Sounds l Graphics, Animation l Text, Speech, Mimic Select & Design Retrieved Results Media Objects l Texts, Sounds, Videos l Pictures, Maps, Animations Coordinate Media Objects l Temporal Synchronization l Spatial Layout Transform Media Objects l Clip, Convert, Abstract l Zoom, Pan, Transition Effects Reuse & Transform © W. Wahlster, DFKI

Operational Models of Referential Semantics for Robots and Netbots Robot Netbot Set of Subsumption Operational Models of Referential Semantics for Robots and Netbots Robot Netbot Set of Subsumption Relations in an Ontology “Screw” “Departure Time” Set of Recognizers Physical Objects Screw 1 . . . Screw N Set of Wrappers WWW Objects DT 1 . . . DT N © W. Wahlster, DFKI

Enhancement of User Interfaces through Personalization System is able to flexibly tailor presentations to Enhancement of User Interfaces through Personalization System is able to flexibly tailor presentations to the individual user and the current situation. An animated character serves as “Alter Ego” of the presentation system. Personalized Presenters at DFKI © W. Wahlster, DFKI

The Role of Ontological Annotations for the Generation and Analysis of virtual Webpages with The Role of Ontological Annotations for the Generation and Analysis of virtual Webpages with Ontological Annotations Webpages without Ontological Annotations Information Extraction Agent Presentation Planner Virtual Webpage Presentation Agent Persona With Ontological Annotations in: SHOE, OML, XOL, OIL and Persona Annotation in PML Information Extraction Agents Tri. AS © W. Wahlster, DFKI

Video Dresdner Bank © W. Wahlster, DFKI Video Dresdner Bank © W. Wahlster, DFKI

The Generation of Virtual Webpages with PAN and Ai. A Hotel Agent Address Map The Generation of Virtual Webpages with PAN and Ai. A Hotel Agent Address Map Agent Ai. A Pictures and Graphics Pieces of Text Trip Data Netbot PAN Presentation Planner Components of virtual Webpages Coordinates for Pointing Gestures Input for Speech Synthesis Icons for Hyperlinks Weather Agent Train & Flight Scheduling Agent Virtual Web Presentation Persona Server Constraintbased Online Layout Major Event Agent © W. Wahlster, DFKI

Persona as a Personal Travel Consultant © W. Wahlster, DFKI Persona as a Personal Travel Consultant © W. Wahlster, DFKI

AIA’s presentation planner has been extended to accommodate for various target platforms through the AIA’s presentation planner has been extended to accommodate for various target platforms through the introduction of a mark-up language layer Presentation Planner PETPML PET Persona Player Agent Script MS-Agent Controller SMIL Player WMLBrowser © W. Wahlster, DFKI

PPP’s Persona Server implements a generic Presentation Agent that can be easily adapted to PPP’s Persona Server implements a generic Presentation Agent that can be easily adapted to various applications Visual Appearances Behaviors l l Presentation Gestures Reactive Behaviors Idle-time actions Navigation actions Hand-drawn Cartoon Bitmaps Persona Server Auditory Characteristics l Sound effects, auditory icons l Voice: male, female Video Bitmaps Generated Bitmaps from 3 D-Models © W. Wahlster, DFKI

Classification of Persona Gestures Gesture Catalogue Talking Posture 1 • cautious, hesitant • appeal Classification of Persona Gestures Gesture Catalogue Talking Posture 1 • cautious, hesitant • appeal for compliance • avoids body-gestures Talking Posture 2 • active, attentive • self-confident • uses body-gestures © W. Wahlster, DFKI

Context-Sensitive Decomposition of Persona Actions High-Level Persona Actions take-position (t 1 t 2 ) Context-Sensitive Decomposition of Persona Actions High-Level Persona Actions take-position (t 1 t 2 ) Context-Sensitive Expansion (including Navigation Actions) Decomposition into Uninterruptable Basic Postures Bitmaps point-to (t 3 t 4 ) move-to (t 1 t 2 ) r-stick-pointing (t 3 t 4 ) r-turn (t 1 t 21 ) r-hand-lift (t 3 t 31 ) r-step (t 21 t 22 ) f-turn (t 22 t 2 ) . . r-stick-expose (t 31 t 4 ) . . . © W. Wahlster, DFKI

Extensions of the Representation Formalism Distinction between production and presentation acts (i. e. Persona- Extensions of the Representation Formalism Distinction between production and presentation acts (i. e. Persona- or display acts) Explicit representation of qualitative and quantitative constraints Production Act Presentation Act Introduce Create. Graphics S-Show S-Position Elaborate-Parts S-Wait Label S-Create. Window S-Depict S-Point S-Speak Qualitative constraints: Create-Graphics meets S-Show, . . . Metric constraints: 1 <= Duration S-Wait <= 1, . . . © W. Wahlster, DFKI

Persona Presents an Automatically Designed Business Chart © W. Wahlster, DFKI Persona Presents an Automatically Designed Business Chart © W. Wahlster, DFKI

Virtual Webpage with Animation Effects Based on a Single GIF Image © W. Wahlster, Virtual Webpage with Animation Effects Based on a Single GIF Image © W. Wahlster, DFKI

Transition Effects in a Series of Retrieved Pictures © W. Wahlster, DFKI Transition Effects in a Series of Retrieved Pictures © W. Wahlster, DFKI

PET: Persona-Enabling Toolkit Objective: l Enable non-professional computer users to populate their web pages PET: Persona-Enabling Toolkit Objective: l Enable non-professional computer users to populate their web pages with lifelike characters PET comes with: l a set of characters and basic gestures l an easy-to-learn Persona markup language Developer’s PET will include: l a character design tool which enables users to build their own characters Technical Realization: l Based on XML and Java © W. Wahlster, DFKI

The Persona Markup Language Specification of Persona actions <html> <head> Specification of the <title> The Persona Markup Language Specification of Persona actions Specification of the Persona Test character to be used Features: – XML-based – easy to learn © W. Wahlster, DFKI

Tasks of PET-Parser l Preprocessing (required for non-standard HTML pages) l Identification of Persona Tasks of PET-Parser l Preprocessing (required for non-standard HTML pages) l Identification of Persona Tags PET-Generator l Replacement of Persona Tag with Java Applet l Determination of parameters for Java Applet l Realization of Persona active elements as Java. Script l Generation of DHTML layers (transparency of Applet box) l Generation of data, e. g. audio files for speech output l Generation of scripts © W. Wahlster, DFKI

Functional View of PET Bitmaps Webpage with Reference to Java Applet URL of Webpage Functional View of PET Bitmaps Webpage with Reference to Java Applet URL of Webpage with Persona Tag Persona Test PET Application Server PET Parser PET Generator . . . . . . Persona Scripts waitscreen 4 gesture greet 0 0 null gesture laugh 0 0 null. . . Persona Engine Behavior Monitor Audio Data Event Handler Character Composer © W. Wahlster, DFKI

The Bidirectional Control Flow on Persona-Enabled Webpages Triggers actions of the Persona l Mouse The Bidirectional Control Flow on Persona-Enabled Webpages Triggers actions of the Persona l Mouse Clicks l Mouse Movements l Text Input l Speech Input l Menu Input l Direct Manipulation Input Web Persona Triggers operations on elements of the webpage © W. Wahlster, DFKI

Video 911 & Boxter © W. Wahlster, DFKI Video 911 & Boxter © W. Wahlster, DFKI

Persona Active Elements (PAE) l Active Images An active image starts a persona action Persona Active Elements (PAE) l Active Images An active image starts a persona action when clicked. l Addressable Objects An addressable object is an object which can be addressed and manipulated by Persona via its name and its position. some HTML elements © W. Wahlster, DFKI

Spatial and Temporal Coordination of Multimedia Presentations • Multimedia coordination in previous systems: – Spatial and Temporal Coordination of Multimedia Presentations • Multimedia coordination in previous systems: – no declarative representation of spatial and temporal layout – no synchronization of Persona actions with other dynamic multimedia objects • Solution: Use Pre. Plan for automated generation of SMIL expressions – Synchronized Multimedia Integration Language – officially recommended by W 3 consortium © W. Wahlster, DFKI

Approach • High-level specification of temporal and spatial constraints within the operators of the Approach • High-level specification of temporal and spatial constraints within the operators of the Presentation Planner Pre. Plan • Use of the incremental constraint solving toolkit Cassowary – Uniform treatment of spatial and temporal constraints – Transformation of qualitative constraints into metric constraints – Integration of a backtracking mechanism to handle disjunctions © W. Wahlster, DFKI

Processing Steps Decomposition of presentation goals into elementary acts and collection of spatial and Processing Steps Decomposition of presentation goals into elementary acts and collection of spatial and temporal constraints Constraint propagation using extended Cassowary constraint solver Representation of spatial and temporal layout as SMIL constructs © W. Wahlster, DFKI

Example of a Plan Operator (define-plan-operator : header (A 0 (Show. Presentation ? topic)) Example of a Plan Operator (define-plan-operator : header (A 0 (Show. Presentation ? topic)) : constraints (*and* (BELP (Illustrates ? video ? topic)) (BELP (Video ? video)) (BELP (Sets-to-Music ? audio ? topic)) (BELP (Audio ? audio)) (BELP (Summarizes ? title ? topic))) : inferiors (A 1 (SAdd. Smil. Code (? video))) (A 2 (SAdd. Smil. Code (? audio))) (A 3 (SAdd. Smil. Code (? title))) : temporal ((A 1 (d) A 3) (2 <= begin A 1 - begin A 2)) : spatial ( (aligntop A 1) (centerh A 3) (20 <= top A 1 - bottom A 3 <= 20)) © W. Wahlster, DFKI

Representation of Spatial and Temporal Constraints • Spatial constraints – Qualitative constraints (Image 1 Representation of Spatial and Temporal Constraints • Spatial constraints – Qualitative constraints (Image 1 Left. Of Image 2), (Image 1 Top. Align Image 2) – Quantitative constraints (4 < Top Image 1 - Top Image 2) • Temporal constraints – Qualitative constraints Allen constraints: (meets, before, overlaps, . . . ) – Quantitative constraints Linear inequalities: (Begin Audio 1 - End Audio 2 < 3), (2 <= Duration Audio 1 <= 6) © W. Wahlster, DFKI

Resulting SMIL-Document © W. Wahlster, DFKI

Interactive Web Demo © W. Wahlster, DFKI Interactive Web Demo © W. Wahlster, DFKI

Synchronization of Persona Actions with other Media Objects Some Examples: • point to an Synchronization of Persona Actions with other Media Objects Some Examples: • point to an object when it appears in a video • comment on a video 3 seconds after it has started • Smile when video appears on the screen • repeat lip movements until audio stops © W. Wahlster, DFKI

Synchronization of Persona Actions (define-plan-operator : header (A 0 (Show. Presentation ? topic)) : Synchronization of Persona Actions (define-plan-operator : header (A 0 (Show. Presentation ? topic)) : constraints (*and* (BELP (Describes ? audio ? topic)) (BELP (Audio ? audio)) (BELP (Talking. Gesture ? video)) (BELP (Repeat. Gesture ? video)) : inferiors (A 1 (Present. Picture. Sequence (? topic))) (A 2 (SAdd. Smile. Code (? audio))) (A 3 (SAdd. Smil. Code (? video))) : temporal ((A 2 (d) A 1) (2 <= begin A 2 - begin A 1) (A 2 (e) A 3)) : spatial ( (aligntop A 1) (alignleft A 1) (1 <= bottom A 0 - bottom A 3 <= 1) (1 <= right A 0 - right A 3 <= 1)) © W. Wahlster, DFKI

SMIL Specification for Persona Presentation <smil> <head> <meta name= SMIL Specification for Persona Presentation © W. Wahlster, DFKI

Resulting Timeline Diagram . . At archeological finds in Schwarzenacker, . . © W. Resulting Timeline Diagram . . At archeological finds in Schwarzenacker, . . © W. Wahlster, DFKI

Using SMIL to Synchronize Persona Actions © W. Wahlster, DFKI Using SMIL to Synchronize Persona Actions © W. Wahlster, DFKI

Video Otto 1 © W. Wahlster, DFKI Video Otto 1 © W. Wahlster, DFKI

Information Extraction Agents • Information Filtering • Information Retrieval • Information Integration identify relevant Information Extraction Agents • Information Filtering • Information Retrieval • Information Integration identify relevant documents –. . . – identify and extract relevant pieces of information – transform them into canonical form wrappers • operational descriptions of a target concept • abstract from concrete occurrence within document • robust against modifications © W. Wahlster, DFKI

The Trainable Information Agents Framework specifications results combination of The Trainable Information Agents Framework specifications results combination of "classical" problem-solving methods and information agents Application info requests User requests training preferences/heuristics planning knowledge user preferences info domain ontology Browser Info. Broker improved dialog guidance PBD dialog info requests site info/ upd ate info or script query Info Extraction Trainer planning, optimization, Web site annotations and execution rma info ite s © W. Wahlster, DFKI

High Degree of Parallelism of Queries © W. Wahlster, DFKI High Degree of Parallelism of Queries © W. Wahlster, DFKI

Knowledge about a Webpage Shared by User and Agent common part (usable for communication) Knowledge about a Webpage Shared by User and Agent common part (usable for communication) structural visual/semantic Naive User procedural Learning Annotation Agent © W. Wahlster, DFKI

Video PAN © W. Wahlster, DFKI Video PAN © W. Wahlster, DFKI

Example - Ontology Train_Connection [ from =>> Location; to =>> Location; travel_date =>> Date; Example - Ontology Train_Connection [ from =>> Location; to =>> Location; travel_date =>> Date; time =>> Time; depart_time =>> Time; arrive_time =>> Time; cost =>> Price; travel_duration =>> Duration; info_url =>> URL; . . . ] © W. Wahlster, DFKI

Query Planning - I • states: information states – concepts / attributes and instantiations Query Planning - I • states: information states – concepts / attributes and instantiations • operators: querying schemes – preconditions (´+´) and effects (´-´) to time arrive_time travel_duration < +, +, -, -, -, u, - > from travel_date depart_time cost info_url © W. Wahlster, DFKI

Query Planning - II • merging of alternative plans with identical intermediate states • Query Planning - II • merging of alternative plans with identical intermediate states • (a) op 2 • (a, b) op 3 • op 1 (a, c) op 4 • (a, b, c, d) • • (a, b, c, d, e) • (a, b) op 3 (a, b, c, d, f) • (a, c) op 4 • (a, b, c, d) op 6 op 5 merging • • op 1 • (a)op 2 op 5 op 6 • (a, b, c, d, e) • (a, b, c, d, f) © W. Wahlster, DFKI

Query Planning - III Top State Ontology City. value = München City. language = Query Planning - III Top State Ontology City. value = München City. language = German . . . String . . . Operators City. Name 1 Language City value – – + + < String, Language, String, Language > babelfish . . . language " c Î pre (op ) $ i : S 0 = i : : c ¢ Ù c c ¢Ù S 0 = Int ( op ) © W. Wahlster, DFKI

Query Plan Visualization • Features – alternative queries – past states – future states Query Plan Visualization • Features – alternative queries – past states – future states – state descriptions – PBD requests – accept / reject PBD request – assessment of plans – expected completion time © W. Wahlster, DFKI

Using the presentation planner to serve mobile users The new challenge accommodate for device-specific Using the presentation planner to serve mobile users The new challenge accommodate for device-specific display and interaction limitations (e. g. 60*90 pixel displays) Approach - develop new designs for presenting information on mobile devices - identify elementary design elements - define new presentation strategies for the automated composition of designs © W. Wahlster, DFKI

Application example: Route Descriptions over the Mobile Phone Planner a selected path gets translated Application example: Route Descriptions over the Mobile Phone Planner a selected path gets translated by the planner into a sequence of WML pages to be displayed on a mobile phone “vertical bar view” “runway view” “isometric view” © W. Wahlster, DFKI

Video Otto 2 © W. Wahlster, DFKI Video Otto 2 © W. Wahlster, DFKI

Use of a Life-like Character for Electronic Commerce Digital Assistant Selector © W. Wahlster, Use of a Life-like Character for Electronic Commerce Digital Assistant Selector © W. Wahlster, DFKI

Simulated Dialogues as a Novel Presentation Technique • Presentation teams convey certain rhetorical relationships Simulated Dialogues as a Novel Presentation Technique • Presentation teams convey certain rhetorical relationships in a more canonical way – Provide pros and cons • The single presenters can serve as indices which help the user to classify information. – Provide information from different points of view, e. g. businessman versus tourist • Presentation teams can serve as rhetorical devices that allow for a continuous reinforcement of beliefs – involve pseudo-experts to increase evidence © W. Wahlster, DFKI

Presentation Teams I recommend you this SLX limousine. © W. Wahlster, DFKI Presentation Teams I recommend you this SLX limousine. © W. Wahlster, DFKI

Underlying Knowledge Base • Representation of domain – FACT attribute car_1 consumption_car_1 • Value Underlying Knowledge Base • Representation of domain – FACT attribute car_1 consumption_car_1 • Value dimensions for cars adopted from a study of the German car market – safety, economy, comfort, sportiness, prestige, family and environmental friendliness – FACT polarity consumption_car_1 economy negative • Difficulty to infer implication of dimension on attribute – FACT difficulty consumption_car_1 economy low © W. Wahlster, DFKI

Example of a Dialogue Strategy Question: How much gas does it consume? Answer: It Example of a Dialogue Strategy Question: How much gas does it consume? Answer: It consumes 8 l per 100 km. Negative Response: I’m worrying about the running costs. Dampening Counter: Forget about the costs. Think of the prestige! Header: (dampening_counter ? agent ? prop ? dim) Constraints: (*and* (positive ? agent) (pol ? prop ? other_dim positive)) Inferiors: (Speak ? agent (“Forget about the ” ? dim “!”)) (Speak ? agent (“Think of the ” ? other_dim “!”)) © W. Wahlster, DFKI

Current and Future Work: Multiple Interface Agents for User-adaptive Decision Support User-Adaptive Search Planning Current and Future Work: Multiple Interface Agents for User-adaptive Decision Support User-Adaptive Search Planning . . . weighted propositions Multiple Decision Support Agents Spare parts for this car are rather expensive! But, it’s fast! © W. Wahlster, DFKI

Conclusion The generation of virtual webpages is heavily based on ontological annotation The plan-based Conclusion The generation of virtual webpages is heavily based on ontological annotation The plan-based approach to generating virtual webpages guarantees high-level ontological annotations Ontological annotations are needed not only for information extraction agents but also for presentation agents Realization procedures and wrappers form an important part of the referential semantics of objects on the web Using presentation planning with temporal and spatial constraint processing the low-level media synchronization can be done automatically using SMIL annotations see www. dfki. de/~wahlster/semanticweb © W. Wahlster, DFKI

The Web. Persona and Virtual Webpages Team at DFKI Computer Science: Graphics Design: Elisabeth The Web. Persona and Virtual Webpages Team at DFKI Computer Science: Graphics Design: Elisabeth André Mathias Bauer Dietmar Dengler Alexander Kröner Marcus Meyer Jochen Müller Stephan Neurohr Gaby Paul Thomas Rist Wolfgang Wahlster Bernhard Kirsch Renato Orsini Peter Rist Cognitive Psychology: Susanne van Mulken © W. Wahlster, DFKI