A Stepwise Modeling Approach for Individual Media Semantics

A Stepwise Modeling Approach for Individual Media Semantics Annett Mitschick, Klaus Meißner TU Dresden, Department of Computer Science, Multimedia Technology Group „Applications of Semantic Technologies“ (AST' 06)

Outline • Introduction • Approach: - Semantics for Personal Media Management - Information Instantiation and Integration - Implemented System Architecture • Conclusion and Future Work 06. 10. 2006 AST 2006 2

Introduction • Media Management @ Home: - Growing amount of digital assets (text, images, video, audio) • complex, high-dimensional - User: • • • non-/semi-professional individual conceptualizations usually minimum efforts to annotate/organize - Tools mostly apply administrative (objective) metadata vs. user interpretation/conceptualization • Goal: automated modeling of essential semantic descriptions from existing metadata to ease annotation work 06. 10. 2006 AST 2006 3

Introduction • Solutions: - Automated feature extraction and indexing techniques to manage complexity of huge multimedia collections • • low-level features Similarity-based Search (Query-by-Example, etc. ) higher-level features Pattern Recognition/Classification (objects, people, genres, etc. ) requires prototype features (user context, user feedback) “Semantic Gap” - Media descriptions based on common metadata standards to enable interoperability & comparability • • MPEG-7 Multimedia Description Schema RDF/OWL representation for semantic modeling - Semantic Desktop • 06. 10. 2006 Personal Information Management based on Semantic Web Technologies AST 2006 4

Approach : : Semantics for Personal Media Management • Knowledge representation - Allow description of background-knowledge (context information like events, activities, actors, locations, etc. ) - Allow integration of domain models and individual extensions (e. g. integration of MPEG-7 MDS, FOAF for description of people, etc. ) - Manageable, straightforward for non-professionals • ABC [Hunter, et al. ] - “facilitate interoperability between metadata ontologies“, “conceptual basis for automated mapping …”, “guidance to communities…” - Base classes to attach domain-specific entities MPEG-7 MDS - Temporal semantics and related actualities (who, what, when, where) evolution/transition of objects 06. 10. 2006 AST 2006 5

Approach : : Semantics for Personal Media Management • Acquisition of as much information as possible from content + context - File information, administrative metadata, structural features, etc. • Information sources depend on application context - naming, availability/accessibility, semantics, etc. of features/attributes e. g. “Date Time Original”/”Date Created”/…, “Creator”/”Author”/…, etc. - properties might be relevant, or redundant in some context e. g. “EXIF: Exposure. Time” and “EXIF: Shutter. Speed. Value” 06. 10. 2006 AST 2006 6

Approach : : Information Instantiation and Integration • First step: - Apply sets of mapping rules to construct media-centric description (MDS) - Mapping rules expandable regarding supported metadata schemes ( developer) Make=FUJIFILM Model=Fine. Pix F 601 FUJIFILM 06. 10. 2006 [(urn: Exif: Make rdf: value ? j) (urn: Exif: Model rdf: value ? k) (urn: Filepath rdf: value ? l) hash. Code(? l, ? t) make. Temp(? x) -> (? t mds: creation. Tool ? x) (? x mds: make ? j) AST 2006 (? x mds: model ? k) (? x rdf: type mds: Device)] . . . . . . FUJIFILM Fine. Pix 7

Approach : : Information Instantiation and Integration • Next steps: - Apply derivation rules (schema level) to establish relationships to actualities and temporalities ( developer, user) - Apply classification rules (instance level) to refine description model regarding known pattern ( user) 06. 10. 2006 AST 2006 8

Approach : : Implemented System Architecture • Design rationale: - Automated import and indexing of media items triggered by file system actions - Registry of media type specific analyzing components providing metadata and feature extraction - Processing of semantics and ontology model wrapped by generic model API independent from RDF/OWL storage and querying solution (Jena, Sesame, …) - Extensible, flexible service architecture plug-in based setup, extension at run-time (OSGi service platform) 06. 10. 2006 AST 2006 9

Conclusion and Future Work • Our approach: - Part of the K-IMM (Knowledge through Intelligent Media Management) project managing private media collections with the help of semantic technologies - Stepwise modeling of resource descriptions to separate distinct problems of construction of media semantics • • Building objective media descriptions from various sources Deriving subjective, user-oriented semantics • Future Work: - Strategies to extend and refine basic rule sets by means of user feedback, context capture - Distribution of modeling steps (Web Services/Agents) - Detailed, comprehensive evaluation 06. 10. 2006 AST 2006 10

Thank you for your attention! Annett Mitschick annett. mitschick@inf. tu-dresden. de Multimedia Technology Group, Department of Computer Science, TU Dresden http: //www-mmt. inf. tu-dresden. de 06. 10. 2006 AST 2006 11