Types and Methods of Content Adaptation T-110 456

Types and Methods of Content Adaptation T-110. 456 Next Generation Cellular Networks Timo-Pekka Viljamaa 2. 3. 2005

Agenda • Motivation for Content Adaptation • Types of Content Adaptation • Methods of Content Adaptation • Real Life Examples • Conclusions • Further Reading

Motivation for Content Adaptation • Terminal diversity • Different display size, bandwidth, memory, processing power, UI, etc. • Available network environment • Depending on the current location and the terminal support • UMTS/GPRS/GSM Data • User preferences • Information presenting styles • e. g. scrollable/splitted to several views • Time issues • e. g. see utilitarian content quickly/wait for flashy content • Cost issues • Definition of Multimedia Unit (MMU) • The unit of data transmitted over a network containing one or more multimedia content elements • e. g. multimedia message in MMS, elements of a web page (typical composed of several MMUs)

Types of Content Adaption • Format Adaption • Characteristics Adaptation • Appearance Adaptation • Size Adaptation • Encapsulation Adaptation

Format Adaption • Converting original content format to the format also understandable by the receiver • e. g. JPG to GIF, MPEG 4 to MPEG, AAC to MP 3 football. jpg football. gif

Characteristics Adaptation • Modifying media object’s characteristics while remaining within a given format • e. g. image/video resolution, frame/bit rate, number of colors 295 x 379 590 x 758

Appearance Adaptation • Modifying the content of an multimedia unit (MMU) for the purpose of changing how it looks or sounds • Needed to conform to the capabilities of the receiver and may even increase usability • e. g. condensed version from a web page, portrait to landscape mode Google (Opera for Mobile S 60) Google (Internet Explorer)

Size Adaptation • Reducing the size of an MMU message to match the capabilities of the receiver and the underlying network environment: 1. Removal of some MMU objects • e. g. remove an image from an MMU message 2. 3. 4. Changing the encapsulation • e. g. split an MMU message to several smaller ones Converting to another format • e. g. MP 3 to AAC Characteristics adaptation • e. g. quality of the image, number of colors

Encapsulation Adaptation • Converting MMU messages from one ”application protocol” to another and therefore involving repacking a message without altering any of the media content • e. g. splitting an e-mail to a sequence of several SMS messages From: sender@foo. com To: receiver@bar. com From: sender@foo. com foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar To: receiver@bar. com E-mail foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar foo bar bar foo bar foo bar foo bar foo bar SMS x 2

Methods of Content Adaptation • Multimedia Transcoding • Content Selection • Rendering at the Client • Hybrid Approaches

Multimedia Transcoding • Modifying the properties of multimedia object to meet the capabilities of the terminal • Usually automatic process • The behavior of the adaptation system is programmed in advance • Can include all previously presented types of adaptation • Multimedia Transcoding Architecture Requests/Responses Application-specific Controller Capability negotiation Extracts capabilities info from protocol, e. g. from User. Agent Header or UAProf Capability DB Contains the capabilities of different terminals. Adaptation Policies Eng. Decides how the content should be adapted. Policies Plugins Media Adaptation Eng. Applies transformation to the media content or selects the right version. Media Plugins MMS Browsing GIF JPEG SIP IM Rich calls WML H. 263

Multimedia Transcoding (2) • Transcoding of audio visual content • Decode object Modify uncompressed Encode to desired format • Also partially decoded objects can be modified • Scalable formats improve the quality and the performance • Transcoding of nonaudiovisual content • Nonaudiovisual content is often XML applications • Can be modified and/or converted to other XML applications using XSLT technology or DOM/SAX interfaces • Transcoding of procedural code • Complexity makes transcoding unattractive • Better to make the code itself adaptive or provide different versions • Advantages and drawbacks + Increased usability + Automatic process - May require a lot of processing resources - Adapted results may not be acceptable or usable - Copyright issues

Content Selection • Multiple versions or modality of each multimedia object is stored in the server and server selects the best version for the given terminal Re so l ut io n • The Infopyramid • A representation scheme providing a multimodal, multiresolution representation hierarchy for multimedia content • Modality axis provides the same information under different media modes • Resolution axis provides, for a given modality, the content at different quality levels Title Short story Full story Text 160 x 40 120 kbps QVGA Image 8 kpbs 96 kbps 1 Mbps Video Modality 64 kbps 128 kbps Audio

Content Selection (2) • The Customizer • After a client’s request selects the best content representation supported by the client terminal’s capabilities and environment from the Infopyramid • Makes content selection decisions in accordance with adaptation policies • Seperating content and its representation • Same content with several representation alternatives • e. g. CSS and XSLT • Advantages and drawbacks + Selecting less processing intensive than transcoding + Increased usability + Automatic process after the creation of all versions + The quality of delivered content has been validated by the author + Solves legal issues - May require a lot of work from the author to manage the content - May require a lot of space to store all the different versions

Rendering at the Client • Adapting the content before the user sees it on the screen by rendering it in a suitable way for the given terminal • Theorically can fit content to variety of different screen size • Binary content can be adapted by the renderer • The renderer also handles alternative content • e. g. alternative text instead of an image if ”Show images” turned off • Advantages and drawbacks + The client knows its capabilities best and has the up 2 date information + Content is received on the terminal and can be readapted if needed - Requires processing resources in the terminal Not suitable for lowend devices

Hybrid Approaches • Mixing different adaptations methods together at the same time • e. g. transcoding can be a part of a content selection system, transcoding and content selection can be performed on the media content and the final layout is left to the terminal when rendering

Real Life Examples • Apache Cocoon • http: //cocoon. apache. org/ • EU Project Consensus • http: //consensus-online. org/ • Nokia MMS Solution • http: //www. nokia. com/nokia/0, , 56882, 00. html • Opera for Mobile, Small-Screen Rendering™ technology • http: //www. opera. com/products/mobile/smallscreen/

Conclusions • Multimedia transcoding works well for automatic adaptation of simple media content • But often fails when the content is more sophisticated and requires more processing resources However only option if the client doesn’t perform adaptation • Content selection gives the author more control on the adapted versions of the content • But requires knowledge of target terminal and some work to create the different versions and establish the selection rules Who will do these versions and keep them up 2 date? • Rendering at the client should work well because the client obviously knows its capabilities best and has the up 2 date information about itself • But requires large amount of processing resources in the terminal Is the consuming of processing resources worth it and what to do with low-level devices? • No best method for adapting content that suits all situations A hybrid approach might work best

Further Reading • Adapting Multimedia Internet Content for Universal Access (Mohan, R. , Smith, J. R. , Chung-Sheng Li) • http: //ieeexplore. ieee. org/xpl/abs_free. jsp? ar. Number=00748175 • MPEG-21 Multimedia Framework • ISO/IEC TR 2100 -1: 2004 • http: //www. iso. org/iso/en/Catalogue. Detail. Page. Catalogue. Detail? CS NUMBER=40611&ICS 1=35&ICS 2=40&ICS 3= • MPEG-21: Goals and Achievements (Burnett, I. , Van de Walle, R. , Hill, K. , Bormans, J. , Pereira, F. ) • http: //ieeexplore. ieee. org/xpl/abs_free. jsp? ar. Number=1237551 • Interoperable Adaptive Multimedia Communication (Timmerer C. , Hellwagner, H. ) • http: //ieeexplore. ieee. org/xpl/abs_free. jsp? ar. Number=01377105 • SVG Mobile (SVG Basic & Tiny) • http: //www. w 3 c. org/TR/SVGMobile/ • Flash Lite • http: //www. macromedia. com/software/flashlite/

Questions? Thank you!