Streaming Architectures for Very Large Collections Jim De

Streaming Architectures for Very Large Collections Jim De. Roest Director Streaming Media Technologies, Research. Channel Assistant Director, University Computing Services University of Washington deroest@cac. washington. edu 206 543 -6343

Who We Are And Support Research. Channel/UWTV - Exploring digital convergence and educational outreach. National Echostar and CATV to over 5 million homes International Vo. D and live streams – I 2 Abilene Experiments - First cinema quality HD over IP KEXP – UW and EMP collaboration exploring digital radio Uncompressed 1. 4 Mb live stream n Real time web playlist generation World music format – DJ choice Faculty, Staff, Student, Clinical, K 20 Other Affiliate/Collaborative Projects

Managing Content Large Collection Problem n Disparate Asset Types HD, Broadcast TV, audio libraries, pictures, rich text n Instantiation: capture, encoding, metadata, storage Multiple sources, formats, schema n Interfaces TV post production, radio DJ, viewer/listener n Discovery and Administration Naming, categorization, relations, internationalization Distributed management, sharing, searching n Access Controls Authentication, authorization, rights, privacy, security

Media. Web Project 18 month technology/requirements review Requirements n Interviewed engineers, DJ’s, librarians and consumers Reviewed vendor products n Proprietary, 70% solution, consumer specific Created prototype during review process n Pluggable, scalable, standards based architecture Buy, build, both, call it quits? n Decision to go with prototype

UW Media. Web – Digital Media Asset Management System Asset Management Services Media Store Remote Stream Manager Transcoding Metadata Creation, Admin TV Broadcast Services Disk Cache Request Director Radio DJ Services Robotic Tape Asset Metadata Delivery Services Image, Quicktime, Windows, Real, HD, Webcasting …

Core Services Media Store n HSM online/nearline multi-vendor High Performance Storage System (HPSS) n n n Gigabit interconnects. Distributed, redundant, recoverable Block I/O stream to servers (smoke and mirrors Metadata Service n SQL, directory, imbedded objects, access control, rights Stream Manager n Maintains asset locality state and aging Request Re-Director n Re-direct requests to target stream or application service

Feeds Video Airtime/Dubtime to 50 Mb MPEG 2 i-frame base n Louth/Mav triggers/APIs Metadata generation n Looking at Virage plugins Audio CD ripping raw audio base Metadata map CDDB sites (Muse, AMG, …) DJ metadata support Transcode from base formats as needed

Metadata Schema – How rich can we afford Consumer specific Standards – Dublin Core, MPEG 7 Vendor (Muse, AMG, Louth playlists, …) Draft submitted for comment I 2 -Vid. Mid Vo. D and Vi. De Access/Rights controls Vendor specific, MPEG 21 Auth*2 – Net. ID Pubcookie, Shibboleth, … Imbedded – Honored by clients Trust Metadata Objects SQL (base), LDAP, Imbedded objects/methods

Delivery Streaming Servers Standard file system or IO API required Stream initiated as first blocks arrive to local cache Tape, disk, OS and network speed vs. hybrid file systems Selectable stream stub support First +30 seconds of clip hide mount time content on tape Low bit rate (no MPEG 1) Quicktime, Real, Windows Media High bit rate Windows Media – KEXP 1. 4 Mb live stream Optivision – RC 5. 6 Mb live stream to remote sites MPEG 2 – Concurrent Computer Xstreme 5. 6 Mb Vo. D MPEG 4 – Looking at IBM Videocharger – interface issues HD – In-house (>270 Mb), other vendor lower bit rates

Media. Web Project Status Prototype operational with simple search Geographically separated/mirrored server environment Video: RC/UWTV content (~700 hrs MPEG 2, lower bit rates) Audio: Sample audio Refining schema Object experiments Imbedded controls (MPEG 7) Reviewing large vendor schemas Deploying admin tools Building feed systems (RC, +35, 000 disc KEXP library) Incorporating faculty/student mgt (expiration, quotas, …) Evaluating other large content requests Ongoing evaluation, scaling, benchmarking architecture Looking for opportunities to collaborate!

Experiments UW CSci animation teaching tool “How Networks Change Teaching: A Case Study” Louis Fox, Kris Martin, Barbara Mones-Hattal I 2 VIMM Friday, October 5 th 1: 30 EDT KEXP live stream archives KEXP automated playlist EMP live webcasting and archival UW Catalyst collection interface HD IP Videoconferencing