The MXF Mastering Format A proposal to address

The MXF Mastering Format A proposal to address existing MXF interoperability issues. Clyde Smith, Steve Fish, Bruce Devlin, et al.

Turner Entertainment Networks Serving North America from Atlanta, South America from Buenos Aires, EMEA from London and Asia/Pacific from Hong Kong 8 Feeds ØEast Coast ØWest Coast ØTNT in HD ØBrazil ØArgentina ØMexico ØVenezuela ØPan Regional 13 Feeds ØTCM Domestic ØTCM Canada ØTCM Latin America ØLA Pan Regional ØFrance ØSpain ØUK ØNordic 5 Feeds ØSuper Station East ØU. S. East ØSuper Station West Coast ØWTBS -17 ØU. S. West ØWTBS-DTV Coast ØWTBS Canada 24 Feeds ØPan European ØIndia ØU. S. East Coast ØSpain ØPakistan ØU. S. West Coast ØItaly ØBrazil ØPoland ØArgentina ØRomania ØMexico ØHungary ØLA Pan Regional ØPan Asia ØU. K. ØAustralia ØItaly ØNordic ØTaiwan ØDenmark ØPhilippines ØFrance ØKorea ØGermany ØSpain 11 Feeds ØItaly ØUnited States ØPan European ØBrazil ØLA Pan Regional ØSouth Asia ØPan Asia ØUK ØFrance ØGermany ØPan European ØGermany ØSky ØAfrica ØPan Asia ØIndia ØUK ØNBA-TV Domestic ØNBA-TV International ØNBA-TV HD

Turner Entertainment Networks Distribution Creation and Acquisition “Playout” Operations Warner Brothers Technical Operations Turner Studios and Original Productions Other Program Providers Repurposing Requires Reprocessing Cable Operators Branded Channels SVOD PPV Satellite Operators Branded Channels PPV Turner Broadcasting System Entertainment Operations New Media Interactive On Line Broadband Mobile Phones Games Retail Sales VHS DVD Licensed Merchandise

Operational Strategy Locating key network support functions within their respective markets is a strategy that we’ve employed successfully in London, Hong Kong and Buenos Aires. Our international operations tend to be lean, which makes their staffs highly collaborative. The marketplace dynamics are changing so we must be innovative More and more “TV” content moving online means more choice for consumers and more opportunity for advertisers to reach highly targeted, highly desirable audiences. Game. Tap, CNN Pipeline, Dramavision, Very. Funny. Ads. com, PGATour. com, ACCSelect. com and Toonami Jetstream offer broadband users a mix of familiar and new.

Workflow Key Enablers • I. T. based facilities • Automated Repurposing and Reprocessing • Standardized protocols and formats • MXF and AAF • BUT are they ready for implementation in our operations?

• • We all know Building facilities based on IT workflows saves money and speeds up time to market of products! But…. To succeed you need standards for interoperability If vendors are using their own “optimised” yet different wrapper formats: Program Stream, VOB, Transport Stream, DV, AVI, Quicktime, MP 4, TARGA, MXF OP 1 a (Sony), MXF P 2, OMF, TIFF, etc. And…If many users have their own

We all know • Then eventually all users will pay all the vendors to implement all the formats and all the options to achieve interoperability

The problem – make multiple versions cheaply

MXF • Was created to deliver business value to the industry • Q: So what’s missing? Why can’t we find a solution off the shelf? • A: Too many MXF tools MXF has many options to address to many problems

Principle Developers and Supporters of the MXF Mastering Format

The Goals • Data entry. Data should be entered once at the appropriate point in the asset life cycle. This data should be available to any system looking at that asset. • Data updates. There should be the ability to revise or edit data. Any revisions or edits must be updated across the ‘Inventory’ • Files usable before being closed (i. e. partitioned. ) This allows preview work to start before the write cycle is completed and the file closed. The time between first byte arriving and first use should be reasonable short. • The concept of an asset ‘Inventory’. This is where everything related to a single asset is stored. Asset in this context means a single episode of a show. • The ability to extract a copy of this ‘Inventory’ and be able to send it to an independent site. Once received, it must be importable to the asset management system (which may be different to the source system), based solely on the supplied file. This means it must be self describing with all essence and metadata bound in. One benefit of this is that best of breed local solutions can be employed and reduce reliance on single monolithic systems across organisations.

The Goals • Storage system independent. As assets are expected to be used in many different places at different times there must be no reliance or limit on particular physical storage devices or systems. • Flexible restores. A need was identified for ‘Partial Restore’. This is where a selection of essence tracks from the inventory would be restored to form a new version of the asset. This functionality would be further extended to give ‘Partial Restore’ where a time sliced selection of essence tracks would be restored. Finally, a ‘Partial Restore’ requirement was identified where a group of time sliced sections of a selection of essence tracks would be restored, this last requirement was particularly aimed at clip selection for edit purposes. • Flexible stores. The need to add tracks (essence or metadata) to an ‘Inventory’ without the overhead of having to re-write the entire ‘Inventory’. • Metadata must reside with, and be updated throughout an assets lifecycle. • Essence files to be external. As flexibility of adding (and

The Goals • Files must represent Multiple versions > Different numbers of breaks in a program > Different language variants > Different titles & credits > Different output deliverables from one master • Using MXF must lead to efficiency improvements > Reduction in transfer bandwidths > Reduction in storage requirements > Increased automation > Partial restore direct from tape > Standardised version identification > Greater workflow choices • Risk reduction for the end user > Choice of multiple vendors working to a common standard

Defining the problem • Put all functional blocks on a diagram, label them and link them at a physical / networking level. • For each major workflow create a table that shows the individual actions required in that workflow. Each action should be documented to describe what happens at the physical, essence and data levels. • Convert each action into a UML activity diagram that shows in sufficient detail what is required from the action, and the messages that are likely to flow. • Look at the lifecycle of material

Defining the problem • Look at the lifecycle of material

Defining the problem • Work through some material lifecycle examples > Identify processes > Identify common states > Identify properties of file which reduce complexity > Identify properties of file which improve operational flexibility > Example…

Defining the problem • Identify all the interfaces clearly – an example > Src ID -8 > Dst ID - 15 > Level -a > Worflow ref - 8. 15. a > Description - move from Ingest to playout > Type - Data & Essence > Req. mech - automation > Metadata src - file > Metadata dst - file > Schema - n/a > Essence src - ingest station > Essence dst - on-line server > Notes - FTP based move of MXF wrapped data

Defining the problem • Identify all the interfaces clearly > Src ID - ID of the source of the data > Dst ID - ID of the destination of the data > Level - sub ID to distinguish which workflow is in use > Worflow ref - unique ID of this workflow step > Description - textual description of workflow step > Type - Data / essence / physical / control / SDI > Req. mech - A Trigger e. g. API, hot folder, keyboard, automation > Metadata src - e. g. Business DB, Asset DB, DAM, Automation, file > Metadata dst - e. g. Business DB, Asset DB, DAM, Automation, file > Schema - schema for metadata if available > Essence src - which machine > Essence dst - which machine > Notes - anything else

Parameterizing the specification Programme Component files Parameter Value embed_VBI True – VBI is carried in the MPEG user_data as well as VBI atom encode_VBI False – no rendering of VBI in active picture within the MXF domain video_essence SMPTE 381 M MPEG 2 with the following options: 50 Mbit I frame only 15 Mbit long-GOP, dynamic GOP, nominally 15 frame GOP for 525, 12 frame GOP for 625 systems 25 Mbit long-GOP, dynamic GOP, nominally 15 frame GOP for 525, 12 frame GOP for 625 systems audio_essence SMPTE 382 M uncompressed WAV audio PCM audio shall be mono, stereo or surround uncompressed Broadcast Wave audio –described with a single track and be manipulated without having to demultiplex separate language tracks from an interleaved bundle. ingest_TC LTC – ingested timecode shall take the LTC from the source material DM_ASlang xx. xx. xx – the application specific language tag Simple Programme Version files Parameter Value min_spv_dur 1 second Complex Programme Version files Parameter Value sub_clip_limit 1 second. In a long GOP file, this may start or end within a GOP complex 3 b True.

Document results… • What do the files look like? > List the different physical requirements of the file > List the metadata requirements of the file > Initially document them as a proprietary format • Categorise requirements > Those that are generally applicable – RPzzz recommendation > Those that are business specific – AS-TBS

The Specifics • Refer to files by their function not their OP > Programme Components > Simple Programme Versions > Complex Programme Versions > Programme Inventories

Components, Versions and Inventories ‘Components’ are single essence or data tracks e. g. a Video track, Audio track or Metadata track. ‘Versions’ are pointers to Components and describe how a collection of Components may be played. There can be many different Versions pointing to the same Components. ‘Inventories’ are a repository for all the Versions and Components for a single asset. There should be no duplication of Components within the Inventory. There maybe many similar Components of the same type, for instance there could be three different Videos – High Resolution edit master, Transmission master and a Browse resolution copy. At least three versions would exist to allow each to be played.

Example of Components Programme Components Video Audio Programme Components

Principal – versions link to essence Inventory Simple Programme Version

File Structure Examples Inventory A Simple Programme Version within the Inventory

Inventories, Components and Versions These definitions can be loosely related to Operating Patterns in the following way: Components OP 1 a. Versions OP 2 B or OP 3 B depending on how complex the program structure is. Inventories could OP 3 B or OP 3 C depending on how many versions are in the Inventory. This shows that talking in ‘OPs’ is variable and also that all Operating Patterns are likely to be met in a complex system.

Example – adding a subtitle Simple Programme Version example_vbi 0. mxf (SMPTE 436 M)

Different title/credits A Complex Programme Version within the Inventory

Different title/credits eng. Complex Programme Inventory fre Complex Programme Version within the Inventory

Metadata Example Audio track Metadata track Annotation Language Spoken Language Full Spoken language Description en-US en-GB-x-Dolby. E-a-Post. Watershed Dolby E surround track with profanity Data track Metadata track Annotation Language Spoken Language Full Annotation Description RFC 4646 (ISO 639 ++) en-US gr-Grek Greek subtitling for use in Greece

Process Benefit example Archive Central Storage Rewrite tape to archive new version Add “fre” audio track Playout Server Broadcast Copy back entire file Interleaved files component files Archive Central Storage Append “fre” audio to archive new version Add “fre” audio track Copy back just the “fre” audio track Playout Server Broadcast

Format Specifics Basic structure of the MXF Master format includes: • Single-essence Programme Component files > OP 1 a files that enable Read While Write & random access • Programme Version files that reference Components > Simple Programme Versions for playout servers > Complex Programme Versions for editing and version management • Programme Inventories that collate multiple versions > An Inventory contains all the versions of an asset • Controlled Descriptive metadata > DM Tracks used to identify the language of an audio component • Controlled way to specify the system > Paramaterizing the Specification for a local facility is done in a consistent way for vendors – enhancing interoperability

Process Analysis • If the underlying file format is standardised & pervasive • Standardised Analysis tools can be used to… > model workflows > define interfaces between departments > create service oriented software interfaces > define, track & model interactions between departments > define, track & model active, passive & redundant operations

MXF • Needs to be constrained in order to deliver its value. • Q: How can this value be delivered for you? What needs to be done to MXF in general? • A: Accurate problem specification and a Recommended practise to constrain range of options in the MXF

Formats Comparison • OP 1 a Interleaved • MXF Master Format Lengthy tape rewrites Bandwidth hogging file transfers Incompatible proprietary formats Simple to repurpose files Built-in Metadata Storage Highly Interoperable

Standardisation • SMPTE > Many are keen to see this work in committee > RDD option fast but is simply a static “we did this” document > EG option lacks enforcement > RP option feels right > Full standard wanted by some • We would welcome your review and comments As well as your support

MXF API Overview • In order to promote digital interoperability, and the exchange of program material using files Turner is promoting a standardisation of the use of MXF in the broadcast production and playout industry. • As part of the standardisation is a requirement not only to specify exact MXF operating patterns to be used but also how they will be manipulated. • Efforts are underway to develop a common MXF Processor API • This API is intended as a universal language for the manipulation of MXF media objects.

A Standardized API for MXF manipulations • With a standard file format • Many operations become standardized > Restore > Partial restore (of a subclip) > Partial restore (of a subset of tracks within a subclip) • MXF version files > Can be represented in MXF > Are small in size > Fully describe the source file > Can fully specify a destination file > Are web services friendly

Why? • Vendors currently develop their own proprietary APIs • These restrict interoperability and thus automation (or control) systems must implement many different APIs • Since we’re trying to recommend an MXF standard let’s do the same for the API

Benefit • A single open API that automation (or controller) can interface to • Vendors interface to a common API • Reduces integration costs and time • Provides interoperability at a control layer for MXF aware devices

More than an API • An open API that each vendor interfaces to • Each toolkit vendor will support the API and will provide a subset (or maybe all) of the transformations/functionality • The calling application (automation) may issue a query to see which of the underlying processors can support a particular function • This approach will allow multiple vendors to collectively provide the overall system functionality.

More than an API • Proposed that the API runs under SOAP > In general set up as a Call / Response system with a ticket or handle to the job in progress > allowable exception to this is when the job terminates and the processor wants to flag the calling system so it doesn’t have to persist the result set • Management of name space • Referential integrity

Basic Functionality • • • Create a media object Append to that object Move a whole object Copy a whole object Delete a component from within that object Extract one or more components from an object • Delete a whole object • Interrogate an object • Manage the activities

Use cases • Complex Program Version to Simple Program Version • Simple Program Version to Complex Program Version • Create/remove empty Program Inventory • Add/delete Complex Program Version to Inventory • Add/delete Program Component to/from Program Inventory • Add/delete Program Component to/from Complex Program Version • Complex Program Version to Complex Program Version

Referential Integrity • This is a set of rules not a set of functions within the API • Only packages actually handled by the processor, or appearing in a configured directory will be managed by the referential integrity checking. • Ensures the header and associated essence files are valid

Name Space? • Should the processor embody & enforce a naming convention for component and version files? If so is this naming convention part of the MXF application specification? • Should this name space be determined solely by the end user, or should a recommended practice be developed to try to encourage standardisation?

Summary • MXF Processor API will provide a common set of functions that all MXF compatible devices will support • Advantage that a given device will require minimal testing/development for integration into environment supporting API • Allow customers/users to quickly determine if a device should be compatible within their environment • Reduce overall system integration time (and cost) for new device • Vendors will have immediate solutions their device can integrate within should they adopt open API

We Are Here Repurpose Many Master Once

NAB 2007 • Tuesday 1 p-5 p MXF Papers in Broadcast Engineering Conference Reception (Immediately Following - Free Drinks!) • Demonstrations Area • files ready now or being developed > Snell & Wilcox > Omneon > Marquis > Softel • API & metadata interop being defined > Metaglue > TMD > Opencube > Pro-bel

Developers and Supporters of the MXF Mastering Format

References • The MXF Book • SMPTE 377 M et al • RPzzz MXF Master format • The File Interchange Handbook

Questions?