Multimedia Applications Ali Saman Tosun Computer Science Department

Multimedia Applications Ali Saman Tosun Computer Science Department

Compression - Necessity Ø E. g. , video sequence Ø 25 images/sec. Ø Ø 3 byte/pixel Ø Ø Ø PAL standard YUV (luminance + 2 chrominance values) RGB (red-green-blue values) Image resolution 640 * 480 pixel Ø Data rate = 640 * 480 * 3 Byte * 25/s = 23040000 byte/s ~ 22 MByte/s Ø Ø Ø Approx. 1/16 stream over Ethernet Approx. 1/2 stream over Fast Ethernet Compression is necessary 2

Multimedia Networking Applications Classes of MM applications 1) Streaming stored audio and video 2) Streaming live audio and video 3) Real-time interactive audio and video Jitter is the variability of packet delays within the same packet stream Fundamental characteristics: Ø Typically delay sensitive Ø end-to-end delay Ø delay jitter Ø But loss tolerant: infrequent losses cause minor glitches Ø Antithesis of data, which are loss intolerant but delay tolerant. 3

From Broadcast to True Media-on-Demand Ø Broadcast (No-Vo. D) Ø Traditional, no control Ø Pay-per-view (PPV) Ø Ø Quasi Video On Demand (Q-Vo. D) Ø Ø Ø Distinction into interest groups Temporal control by group change Near Video On Demand (N-Vo. D) Ø Ø Ø Paid specialized service Same media distributed in regular time intervals Simulated forward / backward True Video On Demand (T-Vo. D) Ø Ø Full control for the presentation, VCR capabilities Bi-directional connection 4

Streaming Stored Video Streaming Ø media stored at source Ø transmitted to client Ø streaming: client playout begins before all data has arrived Ø timing constraint for still-to-be transmitted data: in time for playout 5

Streaming Multimedia variable network delay client video reception constant bit rate video playout at client buffered video Cumulative data constant bit rate video transmission client playout delay Ø time Client-side buffering, playout delay compensate for network-added delay, delay jitter 6

Proxy-based Video Distribution Server Proxy adapts video Proxy caches video Proxy Client 7

Proxy Operations Ø Drop frames Ø Ø Drop B, P frames if not enough bandwidth Quality Adaptation Ø Transcoding Ø Ø Ø Change quantization value Most of current systems don’t support Video staging, caching, patching Ø Ø Ø Staging: store partial frames in proxy Prefix caching: store first few minutes of movie Patching: multiple users use same video 8

Basic Encoding Steps 9

H. 261 (px 64) Ø International Standard Ø Video codec for video conferences at p x 64 kbit/s (ISDN): Ø Ø Ø Intraframe coding Ø Ø Real-time encoding/decoding, max. signal delay of 150 ms Constant data rate DCT as in JPEG baseline mode Interframe coding, motion estimation Ø Search of similar macroblock in previous image and compare Ø Ø Position of this macroblock defines motion vector Difference between similar macroblocks 10

MPEG (Moving Pictures Expert Group) Ø International Standard: Ø Ø Compression of audio and video for playback (1. 5 Mbit/s): Real-time decoding Ø Sequence of I-, P-, and B-Frames: Ø Random access Ø Ø Ø at I-frames at P-frames: i. e. decode previous I-frame first at B-frame: i. e. decode I and P-frames first 11

MPEG-2 Ø Beyond MPEG-1: Higher quality encoding Ø Higher data rates Ø Interleaved modes Higher data rates Ø Ø Ø MPEG-1: about 1. 5 MBit/s MPEG-2: 2 -100 MBit/s Ø Signal to Noise Ration (SNR) scalin Ø Ø Scaling: Ø Ø progressive compression several pixel resolutions Ø frame dropping Spatial scaling Temporal scaling 12

Scalable coding Ø Typically used as Layered coding A base layer Ø Ø Ø Provides basic quality Must always be transferred e Enhancement layer One or more enhancement layers Ø Ø tp a lity ble i oss at gr din sen ua Quality Ø Improve quality Transferred if possible q ible oss p est B Base layer Sending rate 13

Temporal Scalability Frames can be dropped Ø Ø In a controlled manner Frame dropping does not violate dependancies Low gain example: B-frame dropping in MPEG-1 14

Spatial Scalability Ø Base layer Ø Ø Ø Enhancement layer Ø Ø Ø Downsample the original image Send like a lower resolution version Subtract base layer pixels from all pixels Send like a normal resolution version If enhancement layer arrives at client Ø Ø 72 61 75 83 Decode both layers Add layers 73 Base layer Less data to code -1 -12 2 10 Enhancement layer Better compression due to low values 15

SNR Scalability Ø Ø SNR – signal-to-noise ratio Idea Ø Base layer Ø Ø Ø Enhancement layer is regularly DCT encoded Ø Ø Is regularly DCT encoded A lot of data is removed using quantization Run Inverse DCT on quantized base layer Subtract from original DCT encode the result If enhancement layer arrives at client Ø Add base and enhancement layer before running Inverse DCT 16

Multiple Description Coding Ø Idea Ø Ø Ø Encode data in two streams Each stream has acceptable quality Both streams combined have good quality The redundancy between both streams is low Problem Ø Ø Ø The same relevant information must exist in both streams Old problem: started for audio coding in telephony Currently a hot topic 17

Delivery Systems Developments Several Programs or Timelines Network Saving network resources: Stream scheduling 18

Patching Central server Join ! Unicast patch stream multicast cyclic buffer 1 st client 2 nd client 19

Proxy Prefix Caching Ø Split movie Ø Ø Ø Unicast Prefix Suffix Operation Ø Store prefix in prefix cache Ø Ø Coordination necessary! On demand Ø Ø Ø Central server Deliver prefix immediately Prefetch suffix from central server Goal Ø Ø Ø Reduce startup latency Hide bandwidth limitations, delay and/or jitter in backbone Reduce load in backbone Prefix cache Unicast Client 20

Interval Caching (IC) Ø caches data between requests Ø following requests are thus served from the cache Ø sort intervals on length S 11 12 I 33 I 12 I 31 I 11 S 13 Video clip 1 I 21 S 12 S 11 I 12 I 11 S 22 S 21 Video clip 2 I 21 S 34 Video clip 3 S 33 I 33 S 32 I 32 S 31 I 31 21

Receiver-driven Layered Multicast (RLM) Ø Requires Ø Ø Ø Operation Ø Ø Ø IP multicast layered video codec (preferably exponential thickness) Each video layer is one IP multicast group Receivers join the base layer and extension layers If they experience loss, they drop layers (leave IP multicast groups) To add layers, they perform "join experiments“ Advantages Ø Ø Ø Receiver-only decision Congestion affects only sub-tree quality Multicast trees are pruned, sub-trees have only necessary traffic 22

Receiver-driven Layered Multicast (RLM) 23

Multimedia Disk Scheduling Ø Suitability of classical algorithms Ø Ø minimal disk arm movement (short seek times) no provision of time or deadlines generally not suitable Continuous media server requirements Ø Ø Ø serve both periodic and aperiodic requests never miss deadline due to aperiodic requests must not starve support multiple streams balance buffer space and efficiency tradeoff 24

Group Sweep Scheduling (GSS) GSS combines Round-Robin (RR) and SCAN Ø requests are serviced in rounds (cycles) Ø principle: Ø Ø Ø special cases: Ø Ø Ø divide S active streams into G groups service the G groups in RR order service each stream in a group in C-SCAN order playout can start at the end of the group G = S: RR scheduling G = 1: SCAN scheduling tradeoff between buffer space and disk arm movement Ø Ø Ø try different values for G giving minimum buffer requirement – select minimum a large G smaller groups, more arm movements, smaller buffers (reuse) a small G larger groups, less arm movements, larger buffers 25

Power Management Ø Based on functionality Ø Ø Server Proxy Client Based on component Ø Ø Disk Processor 26