Multimedia Networking r Application classes m streamed stored

Multimedia Networking r Application classes m streamed stored audio/video m one-to-many (multicast) streaming of real-time a/v m real-time interactive audio/video r Typical application issues m packet jitter m packet loss / recovery r Internet protocols for multimedia m RTSP m RTP/RTCP m H. 323 r Text: Kurose-Ross, Chapter 6

Example Multimedia Apps r Streamed stored audio/video m movies, CS-653 taped lectures (available on MANIC) r One-to-many streaming m News broadcasts, popular movies r Real-Time Interactive m IP telephony, teleconference, distributed gaming

Multimedia terminology r Multimedia session: a session that contains several media types m e. g. , a movie containing both audio & video r Continuous-media session: a session whose information must be transmitted “continually” m e. g. , audio, video, but not text (unless ticker-tape) r Streaming: application usage of data during its transmission Data stream Playback pt Rcv pt

Multimedia vs. Raw Data r Multimedia m e. g. , Audio/Video m Tolerates some packet loss m Packets have timed playout reqmts r Raw Data m e. g. , FTP, web page, telnet m Lost packets must be recovered m Timing: faster delivery always preferred Why not just use TCP for multimedia traffic? • •

Jitter r The Internet makes no guarantees about time of delivery of a packet r Consider an IP telephony session: Speaker Hi There, What’s up? Listener Hi The Time t’s re, Whaup?

Jitter (cont’d) r A packet pair’s jitter is the difference between the transmission time gap and the receive time gap Sender: Pkt i+1 Pkt i Receiver: Si Ri Pkt i+1 Si+1 r Desired time-gap: Si+1 - Si jitter Ri+1 Received time-gap: Ri+1 - Ri r Jitter between packets i and i+1: (Ri+1 - Ri) - (Si+1 - Si) Time

Buffering: A Remedy to Jitter r Delay playout of received packet i until time Si + C (C is some constant) r How to choose value for C? m m m Bigger jitter need bigger C Small C: more likely that Ri > Si + C Big C: missed deadline • requires more packets to be buffered • increased delay prior to playout m Application timing reqmts might limit C: • Interactive apps (IP telephony) can’t impose large playout delays (e. g. , the international call effect) • non-interactive: more tolerant of delays, but still not infinite. . .

Adaptive Playout r For some applications, the playout delay need not be fixed r e. g. , [Ramjee 1994] / p. 430 in Kurose-Ross m m m Speech has talk-spurts w/ large periods of silence Can make small variations in length of silence periods w/o user noticing Can re-adjust playout delay in between spurts to current network conditions

Packet Loss / Recovery r Problem: Internet might lose / excessively delay packets making them unusable for the session arrival time: app deadline: Pkt i+1 Pkt i i i+1 Pkt i+3 i+2 i+3 usage status: …, i used, i+1 late, i+2 lost, i+3 used, . . . r Solution step 1: Design app to tolerate some loss r Solution step 2: Design techniques to recover some lost packets within application’s time limits

Applications that tolerate some loss r Techniques are medium-specific and influence the coding strategy used (beyond scope of course) m m Video: e. g. , MPEG Audio: e. g. , GSM, G. 729, G. 723, replacing missing pkts w/ white-noise, etc. r Note: loss tolerance is a secondary issue in multimedia coding design r Primary issue: compression

Reducing loss w/in time bounds r Problem: packets must be recovered prior to application deadline r Solution 1: extend deadline, buffer @ rcvr, use ARQ m Recall: unacceptable for many apps (e. g. , interactive) r Solution 2: Forward Error Correction (FEC) m Send “repair” before a loss is reported m Simplest FEC: transmit redundant copies Sender: Receiver: Pkt i+1 Pkt i+1 Pkt i+2 Pkt i+1 duplicate Pkt i+2 lost

More advanced FEC techniques r FEC often used at the bit-level to repair corrupt/missing bits (i. e. , in the data-link layer) FEC bits data header r Here, we will consider using FEC at the packet layer (special repair packets): Data 1 Data 2 Data 3 FEC 1 FEC 2

A Simple XOR code r For low packet loss rates (e. g. 5%), sending duplicates is expensive (wastes bandwidth) r XOR code m m 10101 XOR a group of data pkts together to produce repair pkt Transmit data + XOR: can recover 1 lost pkt 11100 00111 11000 10110 00111

Reed-Solomon Codes r Based on simple linear algebra m can solve for n unknowns with n equations m each data pkt represents a value m Sender and receiver know which “equation” is in which pkt (i. e. , information in header) m Rcvr can reconstruct n data pkts from any set of n data + repair pkts m In other words, send n data pkts + k repair packets, then if no more than any k pkts are lost, then all data can be recovered r In practice m To reduce computation, linear algebra is performed over fields that differ from the usual

Reed Solomon Example over Pkt 1: Data 1 Pkt 2: Data 2 Pkt 3: Pkt 4: Data 3 Data 1 + Data 2 + 2 Data 3 Pkt 5: 2 Data 1 + Data 2 + 3 Data 3 r Pkts 1, 2, 3 are data (Data 1, Data 2, and Data 3) r Pkts 4, 5 are linear combos of data r Assume 1 -5 transmitted, pkts 1 & 3 are lost: m Data 1 = (2 * Pkt 5 - 3 * Pkt 4 + Pkt 2) m Data 2 = Pkt 2 m Data 3 = (2 * Pkt 4 - Pkt 5 - Pkt 2)

Using FEC for continuous-media Data 1 D 3 FEC 1 F 2 D 1 block i blk i blk i+1 D 3 F 1 F 2 D 1 blk i Sender: D 2 blk i+1 Rcvr: Decoder Rcvr App: r Divide data pkts into blocks . . . D 1 D 2 D 3 blk i . . . Block i needed by app r Send FEC repair pkts after corresponding data block r Rcvr decodes and supplies data to app before block i deadline

FEC via variable encodings r Packet contents: m high quality version of frame k m low quality version of frame k-c (c is a constant) m If packet i containing high quality frame k is lost, then can use packet i+c with low quality frame k in place i low: k-c high: k i+1 low: k-c+1 high: k+1 i+2 low: k-c+2 high: k+2

FEC tradeoff r FEC reduces channel efficiency: m Available Bandwidth: B m Fraction of pkts that are FEC: f m Max data-rate (barring pkt loss): B (1 -f) r Need to be careful how much FEC is used!!!

Bursty Loss: r Many codecs can recover from short (1 or 2 packet) loss outages r Bursty loss (loss of many pkts in a row) creates long outages: quality deterioration more noticeab r FEC provides less benefit in a bursty loss scenario (e. g. , consider 30% loss in bursts of 3) D 1: i D 2: i D 3: i F 1: i Too much FEC F 2: i D 1: i+1 D 2: i+1 D 3: i+1 F 1: i+1 Too little FEC F 2: i+1

Interleaving r To reduce effects of burstiness, reorder pkt transmissions Sender schedule Arrival schedule Playback schedule D 1 D 4 D 7 D 2 D 5 D 8 D 4 D 8 D 1 D 3 D 6 D 3 D 4 D 6 r Drawback: induces buffering and playout delay D 8

Multimedia Internet Protocols r We’ll look at 3: m RTP/RTCP: transport layer m RTSP: session layer for streaming media applications m H. 323: session layer for conferencing applications RTSP H. 323 TCP UDP RTP/RTCP UDP/multicast

RTP/RTCP [RFC 1889] r Session data sent via RTP (Real-time Transfer Protocol) r RTP components / support: m sequence # and timestamps m unique source/session ID (SSRC or CSRC) m encryption m payload type info (codec) r Rcvr/Sender session status transmitted via RTCP (Real-time Transfer Control Protocol) m m m last sequence # rcvd from various senders observed loss rates from various senders observed jitter info from various senders member information (canonical name, e-mail, etc. ) control algorithm (limits RTCP transmission rate)

RTP/RTCP details r All of a session’s RTP/RTCP packets are sent to the same multicast group (by all participants) m m All RTP pkts sent to some even-numbered port, 2 p All RTCP pkts sent to port 2 p+1 r Only data senders send RTP packets r All participants (senders/rcvrs) send RTCP packets

RTP header Payload Sequence Type # Timestamp Synchronization Misc Source Identifier r Why do most (all) multimedia apps require m sequence #? m timestamp? m (unique) Sync Source ID? r Why should every pkt carry the 7 -bit payload type? m Why not just when sender initiates session? r Transmission rate: application specific (no congestion control specified in RTP)

RTCP packets r There are several types of RTCP packets m SR: sender report - transmission & reception stats m RR: receiver report - reception stats m SDES: Source description items m BYE: end-of-participation message m APP: application-specific functions r Typically, several RTCP packets of different types are transmitted w/in a single UDP packet

What RTCP provides r Info to detect colliding Synch source ID’s r Contact info (e-mail, true name) of participants r Info to count # of session participants r Reception quality of all participants r How does RTCP avoid creating congestion if all participants send RTCP packets? m consider a broadcast TV transmission

RTCP congestion control r A session’s aggregate RTCP bandwidth usage should be 5% of the session’s RTP bandwidth m m 75% of the RTCP bandwidth goes to the receivers 25% goes to the senders r Tsender = # senders * avg RTCP pkt size . 25 *. 05 * RTP bandwidth r Trcvr = # receivers * avg RTCP pkt size. 25 *. 05 * RTP bandwidth Send at (. 5 + rand(0, 1)) * T : why? How does each member know # of senders, # rcvrs?

RTCP reconsideration r Goal: prevent RTCP bandwidth explosion if everybody joins at once m Receivers who initially join will count small # of session members r Solution when first joining: 1. Compute T, and wait random time interval 2. At end of interval, reassess # of members 3. If # of members increased, compute a new T’ 4. If T’ < T, send immediately 5. If T’ >= T, wait an additional T’, go to step 2 r Other times, use normal wait period

RTSP [RFC 2326] r RTSP: out-of-band protocol used to control transmission of a media-stream m VCR-like functionality (pause/resume, FF, rewind, reposition, etc. ) Web Browser http get setup presentation description play Player pause teardown HTTP Server protocol Media RTSP Server protocol ACK Media Web media stream ACK

H. 323 r A standard for real-time audio / video teleconferncing on the Internet r Network Components: m m end points: end-host H. 323 -compliant app gateways: interface between H. 323 -compliant communication and prior technology (e. g. phone network) gatekeepers: provide services at gateway (e. g. , address translation, billing, authorization, etc. ) Audio Apps Video Apps G. 711 H. 261 G. 722 H. 263 G. 729 etc. RTP / RTCP UDP Gateway RAS Channel H. 225 System Control Call Signaling Channel Q. 931 Call Control Channel H. 245 TCP H. 323 m

H. 323 cont’d r H. 225: notifies gatekeepers of session initiation r Q. 931: signalling protocol for establishing and G. 711 H. 261 G. 722 H. 263 G. 729 etc. RTP / RTCP RAS Channel H. 225 Call Signaling Channel Q. 931 Call Control Channel H. 245 H. 323 terminating calls r H. 245: out-of-band protocol negotiates the audio/video codecs used during a session (TCP)