7 Video Moving Pictures n Vision is

7 Video

Moving Pictures n Vision is probably the most important among our senses n Persistence of vision allows images shown in rapid succession to produce the illusion of motion n Motion obtained above about 15 to 20 images (or frames) per second At lower frame rates, individual frames are noticeable – “flickering” ¨ At higher frame rates, motion becomes smooth ¨ n An image is worth a thousand words Each second of video has about 25 images ¨ Therefore, a second of video is worth about 25, 000 words ¨

Generate Moving Pictures n Video ¨ n Use video camera to capture a sequence of frames Animation ¨ Generate each frame individually either by computer or by other means

Basics of Video n Analog video is represented as a continuous (time varying) signal n Digital video is represented as a sequence of digital images Digital video when duplicated will always retain the same original quality for an infinite amount of time ¨ Crisp, and the highest quality video ¨ Can be kept on DVD discs or CD-ROMs ¨

Types of Color Video Signals n Component video ¨ ¨ ¨ n Each component is sent as a separate video signal Three components: Y (luminance), U and V (color) Often use in production and post-production Best color reproduction Requires more bandwidth and good synchronization of the three components Composite video Combine three components into a signal ¨ Color component (U and V) is allocated half bandwidth as the luminance (Y) ¨ Some interference between the two signals is inevitable ¨ Often use in transmission ¨

… Types of Color Video Signals n S-video (Separated video) Separates the luminance from the two color (total two signals) ¨ A compromise between component video and the composite video ¨

Television Broadcast Standards n NTSC ¨ ¨ ¨ ¨ 1952 by National Television Standards Committee Used in: USA, Japan, Korea, and others 525 lines/frame (vertical resolution) Two passes to draw a single frame at a rate of 60 Hz per second (interlacing) 30 frames/second Resolution approx. 640 x 480 (4: 3 ratio) uses YIQ color model "Never The Same Color"

… Television Broadcast Standards n PAL ¨ ¨ ¨ n Phase-Alternative Line system Western Europe except France 625 lines/frame Interlaced at 50 Hz 25 frames/second uses YUV color model SECAM ¨ ¨ ¨ The Sequential Color and Memory system France and Eastern Europe 625 lines/frame 50 Hz 25 frames/second

… Television Broadcast Standards n HDTV ¨ ¨ ¨ High-Definition Television The Image of the new millennium screen picture quality similar to 35 mm film, along with compact disc (CD) sound quality Some American television stations began transmitting digital HDTV in 1998 Format Wars (18 formats) active lines active aspect ratio frame rate horizontal pixels 720 1280 16/9 progressive 24, 30 or 60 1080 1920 16/9 interlaced 60 1080 1920 16/9 progressive 24, 30

… Television Broadcast Standards n Features ¨ ¨ Higher-resolution picture Wider picture Digital surround sound (Dolby Digital (AC-3) ) Additional data n DTV stands for digital television n All HDTV broadcasts in the United States would be digital

How Video Works n When light reflected from an object passes through a video lens, that light is converted into a electronic signal by a special sensor called a charged-coupled device (CCD) n The signal from the camera contains three channels of color (RGB) and synchronization pulses (sync) n The video signal is recorded on magnetic tapes n One or two channels of sound may also be recorded on the tape n Video Cassette Recorder (VCR) Each strip represents information for one field of a frame Audio track Video track Control track

Video Color n One luminance component n Two chrominance components Original TV was black and white ¨ Adding color had to be done in a compatible way ¨ n NTSC: YIQ n PAL: YUV ¨ In general: YUV and YCr. Cb used as terms

Recording Formats n YUV: Luminance, Hue, Saturation ¨ ¨ ¨ n B. 11. 87. 11 G. 30. 59 4: 1: 1 YUV format ¨ n Luminance (Y) Hue (U) Saturation (V) R. 30. 70 7 -bit Y + U & V data are averaged for 4 pixels S-VHS Video (s-video) 1 Ground 2 Ground 3 Intensity (Luminance) 4 Color (Chrominance) video signal with enhanced quality used for recording Color and luminance information are kept on two separate tracks ¨ 4 -pin DIN connectors ¨ Oriented toward consumer with gaining acceptance among lower-end broadcaster ¨ ¨

… Recording Formats n Video recording format ¨ ¨ n Japan Victor Company VHS-compact used in camcorders Component (YUV) ¨ ¨ ¨ n VHS: VHS-C: Laying the signal on the tape in three channels Four channels of audio Sony Betacam SP Component Digital ¨ ¨ The signal is converted to digital information and stored as bytes Can make unlimited copies without loss of quality Digital Betacam High price tag

Video Hardware Resolution n The lens used and the number, size, and quality of the CCDs determine the resolution n Resolution (in horizontal lines) n Video Type Resolution 8 mm VHS S-VHS Hi-8 Beta-SP MII (Panasonic) Broadcast-quality ¨ 230 240 400 550 1000 Hi-8 is the most widely accepted format used for industrial and corporate video communication

Digital Video n A video sequence consists of a number of frames n Each frame is a single image produced by digitizing time-varying signal generated by video camera n CCIR 601 is a standard established for digitizing NTSC and PAL signals specifies the image format, and coding for digital television signals ¨ It uses Y'CBCR color with 4: 2: 2 chrominance subsampling. The data rate is 166 Mbits per sec ¨

… Digital Video n Think about the size of the uncompressed digital video ¨ NTSC video format Ø Bitmapped images for video frame ¨ Ø 30 frames per second ¨ Ø 900 kb/frame 30 frames/sec = 26 MB/sec 60 seconds per minute ¨ n 640 480 pixels with 24 -bit color = 0. 9 MB/frame 26 MB/sec 60 secs/minute = 1, 600 MB/minute Strains on current processing, storage and data transmission !

Create Digital Video n Get analog/digital video signal from video camera ¨ video tape recorder (VTR) ¨ broadcast signal ¨ n Digitize analog video & compress it

Digitizing Analog Video n In computer ¨ Video capture card Ø Ø ¨ Convert analog to digital & compress Can also decompress & convert digital to analog Compress through Ø Ø Video capture card (hardware codec) Software (software codec)

Digitizing Analog Video n In camera Digitize and compress using circuitry inside camera ¨ Transfer digitized signal from camera to computer through ¨ Ø Ø IEEE 1394 interface (Fire. Wire): 400 Mb/sec USB: 12 Mb/sec(version 1. 1) ~ 480 Mb/sec(version 2. 0)

Digitize in Computer v. s. Camera n Digitize in camera ¨ Advantage Ø ¨ Digital signals are resistant to corruption when transmitted down cables and stored on tape Disadvantage Ø User has no control between picture quality and data rate (file size)

Digital Video Camera n DV camcorders store up to 90 minutes of digital video on small DV cartridges ¨ ¨ n VHS tape format Encoded digitally Digital 8 camcorders store digital video on Hi-8 tapes and can also play Hi-8 and 8 mm tapes ¨ Digital 8 cameras are usually larger than DV cameras n DVD Camcorder n Video Resolution ¨ ¨ The vertical resolution of a digital video image is determined by the number of scan lines in the image The horizontal resolution is the rate at which the moving beam can turn on and off to paint "dots" of color on the screen

Better image quality than previous analog formats DV Lines of Resolution Color Signal-to-Noise S-VHS/Hi 8 8 mm/VHS 500 400 240 Component S-Video Composite 60 45 -46 43 -45

Video Tape Formats Tape Recording Format Analogue Digital DV (Mini. DV) DV CAM DVPRO 8 MM Video 8 Hi 8 Digital 8 VHS S-VHS D 9 MII D 3 Betamax Betacam SP Beta SX Digibeta 24 P Other U'matic D 1 C - format (open reel)

Videotape & Digital Recorders/Players

Digital Cameras

Video Capture Cards n Pinnacle Systems ¨ ¨ n View. Cast ¨ n DV series TARGA series Osprey series Capture ¨ ¨ ¨ to disk to memory full-screen drop frames non-real-time step-frame

Film & Video Editing n Traditional n Timecode SMPTE timecode ¨ Hours, minutes, seconds, frames ¨ n VHS Two copying operations is to produce serious loss of quality ¨ Constructed linearly ¨

Video Production n Pre-production script writing ¨ storyboarding ¨ production schedule for shooting the scene ¨ n Production involves shooting the scene n Post Production ¨ editing the best scenes into the final video program Ø Ø rough cut final cut

Video Time n Time base how time is divided in your project ¨ time base of 30 means each second is divided into 30 units ¨ n frame rate number of frames per second ¨ frame rate of 30 means the camera records the scene every 1/30 th of a second ¨ there are 30 frames per second ¨

Video Time n Time code how the frames are counted ¨ SMPTE time code (society of motion pictures and tv engineers) ¨ hh: mm: ss: frames ¨ 00: 05: 31: 15 - 5 minutes, 31 seconds, 15 frames ¨

Steps n Import clips ¨ ¨ ¨ still image audio video (capture) n Editing n Compositing n Reverse shot ¨ Conversation between two people n Transitions n Titles/motion n Export

Digital Video Editing n Analog editing systems n Avid video editing software ¨ n Avid Xpress DV Adobe ¨ ¨ Premiere After Effects n Microsoft Windows Movie Maker n Pinnacle Systems ¨ n Reel. Time Digital video editing software ¨ ¨ ¨ ¨ conversion and compression capture editing transition effects composition audio editing motion path control text editing

… Digital Video Editing n Random access n Non-destructive n Premiere The standard mid-range video editing application ¨ Three main windows ¨ Ø ¨ Timelines Ø ¨ ¨ Project, timeline, monitor Have several video tracks Transitions Cuts and Transitions Ø Ø In a cut, two clips are butted In transitions, two clips overlap ¨ Image processing is required to construct transitional frames

Digital Video Post-production n Over- or under-exposed, out of focus, color cast, digital artifacts ¨ Provide image manipulation programs Ø Adjust level, sharpen, blur The same correction may be needed for every frame, so the levels can be set for the first frame and the adjustment will be applied to as many frames as user specifies. ¨ If light fades during a sequence, it will necessary to increase the brightness gradually to compensate. ¨ Ø Ø Apply a suitable correction to each frame and allow their values at intermediate frames to be interpolated Varying parameter values over time

Keying n Selecting transparent areas n Blue screening ¨ ¨ ¨ n Select explicitly ¨ ¨ n Chroma keying: any color Alpha channel Luma keying: a brightness threshold is used to determine which areas are transparent Create mask In film and video, mask is called matte Matte out: removing unwanted elements Split-screen effects Alpha channel created in other application

Track matte n Chroma keying and luma keying Color and brightness changes between frames ¨ Use a sequence of masks as matte ¨ Ø ¨ Separate video track: track matte Track matte Ø Ø Painstaking by hand Generated from a single still image applying simple geometrical transformations over time to create a varying sequence of mattes

Adobe After Effects n Apply a filter to a clip and vary it over time n A wide range of controls for the filter’s parameters n Premiere: parameter values are interpolated linearly between key frames n After effect: interpolation can use Bezier curves

Preparing Video for Multimedia Delivery n Frame size, frame rate, color depth, image quality n People sit close to monitors, so a large picture is not necessary n Higher frame rates are needed to eliminate flicker only if display is refreshed at the same rate. ¨ n Computer monitors are refreshed at a much higher rate from VRAM. Limiting colors ¨ Not all codecs support

The Bits and Bytes of Digital Video n Data rates of digital video depends on Frame rate: Frames per second (TV/HDTV/DVD uses 24 -30 fps) ¨ Frame dimensions: The width and height of the image expressed in number of pixels ¨ Ø Ø Ø ¨ TV: 640 x 480 pixels DVD: 720 x 576 pixels HDTV: 1, 920 x 1, 080 pixels Pixel depth: The number of bits per pixel Ø Ø TV/DVD: 16 bits HDTV: 24 bits

… The Bits and Bytes of Digital Video n If we have DVD quality video, the uncompressed data stream would be ¨ n HDTV quality requires significantly more ¨ n 720 x 576 pixels * 16 bits/pixel * 30 fps = 199 Mbits/second 1, 920 x 1, 080 pixels * 24 bits/pixel * 30 fps = 1, 492 Mbits/second Either way, this is huge amount of data

7. 2 Video Formats and Compression

Video Formats n VCD (Video Compact Disc) A compact disc with full motion video on it ¨ Once converted to MPEG (Motion Picture Experts Group), it can be put onto a CD ¨ Has capability to hold 650 -700 MB of quality video (approximately 74 minutes) ¨ Can be played on most DVD players, Computers with a CDROM drive, Sony Playstations, Sega Dreamcasts, etc. ¨ n DVD (Digital Video Disc) A digital video disc that can store up to 2 hours of high quality video ¨ 4. 7 gigabytes of information ¨ Played on DVD players and computers with DVD-ROM ¨

Video Formats n Streaming video: ¨ ¨ ¨ n AVI MOV RM WMV MPEG Other popular multimedia formats (not pure video formats): Shockwave ¨ Flash ¨

Video Throughput n The filesize (download time) that a video requires is dependent on: Window size (e. g. 640 x 480) ¨ Frames per second (e. g. 30 fps) ¨ Compression rate ¨ Ø Inversely impacts image quality ¨ Quality also depends on compression algorithm

Video Throughput n If you thought we needed compression with audio… n At 24 bbp, 640 x 480, 30 fps (which would produce a fairly standard looking TV video) it would take about 100 GB to store an hour of uncompressed video

Video Throughput n Videos can be fully downloaded to your computer and then played ¨ n On formats that support streaming you can start watching the video before it is done loading Or videos can be played off a “streaming server” and sent to your computer via a special protocol ¨ You don’t get a copy of the file on your system

Video Throughput n Because the streaming method is quite popular on the internet, video throughput is also often stated in terms of the speed of your connection ¨ n Your computer can received (and play) X bits per second of video information So the actual size of the video file is not as important as the bandwidth required to play the video ¨ MPEG 2 videos require a connection that can handle 100 Mbits/second in order to show the video in realtime (without large buffering pauses in the middle of the movie)

AVI n Microsoft’s Audio Video Interleave ¨ n The “Interleave” in the name is associated with the fact that packets of Video / image data are interwoven with audio data One of the first video formats on the PC Released with Windows 3. 1 (early 90 s) ¨ Considered an “old” format ¨ n Compression is poor (long download times) n Image quality is poor n Cannot be streamed ¨ Makes is a bad choice for the web

MOV n Quick. Time Movie ¨ n Developed by Apple Was one of the first multitrack formats ¨ ¨ Can have different sound tracks aimed at different languages and then only using one of them Also released in the early 90 s n Compression is average (average bandwidth needed) n Image quality is good n Can be streamed n Also useful as an editing format -- not just for delivery

RM n Real Media ¨ Developed by Real Networks n Constructed specifically for the purpose of streaming n Compression is good (low bandwidth needed) n Image quality is low ¨ Was designed for low bitrates so they decided to sacrifice image quality Ø If using a higher bandwidth, one would probably want to use another format that has better image quality (like MOV)

WMV n Windows Media Video ¨ Developed by Microsoft n Designed to compete directly with RM n Compression is excellent (very low bandwidth needed) n Image quality is somewhat grainy in comparison to Quick. Time ¨ n But the bitrates are also much lower Supports Streaming

MPEG n The MPEG compression algorithm is quite similar to JPEG n MPEGs support streaming

3 Ways to Attach to Web Pages n tag Similar to a normal link, but the href is the video file ¨ Pops up an external player to play the video ¨ n tag Embeds to player in the web page ¨ This has been the most popular of the embedding methods ¨ n