Скачать презентацию Digital Video Compression Fundamentals and Standards Speaker Wei-Yi Скачать презентацию Digital Video Compression Fundamentals and Standards Speaker Wei-Yi

129edd8433b1f679da57f408a1c4b725.ppt

  • Количество слайдов: 53

Digital Video Compression Fundamentals and Standards Speaker: Wei-Yi Wei Advisor: Prof. Jian-Jung Ding 2008/12/26 Digital Video Compression Fundamentals and Standards Speaker: Wei-Yi Wei Advisor: Prof. Jian-Jung Ding 2008/12/26 Digital Image and Signal Processing Lab GICE, National Taiwan University Digital Video Compression Fundamentals and Standards

Outline Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions 2008/12/26 Digital Outline Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions 2008/12/26 Digital Video Compression Fundamentals and Standards 2

Outline Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions 2008/12/26 Digital Outline Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions 2008/12/26 Digital Video Compression Fundamentals and Standards 3

Introduction (1/2) Why video compression technique is important ? One movie video without compression Introduction (1/2) Why video compression technique is important ? One movie video without compression 720 x 480 pixels per frame 30 frames per second Total 90 minutes Full color The total quantity of data = 167. 96 G Bytes !! 2008/12/26 Digital Video Compression Fundamentals and Standards 4

Introduction (2/2) What is the difference between video compression and image compression? Temporal Redundancy Introduction (2/2) What is the difference between video compression and image compression? Temporal Redundancy Coding method to remove redundancy Intraframe Coding Remove spatial redundancy Interframe Coding Remove temporal redundancy 2008/12/26 Digital Video Compression Fundamentals and Standards 5

The most intuitive method to remove Spatiotemporal redundancy 3 -Dimensional DCT Remove spatiotemporal correlation The most intuitive method to remove Spatiotemporal redundancy 3 -Dimensional DCT Remove spatiotemporal correlation Good for low motion video Bad for high motion video 2008/12/26 Digital Video Compression Fundamentals and Standards 6

The most popular method to remove temporal redundancy The Block-Matching Algorithm 2008/12/26 Digital Video The most popular method to remove temporal redundancy The Block-Matching Algorithm 2008/12/26 Digital Video Compression Fundamentals and Standards 7

Matching Function The dissimilarity between two blocks and The matching criteria Mean square error Matching Function The dissimilarity between two blocks and The matching criteria Mean square error (MSE) High precision is needed Mean absolute difference (MAD) Low precision is enough 2008/12/26 Digital Video Compression Fundamentals and Standards 8

The Exhaustive Block-Matching Algorithm Reference Frame Current Frame Motion Vector |A|=12 12 22 33 The Exhaustive Block-Matching Algorithm Reference Frame Current Frame Motion Vector |A|=12 12 22 33 44 55 66 77 88 11 22 33 44 55 66 77 88 1 0 11 23 34 44 55 66 77 88 11 22 33 44 55 66 77 88 0 1 Search 11 22 33 44 55 66 77 88 11 23 34 44 55 66 77 88 0 1 Range 11 22 33 44 55 66 77 88 11 23 34 44 55 66 77 88 0 1 Digital Video Compression Fundamentals and Standards 2008/12/26 0 0 1 1 0 0 0 0 0 0 0 0 0 0 9

Fast Block-Matching Algorithms EBMA needs Intensive computation Fast Algorithm is needed Find the possible Fast Block-Matching Algorithms EBMA needs Intensive computation Fast Algorithm is needed Find the possible local optimal Reference Frame 2008/12/26 Digital Video Compression Fundamentals and Standards Current Frame 10

Fast Block-Matching Algorithms The characteristics of fast algorithm Not accurate as EBMA Save large Fast Block-Matching Algorithms The characteristics of fast algorithm Not accurate as EBMA Save large computation Two famous fast algorithm 2 -D logarithm Search Method Three Steps Search Method 2008/12/26 Digital Video Compression Fundamentals and Standards 11

2 -D logarithm Search Method 1 1 2 4 2 4 3 4 1 2 -D logarithm Search Method 1 1 2 4 2 4 3 4 1 4 4 3 2008/12/26 Digital Video Compression Fundamentals and Standards 12

Three Step Search Method 2 1 1 2 2 3 3 3 1 3 Three Step Search Method 2 1 1 2 2 3 3 3 1 3 2 3 3 2 2 1 1 1 2008/12/26 1 1 1 Digital Video Compression Fundamentals and Standards 13

Multiresolution Motion Estimation (1/3) The number of levels is L l-th level images of Multiresolution Motion Estimation (1/3) The number of levels is L l-th level images of the target frames where is set of pixels at level L At the l-th level, the MV is At the l-th level, the estimated MV is Determine update such that error is minimized The new motion vector is 2008/12/26 Digital Video Compression Fundamentals and Standards 14

Multiresolution Motion Estimation (2/3) Variable block size method 2008/12/26 Digital Video Compression Fundamentals and Multiresolution Motion Estimation (2/3) Variable block size method 2008/12/26 Digital Video Compression Fundamentals and Standards 15

Multiresolution Motion Estimation (3/3) 2008/12/26 Digital Video Compression Fundamentals and Standards 16 Multiresolution Motion Estimation (3/3) 2008/12/26 Digital Video Compression Fundamentals and Standards 16

Outline Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions 2008/12/26 Digital Outline Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions 2008/12/26 Digital Video Compression Fundamentals and Standards 17

The Development of Video Compression Standards 2008/12/26 Digital Video Compression Fundamentals and Standards 18 The Development of Video Compression Standards 2008/12/26 Digital Video Compression Fundamentals and Standards 18

The MPEG-1 Standard Group of Pictures Motion Estimation Motion Compensation Differential Coding DCT Quantization The MPEG-1 Standard Group of Pictures Motion Estimation Motion Compensation Differential Coding DCT Quantization Entropy Coding 2008/12/26 Digital Video Compression Fundamentals and Standards 19

Group of Pictures (1/2) I-frame (Intracoded Frame) Coded in one frame such as DCT. Group of Pictures (1/2) I-frame (Intracoded Frame) Coded in one frame such as DCT. This type of frame do not need previous frame P-frame (Predictive Frame) One directional motion prediction from a previous frame The reference can be either I-frame or P-frame Generally referred to as inter-frame B-frame (Bi-directional predictive frame) Bi-directional motion prediction from a previous or future frame The reference can be either I-frame or P-frame Generally referred to as inter-frame 2008/12/26 Digital Video Compression Fundamentals and Standards 20

Group of Pictures (2/2) The distance between two nearest P-frame or P-frame and I-frame Group of Pictures (2/2) The distance between two nearest P-frame or P-frame and I-frame denoted by M The distance between two nearest I-frames denoted by N Forward Motion Compensation GOP N=9 M=3 I B B P B B I Bidirectional Motion Compensation 2008/12/26 Digital Video Compression Fundamentals and Standards 21

The MPEG-1 Encoder (1/4) Intra-frame Residue DCT Q Entropy Coding Q-1 Motion Compensation Inter-frame The MPEG-1 Encoder (1/4) Intra-frame Residue DCT Q Entropy Coding Q-1 Motion Compensation Inter-frame Motion Vector IDCT Motion Estimation Frame Memory 2008/12/26 Digital Video Compression Fundamentals and Standards 22

The MPEG-1 Encoder (2/4) Differential Coding is the input image is the predictive image The MPEG-1 Encoder (2/4) Differential Coding is the input image is the predictive image DCT 2008/12/26 Digital Video Compression Fundamentals and Standards 23

The MPEG-1 Encoder (3/4) Quantization Intra quantization matrix Inter quantization matrix 2008/12/26 Digital Video The MPEG-1 Encoder (3/4) Quantization Intra quantization matrix Inter quantization matrix 2008/12/26 Digital Video Compression Fundamentals and Standards 24

The MPEG-1 Encoder (4/4) Motion Compensation Exploit motion vector and the previous reconstructed frame The MPEG-1 Encoder (4/4) Motion Compensation Exploit motion vector and the previous reconstructed frame to generate the predictive frame is the compensated image is the previous image is the motion vector Reference frame 2008/12/26 Target frame Digital Video Compression Fundamentals and Standards 25

The MPEG-2 Standard Field/Frame DCT Coding Field/Frame Prediction Mode Selection Alternative Scan Order Various The MPEG-2 Standard Field/Frame DCT Coding Field/Frame Prediction Mode Selection Alternative Scan Order Various Picture Sampling Formats User Defined Quantization Matrix 2008/12/26 Digital Video Compression Fundamentals and Standards 26

Progressive Scan and Interlaced Scan Progressive Scan 2008/12/26 Interlaced Scan Digital Video Compression Fundamentals Progressive Scan and Interlaced Scan Progressive Scan 2008/12/26 Interlaced Scan Digital Video Compression Fundamentals and Standards 27

Field/Frame DCT Coding The field type DCT Fast motion video The frame type DCT Field/Frame DCT Coding The field type DCT Fast motion video The frame type DCT Slow motion video Field DCT Coding 2008/12/26 Luminance MB Frame DCT Coding Digital Video Compression Fundamentals and Standards 28

Alternative Scan Order Zigzag scan order Frame DCT Alternative scan order Field DCT 1 Alternative Scan Order Zigzag scan order Frame DCT Alternative scan order Field DCT 1 0 1 0 2008/12/26 1 0 1 0 Zig-zag 1 1 1 0 0 0 scan 1 1 0 0 2 D DCT 4 0. 72 0 0. 85 0 1. 27 0 3. 62 Alternate 0 0 0 0 0 0 0 0 Digital Video Compression Fundamentals and Standards scan 0 0 0 0 0 0 29

The MPEG-2 Encoder (2/2) Quantization User can change the quantization if necessary Intra quantization The MPEG-2 Encoder (2/2) Quantization User can change the quantization if necessary Intra quantization matrix Inter quantization matrix Various picture sampling formats 4: 4: 4 4: 2: 2 4: 2: 0 2008/12/26 Digital Video Compression Fundamentals and Standards 30

The MPEG-2 Encoder (1/2) SNR Enhanced Layer + Base Layer Bits Enhance Basic quality The MPEG-2 Encoder (1/2) SNR Enhanced Layer + Base Layer Bits Enhance Basic quality requirement For SDTV Q-1 + Enhanced Layer High quality service For HDTV Entropy Coding Q DCT Entropy Coding Q Q-1 + Motion Compensation + Bits Base Layer IDCT Motion Estimation + + Frame Memory 2008/12/26 Digital Video Compression Fundamentals and Standards 31

H. 264/AVC Variable Block Size Multiple Reference Frames Integer Transform Intra Prediction In-loop Deblocking H. 264/AVC Variable Block Size Multiple Reference Frames Integer Transform Intra Prediction In-loop Deblocking Filtering 1/4 -pel Resolution Motion Estimation CAVLC 2008/12/26 Digital Video Compression Fundamentals and Standards 32

Variable Block Size The fixed block size may not be suitable for all motion Variable Block Size The fixed block size may not be suitable for all motion objects Improve the flexibility of comparison Reduce the error of comparison 7 types of blocks for selection 16 x 16 0 8 x 8 0 2008/12/26 16 x 8 0 1 8 x 4 0 1 8 x 16 0 1 4 x 8 0 1 8 x 8 0 1 2 3 4 x 4 0 1 2 3 Digital Video Compression Fundamentals and Standards 33

Multiple Reference Frames The neighboring frames are not the most similar in some cases Multiple Reference Frames The neighboring frames are not the most similar in some cases The B-frame can be reference frame B-frame is close to the target frame in many situations 2008/12/26 Digital Video Compression Fundamentals and Standards 34

Integer Transform for Reducing The Spatial Redundancy (1/2) The transform matrix C 4× 4 Integer Transform for Reducing The Spatial Redundancy (1/2) The transform matrix C 4× 4 Block Size Separable Integer Transform The transform coefficients are CXCT 2008/12/26 Digital Video Compression Fundamentals and Standards 35

Integer Transform for Reducing The Spatial Redundancy (2/2) C 2008/12/26 X CT E Digital Integer Transform for Reducing The Spatial Redundancy (2/2) C 2008/12/26 X CT E Digital Video Compression Fundamentals and Standards 36

Intra Prediction Predict the similarity between the neighboring pixels in one frame in advance, Intra Prediction Predict the similarity between the neighboring pixels in one frame in advance, and exploit differential coding transform coding to remove the redundancy. Intra Prediction Vertical Transform/ Quantization DC + + Horizontal + + Vertical right 2008/12/26 Entropy Coding Diagonal down-left Diagonal right Mean Horizontal down Vertical left Horizontal up Digital Video Compression Fundamentals and Standards 37

Remove Perceptual Redundancy In-loop deblocking filtering Remove blocking artifact Result from block based motion Remove Perceptual Redundancy In-loop deblocking filtering Remove blocking artifact Result from block based motion compensation Result from block based transform coding q 0 q 1 q 2 q 3 QP p 3 p 2 p 1 p 0 2008/12/26 Digital Video Compression Fundamentals and Standards 38

1/4 -pel Resolution Motion Estimation Integer search positions Best integer match Half-pel search positions 1/4 -pel Resolution Motion Estimation Integer search positions Best integer match Half-pel search positions Best half-pel match Quarter-pel search positions Best quarter-pel match 2008/12/26 Digital Video Compression Fundamentals and Standards 39

The H. 264/AVC Encoder Control Data Coder Controller Transform/ Quantization Motion Compensation Intra-frame Prediction The H. 264/AVC Encoder Control Data Coder Controller Transform/ Quantization Motion Compensation Intra-frame Prediction Residue Entropy Coding Inverse Transform/ De-Quantization Motion Vector De-blocking Filter Motion Estimation 2008/12/26 Digital Video Compression Fundamentals and Standards 40

Outline Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions 2008/12/26 Digital Outline Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions 2008/12/26 Digital Video Compression Fundamentals and Standards 41

H. 264 Reference Software JM http: //iphome. hhi. de/suehring/tml/ Current software version: JM 15 H. 264 Reference Software JM http: //iphome. hhi. de/suehring/tml/ Current software version: JM 15 Benchmark 2008/12/26 Digital Video Compression Fundamentals and Standards 42

Reference Software Demo 2008/12/26 Digital Video Compression Fundamentals and Standards 43 Reference Software Demo 2008/12/26 Digital Video Compression Fundamentals and Standards 43

Outline Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions 2008/12/26 Digital Outline Introduction Video Compression Standards Simulation Reference Software Future Work and Conclusions 2008/12/26 Digital Video Compression Fundamentals and Standards 44

Future Work Fast Mode Decision Algorithm Interpolation Filter Design Deblocking Filter Design DCT-Based Motion Future Work Fast Mode Decision Algorithm Interpolation Filter Design Deblocking Filter Design DCT-Based Motion Estimation Implementation Based on TI DSP 2008/12/26 Digital Video Compression Fundamentals and Standards 45

Fast Mode Decision Algorithm The computational cost of H. 264 is large Variable block-size Fast Mode Decision Algorithm The computational cost of H. 264 is large Variable block-size ME Multiple reference frames Fast mode decision is needed for reducing the computation time 2008/12/26 Digital Video Compression Fundamentals and Standards 46

Interpolation Filter Design In order to estimate and compensate the fractional-pel displacements Adaptive Interpolation Interpolation Filter Design In order to estimate and compensate the fractional-pel displacements Adaptive Interpolation filter for replacing the fixed coefficient filter 2008/12/26 Digital Video Compression Fundamentals and Standards 47

Deblocking Filter Design Block based ME and Transform result in the annoying blocking artifact Deblocking Filter Design Block based ME and Transform result in the annoying blocking artifact Reduce the blocking artifact can increase the quality of compressed video 2008/12/26 Digital Video Compression Fundamentals and Standards 48

DCT Based Motion Estimation Robust even in noisy environment Complexity comparison DCT Based ME DCT Based Motion Estimation Robust even in noisy environment Complexity comparison DCT Based ME O(M 2) Block Based ME O(N 2×M 2) Concept Pseudo Phase Similar to 2008/12/26 Digital Video Compression Fundamentals and Standards 49

Implementation Based on TI DSP TMS 320 C 6416 TI DM 642 2008/12/26 Digital Implementation Based on TI DSP TMS 320 C 6416 TI DM 642 2008/12/26 Digital Video Compression Fundamentals and Standards 50

Conclusions Thank You Q&A 2008/12/26 Digital Video Compression Fundamentals and Standards 51 Conclusions Thank You Q&A 2008/12/26 Digital Video Compression Fundamentals and Standards 51

References (1/2) [1] Yun Q. Shi and Huifang Sun, “Image and Video Compression for References (1/2) [1] Yun Q. Shi and Huifang Sun, “Image and Video Compression for Multimedia Engineering: Fundamentals, Algorithms, and Standards”, CRC press, 2000. [2] Yao Wand, Jorn Ostermann and Ya-Qin Zhang, “Video Processing and Communications”, Prentice Hall, 2007. [3] Richardson, Lain E. G. , “Video Codec Design: Developing Image and Video Compression Systems”, John Wiley & Sons Inc, 2002. [4] Barry G, Haskell, Atul Puri and Arun N. Netravali, “Digital Video : An Introduction to MPEG-2”, Boston : Kluwer Academic, 1999. [5] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, “Overview of the H. 264/AVC video coding standard”, IEEE Trans. on Circuits and systems for video Technology, vol. 13, no. 7, pp. 560 -576, July 2003. [6] G. Sullivan and T. Wiegand, “Video Compression - From Concepts to the H. 264/AVC Standard”, Proceedings of the IEEE, Special Issue on Advances in Video Coding and Delivery, December 2004. [7] 酒井善則、吉田俊之 共著,白執善 編譯,“影像壓縮技術”,全華, 2004. 2008/12/26 Digital Video Compression Fundamentals and Standards 52

References (2/2) [8] Thomas Wedi, “Adaptive Interpolation Filters and High-Resolution Displacements For Video Coding”, References (2/2) [8] Thomas Wedi, “Adaptive Interpolation Filters and High-Resolution Displacements For Video Coding”, IEEE Trans. on Circuits and Systems For Video Technology, vol. [9] Dong-Hwan Kim, Hwa-Yong Oh, O˘guzhan Urhan, Sarp Ertürk and Tae. Gyu Chang, “Optimal Post-Process/In-Loop Filtering for Improved Video Compression Performance”, IEEE Trans. on Consumer Electronics, vol. 53, no. 4, Nov. 2007. [10] Shu-Fa Lin, Meng-Ting Lu, and Homer Chen, ” Fast Multi-Frame Motion Estimation for H. 264 and Its Applications to Complexity-Aware Streaming”, IEEE International Symposium on Circuits and Systems, 2005. [11] Kai-Ting Cheng and Hsueh-Ming Hang, “Acceleration and Implementation of H. 264 Encoder and Scalable Extension of H. 264 Decoder on TI DSP Platform”, master thesis, June 2007 2008/12/26 Digital Video Compression Fundamentals and Standards 53