High Efficiency Video Codec

presentation on HEVC/H265 Codec

1. High Efficiency Video Coding HEVC/H.265 Tejus Adiga M Department of Electronics and Communication, NMAMIT, Nitte . Presented By:

2. A New Standard for Video Coding Video is going richer, all growing the bit rate Frame size– 4K to 8K Ultra HD Temporal resolution - 24 FPS to 240 FPS Color resolution, bit depth - 8bpc to 12 bpc Stereo and 3D Video Technology Process intensive Algorithms in consumer electronics is now affordable Devices for Recording/Displaying HD to ultra HD video is affordable Multi Threaded multi Core CPUs and hardware Accelerators Hardware Video Decoders Necessary video data rate grows faster than feasible network transport capacities Better Video compression standard needed. New ITU-T standard H265 addressing needs of an efficient video coder of future Same Visual quality – 50% reduction in bitrate than H264/MPEG4 part 10. B etter visual quality – Same bitrate as of H264/MPEG4 part 10. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 2

3. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 3 Block Diagram Fig 1: Block diagram of HEVC encoder CABAC : Context Adaptive Binary Arithmetic Coding. CTU: Coding Tree Unit . 64x64 L uma block (Y) and two 32x32 C hroma block (U and V) as in YUV 420 format. SAO: Sample Adaptive offset: Non linear amplitude mapping to better reconstruct original pixel amplitudes.

4. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 4 Coding Modes Intra Frame Coding (I Frames) CTU to encode is predicted from previously encoded and decoded CTUs of upper rows and left column of the same frame. Inter Frame Coding (P and B frame) Coded using Motion Estimation and M otion Compensation or Skip mode. P Frame: Coded with reference to Previously encoded and decoded frames. B Frame: Coded with reference to Previous and Future encoded and decoded frames.

5. 28 October 2015 Block Partitioning Frame is partitioned into Non Overlapping 64x64 Coding Tree Units (CTU) A CTU consists of 64x64 Luma Coding Tree Block (CTB) and two 32 x 32 Chroma CTB for U and V components. Every CTB is further split into one or multiple Coding Unit (CU) in Quad tree structure. Every CU has an associated Prediction Unit (PU) and Transform Unit (TU). Prediction Unit (PU) is partitioning to perform Intra/Inter prediction. It has root in CU and can be further split down to 4x4 block including rectangle blocks like 8x4, 16x8 etc. Transform Unit (TU) is partitioning to perform Transform and Entropy coding of Prediction Residual blocks down to 4x4 size. Department of Electronics and Communications, NMAMIT, Nitte. CTU Luma (Y) CTB CTB CTB CB CB CB CB CB CB CB CB CB CB 5

6. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 6 Intra Frame Coding Transform Unit size can be from 4x4 upto 32x32 33 Directional Angular prediction with Planar and DC prediction modes are used to predict a TB A Prediction Block is formed with previously decoded boundary samples from spatially neighboring TBs. A PB with minimum prediction error is selected using the error metric like SAD, MAD, SSE, MSE Prediction Residual block is found by  

7. Integer DCT is applied to Residual block with Butterfly computation . Integer DCT of size 4x4, 8x8, 16x16 and 32x32 computed with fast Separable Butterfly computation. Alternatively for only 4x4 residual block DST can be used as it better fit the statistical property that the residual amplitudes tend to increase as the distance from the boundary samples that are used for prediction becomes larger. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 7 Transform 4x4 transform 8x8 transform

8. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 8 Transform

9. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 9 Scalar Quantization and Scaling Integer DCT introduces Non Linearity. DCT Matrix is split into two integer matrix with appropriate scaling and combined with Quantization to form M f Inverse Operation . .   Quantization and Scaling Inverse Quantization and Scaling

10. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 10 Scalar Quantization and Scaling Transformed Residual coefficients are quantized with Scalar Quantization. Quantization Parameter (QP): Based on constraints like target bitrate, quality a value in between 0 to 51 need to be specified termed as Quantization Parameter. For every 6 QP value increase the Qstep doubles. Lower the QP, higher the video quality but less compression Quantization is done by multiplying M f with Residual block. Inverse Quantization at decoder side can be done by  

11. The selection of the coefficient scanning order depends on the directionalities of the intra picture prediction. Scanning methods Diagonal upright scanning: Default Vertical Scanning: U sed when the prediction direction is close to horizontal Horizontal Scanning : Used when the prediction direction is close to vertical Scanned Coefficients are Entropy Coded using Context Adaptive Binary Arithmetic Coding (CABAC) and formatted in bit stream format as specified by H265 NAL Unit Syntax 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 11 Entropy Coding

12. Current frame is encoded using previously decoded frame as reference. Motion Estimation and Motion Compensation is carried out at CB level to get he best match of current CB in the reference frame in the search window. Offset ( Δ x, Δ y) between current CB and motion estimated CB referred to as Motion Vector . 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 12 Intra Picture Coding

13. Difference between current CB and motion estimated CB is computed to get the Prediction Block which is Transformed, quantized and entropy coded along with Motion Vectors. Various Fast motion estimation algorithms are proposed to speedup the motion estimation process like Three Step, Logarithmic, Hexagonal, Diamond, Small Diamond etc. Optimizations: Motion Vector can be predicted spatially and temporally. Motion Vector Merge: Average of previously encoded motion vectors of current frame. Skip Mode: MVs are copied from MVs of some other CB. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 13 Encoding of motion Estimated Parameters

14. Due to block processing of CBs there occurs discontinuities in adjacent blocks. To prevent excessive filtering filter are applied to PBs greater than 4x4 blocks. Filter parameters β and t c are derived from QP. Types of filtering: Strong Filter: High variation in pixel values at boundary. P0, P1, P2 and Q0, Q1, Q2 are filtered with (1, 2, 2, 2, 1)/8, ( 1, 1, 1, 1)/4, and ( 2, 3, 1, 1, 1)/8 Weak Filter: Slowly varying pixel values at boundary. P0 and Q0 are filtered with (8, 19, − 1, 9, −3)/32 No Filter: Skip filtering for intra frame 4x4 block, inter frame blocks whose MV derived from same reference frame. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 14 In Loop De-Blocking Filter

15. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 15 Comparison of HEVC with H.264 Block Partitioning in H264 and HEVC Subjective quality comparison of H264 and HEVC

16. High degree of parallelism in decoding can take advantage of SIMD instruction sets like MMX, SSE, SSE2, 3D Now etc in Software Decoding. As decoding operation is modular wrt CTUs it is better suited to run on Graphics processors Dedicated VLSI Decoders can take advantage of modularity of HEVC to accelerate decoding. Can provide better video quality for Real time video communication applications. Efficient storage of Video. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 16 Advantages

17. Better suited for parallel processing. HEVC will accelerate Video services on Mobile devices. Competitors like VP9 from Google and Vorbis from XIPH.org provide royalty free codecs that is slightly inferior to HEVC. HEVC Specification is not yet completely finalized. Opportunity to improve efficiency further. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 17 Conclusion and Future

18. Overview of the High Efficiency Video Coding (HEVC) Standard, G. J. Sullivan, J. R . Ohm, W. J . Han, and T. Wiegand , IEEE Transactions on Circuits and Systems for Video Technology , Vol 22, no 12, Jan 2013. HEVC Complexity and Implementation Analysis, Frank Bossen , Benjamin Bross , Karsten Suhring , and David Flynn, IEEE Transactions on Circuits and Systems for Video Technology , vol. 22, no. 12, Jan 2013. Comparison of the Coding Efficiency of Video Coding Standard – Including High Efficient Video Coding (HEVC ), J.R . Ohm, G. J. Sullivan, H. Schwarz, T. K. Tan, T. Wiegand , IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, Jan 2013. HEVC Deblocking Filter, Andrey Norkin , Gisle Bjøntegaard , Arild Fuldseth , Matthias Narroschke , Masaru Ikeda , Kenneth Andersson , Minhua Zhou, and Geert Van der Auwera , IEEE Transactions on Circuits and Systems for Video Technology , vol. 22, no. 12, Jan 2013. Overview of H264/AVC video coding standard, T. Wiegand , G. J. Sullivan, G. Bjontegaard , and A. Luthra , IEEE Transactions on Circuits and Systems for Video Technology, vol 13, no 7, July 2003. Overview of the scalable video coding extension of the H.264/AVC standard, H . Schwarz, D. Marpe , and T. Wiegand , IEEE Trans. Circuits Systems for Video Technology, vol. 17, no. 9, pp. 1103–1120, Sep. 2007 . HEVC : The New Gold Standard for Video Compression, Mahsa T. Pourazad , Colin Doutre , Maryam Azimi , and Panos Nasiopoulos , IEEE Consumer Electronics Magazine, July 2012. White Paper: 4x4 Transformation and Quantization in H264 AVC, Iain Richardson. A High-Throughput VLSI Architecture for Deblocking Filter in HEVC, Weiwei Shen, Qing Shang, Sha Shen, Yibo Fan, Xiaoyang Zeng, IEEE Trans. Circuits Systems for Video Technology , Vol 22, no 12, Jan 2013. CE10: Core transform design for HEVC, A . Fuldseth , G. Bjøntegaard , M. Budagavi and V . Sze, JCTVC-G495, November 2011 ITU-T Recommendation H265, High efficiency video coding. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 18 References