3D-HEVC (The 3D Extension of High Efficiency Video Coding, 3D-HEVC)是目前最新的三維視訊編碼國際標準，以多視角加景深(Multiview plus Depth, MVD)的三維視訊格式，豐富多媒體應用。在編碼彩色視訊時，除了利用時間域與空間域之參考資訊外，還能利用視角間之參考資訊；在3D-HEVC景深圖的畫面內編碼中，Depth Modelling Mode (DMM)與視角合成最佳化等先進的編碼工具能夠提升景深圖編碼效率與維持合成視角品質。眾多的先進編碼技術提高編碼效率，但同時也大幅增加編碼時間與複雜度。本論文提出創新且完整的3D-HEVC快速編碼方法，包含彩色視訊快速編碼、景深圖畫面內快速編碼與景深圖畫面間快速編碼。
針對彩色視訊快速編碼，本論文分別對獨立視角及相依視角計算時間域、空間域與視角域編碼樹單位之位元失真成本的皮爾遜相關係數來分析相關性，依據相關性較高的參考資訊來源，提出快速編碼演算法。本論文提出編碼單位深度範圍預測與調整演算法及計算位元失真成本臨界值以提早終止編碼單位之分割；根據高相關編碼樹單位之最佳預測模式的複雜度與分割方向提出快速預測單位模式決策；並動態調整移動估計之搜尋範圍。
針對景深圖畫面內快速編碼，本論文利用人眼視覺系統的特性，提出基於視覺感知之快速演算法，以加速景深圖的畫面內編碼。首先，我們利用Otsu自動分割法，將景深圖分割成背景、中景、前景三個區域，且透過修正後的MPEG-7標準內之邊緣分類法，接著再利用景深可辨視覺差異模型，偵測出會影響人眼視覺觀測的景深圖感知邊緣；最後我們依照不同的景深分割區域與邊緣分布的情況，篩選所需執行的畫面內預測之角度以及判斷是否執行DMM。我們也利用景深圖影像區塊邊緣的連續性與位元失真成本，結合感知邊緣像素的分布情況，做快速編碼單位決策。
針對景深圖畫面間編碼，本論文同時結合上述所提之彩色視訊快速編碼與景深圖畫面內快速編碼的演算法，整合成一個完整的快速編碼方法。另外加入Merge模式提前決策、Skip模式提前決策與視角間編碼資訊檢測等快速演算法同時結合視角相關性與視覺感知特性來加速景深圖畫面間編碼。
實驗結果顯示本論文所提出彩色視訊快速演算法能夠節省平均40.75%的彩色視訊編碼時間；所提出的景深圖畫面內快速編碼演算法可以節省平均53.09%的景深圖編碼時間；所提出的景深圖畫面間快速編碼演算法可以節省平均30.95%的景深圖編碼時間。整體所節省的平均編碼時間大幅優於數篇參考文獻，同時在畫面品質或編碼效率幾乎沒有損失，驗證了本論文所提演算法的有效性。

3D-HEVC (The 3D Extension of High Efficiency Vide Coding) is the newest 3D video coding standard, which enriches multimedia applications with the video format of multi-view plus (MVD) format. For the texture coding, 3D-HEVC utilizes not only the information of temporal and spatial domains but also that of inter-view domain. For the depth map coding in 3D-HEVC, the advanced coding tools, such as depth modelling mode (DMM) and view synthesis optimization (VSO), enhance the coding efficiency of the depth map and the quality of the synthesized view. The advanced coding tools improve the coding efficiency; however, the time consumption and complexity of 3D-HEVC also increase significantly. In this dissertation, the novel and complete fast coding algorithms for 3D-HEVC are proposed, which includes the fast texture coding, fast depth intra coding and fast depth inter coding.
For fast texture coding, we individually calculate the Pearson correlation coefficients by the rate-optimization cost (RD-cost) of coding tree units (CTUs) in the temporal, spatial and inter-view domains to analyze the correlations for independent view and dependent view. The fast coding algorithms are based on the coding information of CTUs with higher correlations. The proposed algorithm predicts and dynamically adjusts the depth range of the coding unit (CU). The RD-cost threshold is estimated to early terminate the CU split. The fast prediction unit (PU) mode decision is proposed according to the complexity and partition direction of the best PU modes obtained from the highly correlated CTUs. In addition, the search range is adaptively adjusted for motion estimation.
For fast depth intra coding, we utilize the characteristics of human visual system to propose a fast algorithm based on visual perception for the acceleration of the depth intra coding. Firstly, we segment the depth map into background, middle-ground and foreground by Otsu’s auto-thresholding. Then, the edges in depth map are categorized to different angles by the modified MPEG-7 edge discriminator. Furthermore, we detect the perceptual edges based on just noticeable depth difference (JNDD) model to extract the areas that may affect the visual perception. According to depth map segmentation and edge distribution, we condense the corresponding intra angular modes and determine whether to perform DMM. We also incorporate the boundary continuity and rate-distortion cost (RD-cost) thresholding to propose the fast CU decision.
For depth inter coding, an integrated scheme combining the strategies of fast texture coding and fast depth intra coding is proposed. The early merge decision, early skip decision and the checking of the inter-view status are additional designed by combining the inter-view and perceptual properties at the same time.
The experimental results show that the proposed fast texture coding algorithm reduces 40.75% of the texture coding time on average. The proposed fast depth intra coding algorithm eliminates 53.09% of the depth coding time on average. The proposed fast depth inter coding algorithm decreases 30.95% of the depth coding time on average. The coding performances of the proposed algorithms outperform numerous previous works significantly. The loss of the perceptual quality of the video and the coding efficiency are ignorable, which verifies the effectness of the proposed algorithms.

Chapter 1 Introduction 25
1.1 Overview of the 3D extension of High Efficiency Video Coding (3D-HEVC) 25
1.2 Encoder Structure of 3D-HEVC 31
1.2.1 Coding Unit (CU) 31
1.2.2 Prediction Unit (PU) 33
1.2.3 Transform Unit (TU) 34
1.3 Intra Prediction in 3D-HEVC 34
1.3.1 Conventional Intra Prediction 34
1.3.2 Depth Intra Coding 37
1.3.2.1 Depth Modelling Modes (DMM) 38
1.3.2.2 Segment-Wise DC Coding (SDC) 40
1.3.2.3 Procedure of Depth Intra Coding 41
1.4 Inter Prediction in 3D-HEVC 42
1.4.1 Advanced Motion Vector Prediction (AMVP) 44
1.4.2 Merge/Skip in 3D-HEVC 45
1.5 Advanced Coding Techniques 48
1.5.1 Inter-view Coding Tools 48
1.5.1.1 Disparity Vector from Neighboring Blocks (NBDV) 48
1.5.1.2 Depth Oriented Neighboring Block Based Disparity Vector (DoNBDV) 50
1.5.2 Inter-component Coding Tools 52
1.5.2.1 Motion Parameter Inheritance (MPI) 52
1.5.2.2 Quadtree Limitation and Predictive Coding (QTL-PC) 52
1.5.2.3 Depth-based Block Partitioning (DBBP) 53
1.5.3 Depth Map Coding Tools 54
1.5.3.1 Depth Intra Skip (DIS) 54
1.5.3.2 View Synthesis Optimization (VSO) 55
1.6 Prediction Procedure of PU Modes 56
1.7 Prediction Structure 58
1.8 Motivation and Analysis 61
1.9 Dissertation Organization 68
Chapter 2 Literature Review 71
2.1 Review of Fast Algorithms for Color Texture Coding 71
2.2 Review of Fast Algorithms for Depth Map Coding 73
Chapter 3 Fast Texture Coding Based on Spatial, Temporal and Inter-view Correlations 78
3.1 Analysis of Spatial, Temporal and Inter-View Correlations 79
3.1.1 Pearson Correlation Coefficient 79
3.1.2 Calculation of Pearson Correlation Coefficients in Independent View 82
3.1.3 Calculation of Pearson Correlation Coefficients in Dependent View 84
3.1.4 Extraction of the Reference information 88
3.2 CU Depth Range Prediction and Dynamic Adjustment 93
3.2.1 CU Depth Range Prediction 93
3.2.2 Dynamic Adjustment for CU Depth Range 95
3.3 Fast Inter PU Mode Decision 97
3.4 Search Range Adjustment 101
3.5 CU Early Termination Based on RD-cost Estimation 104
3.6 Overall Algorithm 113
3.7 Simulation Results 115
Chapter 4 Fast Depth Map Coding by Visual Perception 131
4.1 Depth Map Segmentation 132
4.2 Edge Classification by Modifying Edge Discriminator in MPEG-7 137
4.3 Perceptual Edge Detection 140
4.4 Fast Depth Intra Coding Based on Visual Perception 143
4.4.1 Fast Intra Mode Decision 143
4.4.2 CU Early Termination Algorithm 150
4.4.3 Overall Algorithm 155
4.5 Fast Depth Inter Coding 158
4.5.1 Fast Inter Prediction 158
4.5.1.1 Early Merge Decision for Dependent View 159
4.5.1.2 Early Skip Decision 160
4.5.1.3 CU Early Termination Based on Inter-view Merge Status 161
4.5.2 Fast Intra Prediction 161
4.6 Simulation Results of Fast Depth Intra Coding 162
4.7 Simulation Results of Fast Depth Inter Coding 173
Chapter 5 Conclusion and Future Work 183
Reference 187

[1] M. Tanimoto, M. P. Tehrani, T. Fujii, and T. Yendo, “Free-Viewpoint TV,” IEEE Signal Processing Magazine, vol. 28, no. 1, pp. 67-76, January 2011.
[2] A. Smolic, K. Mueller, P. Merkle, C. Fehn, P. Kauff, P. Eisert, and T. Wiegand, “3D Video and Free Viewpoint Video - Technologies, Applications and MPEG Standards,” 2006 IEEE International Conference on Multimedia and Expo, pp. 2161-2164, July 2016.
[3] G. C. Burdea and P. Coiffet, Virtual Reality Technology, Hoboken, NJ, USA: Wiley, 2003.
[4] R. Azuma, Y. Baillot, R. Behringer, S. Feiner, S. Julier, and B. MacIntyre, “Recent Advances in Augmented Reality,” IEEE Computer Graphics and Applications, vol. 21, no. 6, pp. 34-47, Nov./Dec. 2001.
[5] G. Tech, Y. Chen, K. Müller, J. R. Ohm, A. Vetro, and Y. K. Wang, “Overview of the Multiview and 3D Extensions of High Efficiency Video Coding,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 11, pp. 35-49, January 2016.
[6] Y. Chen, G. Tech, K. Wegner, and S. Yea, “Test Model 11 of 3D-HEVC and MV-HEVC,” Document JCT3V-K1003, February 2015.
[7] P. Merkle, A. Smolic´, K. Müller, and T. Wiegand, “Efficient Prediction Structures for Multiview Video Coding,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 17, no. 11, pp. 1461-1473, November 2007.
[8] P. Merkle, A. Smolic, K. Müller, and T. Wiegand, “Multi-View Video Plus Depth Representation and Coding,” in Proceeding of 2007 IEEE International Conference on Image Processing, vol. 1, pp. I-203-I-204, 2007.
[9] M. M. Hannuksela, Y. Yan, X. Huang, and H. Li, “Overview of the Multiview High Efficiency Video Coding (MV-HEVC) Standard,” in Proceedings of 2015 IEEE International Conference on Image Processing (ICIP), pp.2154-2158, September 2015.
[10] C. Fehn, “Depth-Image-Based Rendering (DIBR), Compression, and Transmission for a New Ppproach on 3D-TV,” in Proceedings of SPIE Conference Stereoscopic Displays and Virtual Reality Systems XI, vol. 5291, pp. 93-104, January 2004.
[11] G. J. Sullivan, J. Ohm, W. J. Han, and T. Wiegand, “Overview of the High Efficiency Video Coding (HEVC) Standard,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, pp. 1649-1668, December 2012.
[12] G. Sanchez, R. Cataldo, R. Fernandes, L. Agostini, and C. Marcon, “3D-HEVC Depth Maps Intra Prediction Complexity Analysis,” in proceedings of 2016 IEEE International Conference on Electronics, Circuits and Systems (ICECS), Monte Carlo, Monaco, December 2016.
[13] Y. S. Heo, G. Bang, and G. H. Park, “Adaptive Merge List Construction for 3D-HEVC Fast Encoder,” Electronics Letters, vol. 52, no. 8, pp. 604-605, April 2016.
[14] H. Ding, W. An, T. Yan, and Q. Zhang, “Fast Depth Level Range Determination Algorithm for 3D-HEVC System Application,” International Journal of Performability Engineering, vol. 14, no. 8, pp. 1713-1718, August 2018.
[15] Y. X. Song and K. B. Jia, “Early Merge Mode Decision for Texture Coding in 3D-HEVC,” Journal of Visual Communication and Image Representation, vol. 33, pp. 60-68, November 2015.
[16] Q. Zhang, H. Chang, X. Huang, L. Huang, R. Su, and Y. Gan, “Adaptive Early Termination Mode Decision for 3D-HEVC Using Inter-View and Spatio-Temporal Correlations,” AEU - International Journal of Electronics and Communications, vol. 70, no. 5, pp. 727-737, May 2016.
[17] Z. Pan, X. Yi, and L. Chen, “Motion and Disparity Vectors Early Determination for Texture Video in 3D-HEVC,” Multimedia Tools and Applications, pp. 1-18, November 2018.
[18] G. Avila, R. Conceição, T. Bubolz, B. Zatt, M. Porto, L. Agostini, and G. Correa, “Complexity Reduction of 3D-HEVC Based on Depth Analysis for Background and ROI Classification,” in proceedings of 2017 25th European Signal Processing Conference (EUSIPCO), Kos, Greece, August 2017.
[19] S. Bakkouri, A. Elyousfi, and H. Hamout, “Fast CU Size and Mode Decision Algorithm for 3D-HEVC Interceding,” Multimedia Tools and Applications, vol. 79, pp. 6987-7004, March 2020.
[20] J. Chen, B. Wang, J. Liao, and C. Cai, “Fast 3D-HEVC Inter Mode Decision Algorithm Based on the Texture Correlation of Viewpoints,” Multimedia Tools and Applications, vol. 78, pp. 29291–29305, October 2019.
[21] Y. Li, G. Yang, Y. Zhu, X. Ding, and R. Gong, “Probability Model-Based Early Merge Mode Decision for Dependent Views Coding in 3D-HEVC,” ACM Transactions on Multimedia Computing, Communications, and Applications, vol. 14, no.4, November 2018.
[22] G. Sanchez, L. Agostini, and C. Marcon, “A Reduced Computational Effort Mode-Level Scheme for 3D-HEVC Depth Maps Intra-Frame Prediction,” Journal of Visual Communication and Image Representation, vol. 54, pp. 193-203, July 2018.
[23] R. Jing, Q. Zhang, B. Wang, P. Cui, and J. Huang, “CART-Based Fast CU Size Decision and Mode Decision Algorithm for 3D-HEVC,” Signal, Image and Video Processing, vol. 13, pp. 209-216, March 2019.
[24] H. Hamout and A. Elyousfi, “Fast 3D‑HEVC PU Size Decision Algorithm for Depth Map Intra‑Video Coding,” Journal of Real-Time Image Processing, Early Access, pp. 1-15, June 2019.
[25] J. Chen, B. Wang, H. Zeng, C. Cai, and K. K. Ma, “Sum-of-Gradient Based Fast Intra Coding in 3D-HEVC for Depth Map Sequence (SOG-FDIC),” Journal of Visual Communication and Image Representation, vol. 48, pp. 329-339, October 2017.
[26] H. B. Zhang, C. H. Fu, Y. L. Chan, S. H. Tsang, and W. C. Siu, “Probability-based Depth Intra Mode Skipping Strategy and Novel VSO Metric for DMM Decision in 3D-HEVC,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 28, no. 2, pp. 513-527, February 2018.
[27] J. Zuo, J. Chen, H. Zeng, C. Cai, and K. K. Ma, “Bi-Layer Texture Discriminant Fast Depth Intra Coding for 3D-HEVC,” IEEE Access, vol. 7, pp. 34265-34274, February 2019.
[28] Q. Zhang, R. Jing, B. Wang, P. Cui, C. Zhou, T. Yan, S. Kanyukt, and W. Si, “Fast Mode Decision Based on Gradient Information in 3D-HEVC,” IEEE Access, vol. 7, pp. 135448-135456, September 2019.
[29] R. Zhang, K. Jia, P. Liu, and Z. Sun, “Fast Intra‑Mode Decision for Depth Map Coding in 3D‑HEVC,” Journal of Real-Time Image Processing, pp. 1-10, October 2019.
[30] H. Hamout and A. Elyousfi, “Fast Depth Map Intra-Mode Selection for 3D-HEVC Intra-Coding,” Signal, Image and Video Processing, pp. 1-10, March 2020.
[31] E. G. Mora, J. Jung, M. Cagnazzo, and B. Pesquet-Popescu “Initialization, Limitation, and Predictive Coding of the Depth and Texture Quadtree in 3D-HEVC,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 24, no. 9, pp. 1554-1564, September 2014.
[32] H. Chen, C. H. Fu, Y. Zhang, Y. L. Chan, and W. C. Siu, “Early Merge Mode Decision for Depth Maps In 3D-HEVC,” in Proceedings of International Conference on Digital Signal Processing, London, UK, November 2017.
[33] N. Zhang, D. Zhao, Y. W. Chen, J. L. Lin, W. Gao, “Fast encoder decision for texture coding in 3D-HEVC,” Signal Processing: Image Communication, vol. 29, pp. 951-961, June 2014.
[34] F. Chen, S. Liu, Z. Peng, Q. Hu, G. Jiang, and M. Yu, “Bayesian-Theory-Based Fast CU Size and Mode Decision Algorithm For 3D-HEVC Depth Video Inter-Coding,” KSII Transactions on Internet and Information Systems, vol. 12, no. 4, pp. 1730-1747, April 2018.
[35] J. Lei, J. Duan, and F. Wu, “Fast Mode Decision Based on Grayscale Similarity and Inter-View Correlation For Depth Map Coding In 3D-HEVC,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 28, no. 3, pp. 706-718, March 2016.
[36] G. Sanchez, M. Saldanha, B. Zatt, M. Porto, L. Agostini, and C. Marcon, “Edge-Aware Depth Motion Estimation – A Complexity Reduction Scheme for 3D-HEVC,” 2017 25th European Signal Processing Conference (EUSIPCO), in proceedings of 2017 25th European Signal Processing Conference (EUSIPCO), Kos, Greece, August 2017.
[37] M. Chen, Y. Yang, Q. Zhangm X. Zhao, and X. Huang, “Low Complexity Depth Mode Decision for HEVC-Based 3D Video Coding,” Optik, vol. 127, no. 11, pp. 4758-4767, June 2016.
[38] Y. W. Liao, M. J. Chen, C. H. Yeh, J. R. Lin, and C. W. Chen, “Efficient Inter-Prediction Depth Coding Algorithm Based on Depth Map Segmentation For 3D-HEVC,” Multimedia Tools and Applications, vol. 78, pp. 10181-10205, April 2019.
[39] K. Pearson, “Mathematical Contributions to the Theory of Evolution. III. Regression, Heredity, and Panmixia,” Philosophical Transactions of the Royal Society of London. Series A, Containing Papers of a Mathematical or Physical Character, vol. 187, pp. 253-318, 1896.
[40] J. L. Rodgers and W. A. Nicewander, “Thirteen Ways to Look at the Correlation Coefficient,” The American Statistician, vol. 42, pp. 59–66, February 1988.
[41] A. G. Asuero, A. Sayago, and A. G. Gonz´alez, “The Correlation Coefficient: An Overview,” Critical Reviews in Analytical Chemistry, vol. 36, pp. 41-59, 2006.
[42] P. Mukhopadhyay, An Introduction to the Theory of Probability, World Scientific, 2012.
[43] A. W. Marshall and I. Olkin, “A One-Sided Inequality of the Chebyshev Type,” The Annals of Mathematical Statistics, vol. 31, no. 2, pp. 488-491, 1960.
[44] 3D-HEVC reference software version 16.0 (HTM-16.0), available online at https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/tags/ HTM-16.0/.
[45] K. Müller and A. Vetro, “Common Test Conditions of 3DV Core Experiments,” JCT3V-G1100, January 2014.
[46] G. Bjontegaard, “Calculation of Average PSNR Differences between RD Curves,” ITU-T SG16/Q6 Document, VCEG-M33, Austin, April 2001.
[47] G. Bjontegaard, “Improvements of the BD-PSNR Model,” ITU-T SG16/Q6, Document, VCEG-AI11, Berlin, July 2008.
[48] D. V. S. X. D. Silva, W. A. C. Fernando, G. Nur, E. Ekmekcioglu, and S.T. Worrall, “3D Video Assessment with Just Noticeable Difference in Depth Evaluation,” in Proceedings of 2010 IEEE International Conference on Image Processing, pp. 4013-4016, September 2010.
[49] D. V. S. X. D. Silva, E. Ekmekcioglu, W. A. C. Fernando, and S. T. Worrall, “Display Dependent Preprocessing of Depth Maps Based on Just Noticeable Depth Difference Modeling,” IEEE Journal of Selected Topics in Signal Processing, vol. 5, no. 2, pp. 335-351, April 2011.
[50] S. W. Jung and S. J. Ko, “Depth Sensation Enhancement Using the Just Noticeable Depth Difference,” IEEE Transactions on Image Processing, vol. 21, no. 8, pp. 3624-3637, August 2012.
[51] N. Jayant, “Signal compression: technology targets and research directions,” IEEE Journal on Selected Areas in Communications, vol. 10, no. 5, pp. 796-818, June 1992.
[52] N. Jayant, J. Johnston, and R. Safranek, “Signal compression based on models of human perception,” Proceedings of the IEEE, vol. 81, no 10, pp. 1385-1422, Oct. 1993.
[53] N. Otsu, “A Threshold Selection Method from Gray Level Histograms,” IEEE Transactions on Systems, Man, and Cybernetics, vol. 9, no. 1, pp. 62-66, January 1979.
[54] ISO/IEC/JTC1/SC29/WG11, “MPEG-7 XM Document: MPEG-7 Visual Part Experimentation Model Version 10.0,” MPEG Document N4063, March 2001.
[55] C. M. Fu, E. Alshina, A. Alshin, Y. W. Huang, C. Y. Chen, C. Y. Tsai, C. W. Hsu, S. M. Lei, J. H. Park, and W. J. Han, “Sample adaptive offset in the HEVC standard,” IEEE Transactions on Circuits and System for Video Technology, vol. 22, no. 12, pp. 1755–1764, Dec. 2012.
[56] Yu-Chih Hsu, Jie-Ru Lin, Mei-Juan Chen, Chia-Hung Yeh, Min-Hui Lin and Wei-Chieh Lu, “Acceleration of Depth Intra Coding for 3D-HEVC by Efficient Early Termination Algorithm,” in Proceedings of IEEE Asia Pacific Conference on Circuits and Systems (APCCAS 2018), Chengdu, China, October 2018.
[57] W. Zhou, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image Quality Assessment: from Error Visibility to Structural Similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600-612, April 2004.