Region-based Template Matching for Next
Generation Video Coding
vorgelegt von
Gayathri Venugopal, M.Sc
von der Fakultät IV — Elektrotechnik und Informatik
der Technischen Universität Berlin
zur Erlangung des akademischen Grades
Doktorin der Ingenieurwissenschaften
— Dr.-Ing. —
genehmigte Dissertation
Promotionsausschuss:
Vorsitzender: Prof. Dr. Klaus-Robert Müller
Gutachter: Prof. Dr.-Ing. Thomas Wiegand
Prof. Dr.-Ing. Heiko Schwarz
Prof. Dr.-Ing. Jens-Rainer Ohm
Tag der wissenschaftlichen Aussprache: 01 September 2022
Berlin 2022
ii
Contents
Acknowledgements v
Abstract vii
Zusammenfassung ix
List of Figures xi
List of Tables xv
1 Introduction 1
1.1 Motivation ............................................ 1
1.2 Problemstatement........................................ 2
1.3 Maincontributions ....................................... 3
2 Background 5
2.1 Outlineofvideocoding..................................... 5
2.2 State-of-the-art.......................................... 9
2.3 Copypredictionmethods.................................... 15
2.3.1 Intrablockcopy..................................... 15
2.3.2 Templatematching................................... 16
2.4 Literature review of template matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.5 Summary............................................. 23
3 Region-based template matching for intra coding 25
3.1 Introduction ........................................... 25
3.2 Theoreticalframework ..................................... 27
3.2.1 Prediction efficiencyofRTM.............................. 27
3.2.2 Rate-distortion performance of RTM . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.3 Algorithmdescription...................................... 43
3.4 Searchdesign .......................................... 45
3.5 Signalization........................................... 49
3.6 Complexityanalysis....................................... 50
3.7 Extension to the chroma components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.7.1 RTM-chroma as an additional prediction mode . . . . . . . . . . . . . . . . . . . 55
3.7.2 RTM-chroma as an intra derived mode only . . . . . . . . . . . . . . . . . . . . . 56
3.7.3 Discussion........................................ 57
3.8 Summary............................................. 58
4 Investigation of the RTM parameters for intra coding 59
4.1 Experimentalset-up....................................... 59
4.2 Template width η........................................ 61
4.3 Templateshape ......................................... 63
4.4 Region size parameter δ.................................... 66
4.5 Number of regions nr...................................... 68
4.6 Number of predictors np.................................... 72
4.7 Unequal region size parameter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.8 Regionshape........................................... 78
4.9 Regiondown-sampling ..................................... 82
iii
iv Contents
4.10 Summary............................................. 84
5 Region-based template matching for inter coding 85
5.1 Algorithmdescription...................................... 85
5.2 Predictionsignal......................................... 88
5.3 Centre Zofthesearchwindow................................. 92
5.4 Searchdesign .......................................... 93
5.5 Encodersearch.......................................... 95
5.6 Signalization........................................... 96
5.7 Investigation of the RTM parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.7.1 Region size parameter δInter .............................. 97
5.7.2 Number of regions nr................................. 98
5.7.3 Number of predictors from each list np........................ 100
5.8 Summary............................................. 102
6 Evaluation of the final RTM version 103
6.1 Final version and experimental results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
6.1.1 Intra-RTM........................................ 103
6.1.2 Inter-RTM........................................ 109
6.1.3 Combined results of intra-RTM and inter-RTM . . . . . . . . . . . . . . . . . . . 112
6.2 Statisticalanalysis........................................ 114
6.3 Comparative analysis against conventional TM . . . . . . . . . . . . . . . . . . . . . . . . 116
6.3.1 Intra-RTM........................................ 116
6.3.2 Inter-RTM........................................ 120
6.4 Analysis of intra-RTM for SCC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
6.4.1 Comparative analysis between intra-RTM and IBC . . . . . . . . . . . . . . . . . 121
6.4.2 Configuration of intra-RTM for SCC . . . . . . . . . . . . . . . . . . . . . . . . . 123
6.5 Further optimizations for complexity reduction . . . . . . . . . . . . . . . . . . . . . . . . 126
6.5.1 Block-sizerestriction .................................. 126
6.5.2 Encoder search optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
6.5.3 Chroma search optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
6.5.4 Discussion........................................ 132
6.6 Summary............................................. 133
7 Summary and conclusions 135
Publications 141
Appendix I Supplementary experimental results 143
Abbreviations and Acronyms 149
Symbols 151
Acknowledgements
It was after a few years of recurring thoughts and deliberations that I had convinced
myself to sign-up for the mission of accomplishing a doctoral title. It was indeed a
challenging and nerve-racking, as well as a memorable and proud journey.
First and foremost, I would like to express my sincere gratitude to Prof. Dr.-Ing.
Thomas Wiegand for allowing me to do a Ph.D. under him. I also thank him for his
valuable guidance and constructive feedback throughout this process. Further, I would
like to extend my gratitude to Dr.-Ing. Karsten Müller, Dr. rer. nat. Jonathan Pfaff,
Prof. Dr.-Ing. Heiko Schwarz and Dr.-Ing. Detlev Marpe for their time-to-time advice,
suggestions, and critiques for the successful completion of this thesis.
Next, I would like to thank the doctoral committee members: Prof. Dr. Klaus-
Robert Müller (Chairman), Prof. Dr.-Ing. Thomas Wiegand, Prof. Dr.-Ing. Heiko
Schwarz and Prof. Dr.-Ing. Jens-Rainer Ohm. Many thanks for accepting my invitation
to be a part of the committee and for taking up the pain of reviewing my thesis. I am
indebted to their valuable time and patience.
Also, I would like to convey my deepest appreciation to all the colleagues and staff
at Fraunhofer Heinrich Hertz Institute (HHI), Berlin. Their support, encouragement,
and assistance have helped the continuous progress of my research activities. In par-
ticular, I thank Dr.-Ing. Philipp Merkle, Santiago De-Luxán-Hernández, Valeri George,
Johannes Erfurt and Dr. Ivan Zupancic (former colleague) for their valuable experience,
knowledge and insightful suggestions. Special thanks to Johannes Erfurt for proofread-
ing and helping me with the thesis-summary in the German language. Furthermore, I
would like to acknowledge Gabriele Thiele for all the administrative support.
Lastly, I am eternally grateful to my parents and sister for their never-ending love,
care, and support. I don’t have enough words to thank them for bearing with the short-
tempered, grumpy and imperfect me. I also thank my late grandfather, whose void can
never be filled in my life, for showering his blessings and unconditional love on me.
I dedicate this work to my doting parents. Love you Acha and Amma!
v
Loading more pages...