Murn, Luka ORCID: 0000-0001-9041-647X, Blasi, Saverio, Smeaton, Alan F. ORCID: 0000-0003-1028-8389 and Mrak, Marta (2021) Improved CNN-based Learning of interpolation filters for low-complexity Inter prediction in video coding. IEEE Open Journal of Signal Processing, 2 . pp. 453-465. ISSN 2644-1322
Abstract
The versatility of recent machine learning approaches makes them ideal for improvement of next generation video compression solutions. Unfortunately, these approaches typically bring significant increases in computational complexity and are difficult to interpret into explainable models, affecting their potential for implementation within practical video coding applications. This paper introduces a novel explainable neural network-based inter-prediction scheme, to improve the interpolation of reference samples needed for fractional precision motion compensation. The approach requires a single neural network to be trained from which a full quarter-pixel interpolation filter set is derived, as the network is easily interpretable due to its linear structure. A novel training framework enables each network branch to resemble a specific fractional shift. This practical solution makes it very efficient to use alongside conventional video coding schemes. When implemented in the context of the state-of-the-art Versatile Video Coding (VVC) test model, 0.77%, 1.27% and 2.25% BD-rate savings can be achieved on average for lower resolution sequences under the random access, low-delay B and low-delay P configurations, respectively, while the complexity of the learned interpolation schemes is significantly reduced compared to the interpolation with full CNNs.
Metadata
Item Type: | Article (Published) |
---|---|
Refereed: | Yes |
Uncontrolled Keywords: | video compression; motion compensation; interpolation; machine learning; complexity reduction |
Subjects: | Computer Science > Artificial intelligence Computer Science > Image processing Computer Science > Machine learning Computer Science > Video compression |
DCU Faculties and Centres: | DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing Research Institutes and Centres > INSIGHT Centre for Data Analytics Research Institutes and Centres > FUJO. Institute for Future Media, Democracy and Society |
Publisher: | IEEE |
Official URL: | https://dx.doi.org/10.1109/OJSP.2021.3089439 |
Copyright Information: | © 2021 IEEE. Open access (CC-BY-4.0) |
Funders: | European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska Curie grant agreement No 765140, Science Foundation Ireland |
ID Code: | 26005 |
Deposited On: | 18 Jun 2021 10:45 by Alan Smeaton . Last Modified 05 Jan 2022 13:39 |
Documents
Full text available as:
Preview |
PDF
- Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
2MB |
Metrics
Altmetric Badge
Dimensions Badge
Downloads
Downloads
Downloads per month over past year
Archive Staff Only: edit this record