Login (DCU Staff Only)
Login (DCU Staff Only)

DORAS | DCU Research Repository

Explore open access research and scholarly works from DCU

Advanced Search

Improved CNN-based Learning of interpolation filters for low-complexity Inter prediction in video coding

Murn, Luka orcid logoORCID: 0000-0001-9041-647X, Blasi, Saverio, Smeaton, Alan F. orcid logoORCID: 0000-0003-1028-8389 and Mrak, Marta (2021) Improved CNN-based Learning of interpolation filters for low-complexity Inter prediction in video coding. IEEE Open Journal of Signal Processing, 2 . pp. 453-465. ISSN 2644-1322

Abstract
The versatility of recent machine learning approaches makes them ideal for improvement of next generation video compression solutions. Unfortunately, these approaches typically bring significant increases in computational complexity and are difficult to interpret into explainable models, affecting their potential for implementation within practical video coding applications. This paper introduces a novel explainable neural network-based inter-prediction scheme, to improve the interpolation of reference samples needed for fractional precision motion compensation. The approach requires a single neural network to be trained from which a full quarter-pixel interpolation filter set is derived, as the network is easily interpretable due to its linear structure. A novel training framework enables each network branch to resemble a specific fractional shift. This practical solution makes it very efficient to use alongside conventional video coding schemes. When implemented in the context of the state-of-the-art Versatile Video Coding (VVC) test model, 0.77%, 1.27% and 2.25% BD-rate savings can be achieved on average for lower resolution sequences under the random access, low-delay B and low-delay P configurations, respectively, while the complexity of the learned interpolation schemes is significantly reduced compared to the interpolation with full CNNs.
Metadata
Item Type:Article (Published)
Refereed:Yes
Uncontrolled Keywords:video compression; motion compensation; interpolation; machine learning; complexity reduction
Subjects:Computer Science > Artificial intelligence
Computer Science > Image processing
Computer Science > Machine learning
Computer Science > Video compression
DCU Faculties and Centres:DCU Faculties and Schools > Faculty of Engineering and Computing > School of Computing
Research Institutes and Centres > INSIGHT Centre for Data Analytics
Research Institutes and Centres > FUJO. Institute for Future Media, Democracy and Society
Publisher:IEEE
Official URL:https://dx.doi.org/10.1109/OJSP.2021.3089439
Copyright Information:© 2021 IEEE. Open access (CC-BY-4.0)
Funders:European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska Curie grant agreement No 765140, Science Foundation Ireland
ID Code:26005
Deposited On:18 Jun 2021 10:45 by Alan Smeaton . Last Modified 05 Jan 2022 13:39
Documents

Full text available as:

[thumbnail of Final_NoAnnotations_OJSPSubmission_BBC_NNFractionalInterpolation.pdf]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
2MB
Metrics

Altmetric Badge

Dimensions Badge

Downloads

Downloads

Downloads per month over past year

Archive Staff Only: edit this record