Byeongdoo Choi

Video compression enables the transmission of video content at low rates and high qualities to our customers. In this paper, we consider the problem of embedding a neural network directly into a video decoder. This requires a design capable of operating at latencies low enough to decode tens to hundreds of high-resolution images per second. And, additionally, a network with a complexity suitable for implementation on mobile and power constrained devices.

Convolutional neural networks (CNNs) have shown promising improvements in video coding efficiency when included in traditional block-based codecs as a loop filter. Unfortunately, these coding gains are often accompanied by significant increases in complexity, measured by the number of multiply-accumulate (MAC) operations, that make them intractable in practice. As a result, there is considerable interest in reducing complexity for these CNN-based approaches.

Super-resolution video coding describes the process of coding video at lower resolution and upsampling the result. This process is included in the AV1 standard, which ensures the same super-resolution process is employed on all receiving devices. Regrettably, the design is limited to horizontal scaling with a maximum scale factor of two. In this paper, we analyze the benefit of enabling two-dimensional upsampling with larger scale factors.