DIRECT: Discrete Image Rescaling with Enhancement from Case-specific Textures
Abstract
This paper addresses image rescaling, the task of which is to downscale an input image followed by upscaling for the purposes of transmission, storage, or playback on heterogeneous devices. The state-of-the-art image rescaling network (known as IRN) tackles image downscaling and upscaling as mutually invertible tasks using invertible affine coupling layers. In particular, for upscaling, IRN models the missing high-frequency component by an input-independent (case-agnostic) Gaussian noise. In this work, we take one step further to predict a case-specific high-frequency component from textures embedded in the downscaled image. Moreover, we adopt integer coupling layers to avoid quantizing the downscaled image. When tested on commonly used datasets, the proposed method, termed DIRECT, improves high-resolution reconstruction quality both subjectively and objectively, while maintaining visually pleasing downscaled images.
Contributions
- To the best of our knowledge, DIRECT is the first attempt to model image rescaling in a discrete manner.
- DIRECT decomposes an HR image into an LR image and the HFInfo in which we can explicitly model the one-to-many mapping between the missing HFInfo and the information-embedded LR image.
Proposed Method
Extended from IRN, our proposed method aims to alleviate the quantization issue and introduce a case-specific HFInfo. First, the input HR image goes through the discrete invertible rescaling network (DRN) to acquire its LR image. Pixel values of LR image are clipped according to the specified bit-depth. Conditioned on the LR image, the case-specific HFInfo is then extracted by CTM. Finally, the upscaling process is conducted by the inverse operation of DRN.
Experimental Results
Qualitative result comparison for HR images (zoom in for better view)
Comparison in terms of PSNR-Y/SSIM-Y on 6 datasets for x4 upscaling. Numbers in red and blue indicate the best and the second-best performance, respectively.
Conclusion
This work proposes a Discrete Invertible Rescaling network, termed DIRECT, for image rescaling. DIRECT leverages the case-specific information embedded in the LR image to predict HFInfo for better HR reconstruction. It also relieves the degradation caused by quantizing LR images. DIRECT consists of two sub-modules: CTM and DRN. CTM provides an efficient way to exploit the embedded information for rescaling, and DRN uses discretization to alleviate the quantization effect. Experimental results show that DIRECT outperforms the state-of-the-art methods both in PSNR-Y and SSIM-Y. Being tested in various public datasets, DIRECT achieves a 0.2∼0.7dB average gain in PSNR-Y. Moreover, the reconstructed HR images also have better subjective qualities.