Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method

1Nanjing University, 2Australian National University, 3Shenzhen Campus of Sun Yat-sen University, 4Rakuten Institute of Technology
Interpolation end reference image.

4K sample paired images (normal-light/low-light) from our UHD-LOL4K dataset. The images are from indoor and outdoor scenes, including buildings, streets, people, natural landsca etc.

Interpolation end reference image.

8K sample paired images (normal-light/low-light) from our UHD-LOL8K dataset. The images are from indoor and outdoor scenes, including buildings, streets, people, natural landsca etc.

Abstract

As the quality of optical sensors improves, there is a need for processing large-scale images. In particular, the ability of devices to capture ultra-high definition (UHD) images and video places new demands on the image processing pipeline. In this paper, we consider the task of low-light image enhancement (LLIE) and introduce a large-scale database consisting of images at 4K and 8K resolution. We conduct systematic benchmarking studies and provide a comparison of current LLIE algorithms. As a second contribution, we introduce LLFormer, a transformer-based low-light enhancement method. The core components of LLFormer are the axis-based multi-head self-attention and cross-layer attention fusion block, which significantly reduces the linear complexity. Extensive experiments on the new dataset and existing public datasets show that LLFormer outperforms state-of-the-art methods. We also show that employing existing LLIE methods trained on our benchmark as a pre-processing step significantly improves the performance of downstream tasks, e.g., face detection in low-light conditions. The source code and pre-trained models are available at https://github.com/TaoWangzj/LLFormer.

LLFormer

Interpolation end reference image.

LLFormer architecture . The core design of LLFormer includes an axis-based transformer block and a cross-layer attention fusion block.

Benchmarking Results

Interpolation end reference image.

Dataset Structure

The UHD-LOL4K subset consists of 8099 paired 4K images, and 5999 for training and 2100 for testing.

Name Size Type Links Description
UHD-4K 94.9 G 百度网盘 Main folder
├ Train Set 25.2 G normal/low - 8099 paired normal/low-light 8K images.
├ Test Set 69.7 G normal/low - 2100 paired normal/low-light 8K images.

The UHD-LOL8K subset consists of 2966 paired 8K images, and 2029 for training and 937 for testing.

Name Size Type Links Description
UHD-8K 97.8 G 百度网盘 Main folder
├ Train Set 67.3 G normal/low - 2029 paired normal/low-light 8K images.
├ Test Set 30.5 G normal/low - 937 paired normal/low-light 8K images.

Agreement

  • The UHD-LOL dataset is only available to download for non-commercial research purposes. The copyright remains with the original owners of the image. A complete version of the license can be found here.
  • You agree not to reproduce, duplicate, copy, sell, trade, resell or exploit for any commercial purposes, any portion of the videos and any portion of derived data. You agree not to further copy, publish or distribute any portion of the UHD-LOL dataset.

BibTeX

If you find this helpful, please cite our work:

  @article{wang2022ultra,
     title={Ultra-High-Definition Low-Light Image Enhancement: A Benchmark and Transformer-Based Method},
     author={Wang, Tao and Zhang, Kaihao and Shen, Tianrun and Luo, Wenhan and Stenger, Bjorn and Lu, Tong},
     journal={arXiv preprint arXiv:2212.11548},
     year={2022}
    }

    @inproceedings{zhang2021benchmarking,
      title={Benchmarking ultra-high-definition image super-resolution},
      author={Zhang, Kaihao and Li, Dongxu and Luo, Wenhan and Ren, Wenqi and Stenger, Bjorn and Liu, Wei and Li, Hongdong and Yang, Ming-Hsuan},
      booktitle={ICCV},
      pages={14769--14778},
      year={2021}
    }