Skip to content

Latest commit

 

History

History
149 lines (119 loc) · 15.2 KB

readme.md

File metadata and controls

149 lines (119 loc) · 15.2 KB
Raw

PWC PWC PWC PWC PWC PWC PWC PWC PWC PWC

NAFNet: Nonlinear Activation Free Network for Image Restoration

The official pytorch implementation of the paper Simple Baselines for Image Restoration (ECCV2022)

Liangyu Chen*, Xiaojie Chu*, Xiangyu Zhang, Jian Sun

Although there have been significant advances in the field of image restoration recently, the system complexity of the state-of-the-art (SOTA) methods is increasing as well, which may hinder the convenient analysis and comparison of methods. In this paper, we propose a simple baseline that exceeds the SOTA methods and is computationally efficient. To further simplify the baseline, we reveal that the nonlinear activation functions, e.g. Sigmoid, ReLU, GELU, Softmax, etc. are not necessary: they could be replaced by multiplication or removed. Thus, we derive a Nonlinear Activation Free Network, namely NAFNet, from the baseline. SOTA results are achieved on various challenging benchmarks, e.g. 33.69 dB PSNR on GoPro (for image deblurring), exceeding the previous SOTA 0.38 dB with only 8.4% of its computational costs; 40.30 dB PSNR on SIDD (for image denoising), exceeding the previous SOTA 0.28 dB with less than half of its computational costs.

NAFNet For Image Denoise NAFNet For Image Deblur NAFSSR For Stereo Image Super Resolution
Denoise Deblur StereoSR(NAFSSR)

PSNR_vs_MACs

News

2022.08.02 The Baseline, including the pretrained models and train/test configs, are available now.

2022.07.03 Related work, Improving Image Restoration by Revisiting Global Information Aggregation (TLC, a.k.a TLSC in our paper) is accepted by ECCV2022 🎉 . Code is available at https://github.com/megvii-research/TLC.

2022.07.03 Our paper is accepted by ECCV2022 🎉

2022.06.19 NAFSSR (as a challenge winner) is selected for an ORAL presentation at CVPR 2022, NTIRE workshop 🎉 Presentation video, slides and poster are available now.

2022.04.15 NAFNet based Stereo Image Super-Resolution solution (NAFSSR) won the 1st place on the NTIRE 2022 Stereo Image Super-resolution Challenge! Training/Evaluation instructions see here.

Installation

This implementation based on BasicSR which is a open source toolbox for image/video restoration tasks and HINet

python 3.9.5
pytorch 1.11.0
cuda 11.3
git clone https://github.com/megvii-research/NAFNet
cd NAFNet
pip install -r requirements.txt
python setup.py develop --no_cuda_ext

Quick Start

  • Image Denoise Colab Demo: google colab logo
  • Image Deblur Colab Demo: google colab logo
  • Stereo Image Super-Resolution Colab Demo: google colab logo
  • Single Image Inference Demo:
    • Image Denoise:
    python basicsr/demo.py -opt options/test/SIDD/NAFNet-width64.yml --input_path ./demo/noisy.png --output_path ./demo/denoise_img.png
    • Image Deblur:
    python basicsr/demo.py -opt options/test/REDS/NAFNet-width64.yml --input_path ./demo/blurry.jpg --output_path ./demo/deblur_img.png
    • --input_path: the path of the degraded image
    • --output_path: the path to save the predicted image
    • pretrained models should be downloaded.
    • Integrated into Huggingface Spaces 🤗 using Gradio. Try out the Web Demo for single image restorationHugging Face Spaces
  • Stereo Image Inference Demo:
    • Stereo Image Super-resolution:
    python basicsr/demo_ssr.py -opt options/test/NAFSSR/NAFSSR-L_4x.yml \
--input_l_path ./demo/lr_img_l.png --input_r_path ./demo/lr_img_r.png \
--output_l_path ./demo/sr_img_l.png --output_r_path ./demo/sr_img_r.png
    • --input_l_path: the path of the degraded left image
    • --input_r_path: the path of the degraded right image
    • --output_l_path: the path to save the predicted left image
    • --output_r_path: the path to save the predicted right image
    • pretrained models should be downloaded.
    • Integrated into Huggingface Spaces 🤗 using Gradio. Try out the Web Demo for stereo image super-resolutionHugging Face Spaces
  • Try the web demo with all three tasks here: Replicate

Results and Pre-trained Models

name Dataset PSNR SSIM pretrained models configs
NAFNet-GoPro-width32 GoPro 32.8705 0.9606 gdrive | 百度网盘 train | test
NAFNet-GoPro-width64 GoPro 33.7103 0.9668 gdrive | 百度网盘 train | test
NAFNet-SIDD-width32 SIDD 39.9672 0.9599 gdrive | 百度网盘 train | test
NAFNet-SIDD-width64 SIDD 40.3045 0.9614 gdrive | 百度网盘 train | test
NAFNet-REDS-width64 REDS 29.0903 0.8671 gdrive | 百度网盘 train | test
NAFSSR-L_4x Flickr1024 24.17 0.7589 gdrive | 百度网盘 train | test
NAFSSR-L_2x Flickr1024 29.68 0.9221 gdrive | 百度网盘 train | test
Baseline-GoPro-width32 GoPro 32.4799 0.9575 gdrive | 百度网盘 train | test
Baseline-GoPro-width64 GoPro 33.3960 0.9649 gdrive | 百度网盘 train | test
Baseline-SIDD-width32 SIDD 39.8857 0.9596 gdrive | 百度网盘 train | test
Baseline-SIDD-width64 SIDD 40.2970 0.9617 gdrive | 百度网盘 train | test

Image Restoration Tasks

Task Dataset Train/Test Instructions Visualization Results
Image Deblurring GoPro link gdrive | 百度网盘
Image Denoising SIDD link gdrive | 百度网盘
Image Deblurring with JPEG artifacts REDS link gdrive | 百度网盘
Stereo Image Super-Resolution Flickr1024+Middlebury link gdrive | 百度网盘

Citations

If NAFNet helps your research or work, please consider citing NAFNet.

@article{chen2022simple,
  title={Simple Baselines for Image Restoration},
  author={Chen, Liangyu and Chu, Xiaojie and Zhang, Xiangyu and Sun, Jian},
  journal={arXiv preprint arXiv:2204.04676},
  year={2022}
}

If NAFSSR helps your research or work, please consider citing NAFSSR.

@InProceedings{chu2022nafssr,
    author    = {Chu, Xiaojie and Chen, Liangyu and Yu, Wenqing},
    title     = {NAFSSR: Stereo Image Super-Resolution Using NAFNet},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
    month     = {June},
    year      = {2022},
    pages     = {1239-1248}
}

Contact

If you have any questions, please contact [email protected] or [email protected]

statistics

visitors