DnCNN-matlab.zip

# [DnCNN](http://www4.comp.polyu.edu.hk/~cslzhang/paper/DnCNN.pdf) # [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](http://ieeexplore.ieee.org/document/7839189/)  **News:** [DRUNet](https://github.com/cszn/DPIR) * State-of-the-art denoising performance * Can be used for plug-and-play image restoration * <img src="https://github.com/cszn/DPIR/raw/master/figs/grayscale_psnr.png" width="550px"/> * [https://github.com/cszn/DPIR/blob/master/main_dpir_denoising.py](https://github.com/cszn/DPIR/blob/master/main_dpir_denoising.py) # [PyTorch training and testing code](https://github.com/cszn/KAIR) - 18/12/2019 `I recommend to use the PyTorch code for training and testing. The model parameters of MatConvnet and PyTorch are same.` - [main_train_dncnn.py](https://github.com/cszn/KAIR/blob/master/main_train_dncnn.py) - [main_test_dncnn.py](https://github.com/cszn/KAIR/blob/master/main_test_dncnn.py) - [main_test_dncnn3_deblocking.py](https://github.com/cszn/KAIR/blob/master/main_test_dncnn3_deblocking.py) ## Merge batch normalization (PyTorch) ```python import torch import torch.nn as nn def merge_bn(model): ''' merge all 'Conv+BN' (or 'TConv+BN') into 'Conv' (or 'TConv') based on https://github.com/pytorch/pytorch/pull/901 by Kai Zhang (cskaizhang@gmail.com) https://github.com/cszn/DnCNN 01/01/2019 ''' prev_m = None for k, m in list(model.named_children()): if (isinstance(m, nn.BatchNorm2d) or isinstance(m, nn.BatchNorm1d)) and (isinstance(prev_m, nn.Conv2d) or isinstance(prev_m, nn.Linear) or isinstance(prev_m, nn.ConvTranspose2d)): w = prev_m.weight.data if prev_m.bias is None: zeros = torch.Tensor(prev_m.out_channels).zero_().type(w.type()) prev_m.bias = nn.Parameter(zeros) b = prev_m.bias.data invstd = m.running_var.clone().add_(m.eps).pow_(-0.5) if isinstance(prev_m, nn.ConvTranspose2d): w.mul_(invstd.view(1, w.size(1), 1, 1).expand_as(w)) else: w.mul_(invstd.view(w.size(0), 1, 1, 1).expand_as(w)) b.add_(-m.running_mean).mul_(invstd) if m.affine: if isinstance(prev_m, nn.ConvTranspose2d): w.mul_(m.weight.data.view(1, w.size(1), 1, 1).expand_as(w)) else: w.mul_(m.weight.data.view(w.size(0), 1, 1, 1).expand_as(w)) b.mul_(m.weight.data).add_(m.bias.data) del model._modules[k] prev_m = m merge_bn(m) def tidy_sequential(model): for k, m in list(model.named_children()): if isinstance(m, nn.Sequential): if m.__len__() == 1: model._modules[k] = m.__getitem__(0) tidy_sequential(m) ``` # Training ([MatConvNet](http://www.vlfeat.org/matconvnet/)) - [Simplenn](http://www.vlfeat.org/matconvnet/wrappers/) version * [DnCNN_TrainingCodes_v1.1](https://github.com/cszn/DnCNN/tree/master/TrainingCodes/DnCNN_TrainingCodes_v1.1) - [DagNN](http://www.vlfeat.org/matconvnet/wrappers/) version * [DnCNN_TrainingCodes_DagNN_v1.1](https://github.com/cszn/DnCNN/tree/master/TrainingCodes/DnCNN_TrainingCodes_DagNN_v1.1) # Testing ([MatConvNet](http://www.vlfeat.org/matconvnet/) or Matlab) - [demos] `Demo_test_DnCNN-.m`. - [models] including the trained models for Gaussian denoising; a single model for Gaussian denoising, single image super-resolution (SISR) and deblocking. - [testsets] BSD68 and Set10 for Gaussian denoising evaluation; Set5, Set14, BSD100 and Urban100 datasets for SISR evaluation; Classic5 and LIVE1 for JPEG image deblocking evaluation. # New FDnCNN Models I have trained new Flexible DnCNN (FDnCNN) models based on [FFDNet](https://github.com/cszn/FFDNet). FDnCNN can handle noise level range of [0, 75] via a single model. [Demo_FDnCNN_Gray.m](Demo_FDnCNN_Gray.m) [Demo_FDnCNN_Gray_Clip.m](Demo_FDnCNN_Gray_Clip.m) [Demo_FDnCNN_Color.m](Demo_FDnCNN_Color.m) [Demo_FDnCNN_Color_Clip.m](Demo_FDnCNN_Color_Clip.m) # Network Architecture and Design Rationale - Network Architecture <img src="figs/dncnn.png" width="800px"/> - Batch normalization and residual learning are beneficial to Gaussian denoising (especially for a single noise level). The residual of a noisy image corrupted by additive white Gaussian noise (AWGN) follows a constant Gaussian distribution which stablizes batch normalization during training. * Histogram of noisy patches, clean patches, and residual (noise) patches from a batch of training. The noise level is 25, the patch size is 40x40, the batch size is 128. <img src="figs/batch1.png" width="800px"/> * Histogram of noisy patches, clean patches, and residual (noise) patches from another batch of training. The noise level is 25, the patch size is 40x40, the batch size is 128. <img src="figs/batch2.png" width="800px"/> * Noise-free image super-resolution does not have this property. - Predicting the residual can be interpreted as performing one gradient descent inference step at starting point (i.e., noisy image). * The parameters in DnCNN are mainly representing the image priors (task-independent), thus it is possible to learn a single model for different tasks, such as image denoising, image super-resolution and JPEG image deblocking. * The left is the input image corrupted by different degradations, the right is the restored image by DnCNN-3. <img src="figs/input.png" width="350px"/> <img src="figs/output.png" width="350px"/> # Results ## Gaussian Denoising **The average PSNR(dB) results of different methods on the BSD68 dataset.** | Noise Level | BM3D | WNNM | EPLL | MLP | CSF |TNRD | DnCNN | DnCNN-B | FDnCNN | DRUNet | |:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:|:-------:| | 15 | 31.07 | 31.37 | 31.21 | - | 31.24 | 31.42 | **31.73** | **31.61** |**31.69** | 31.91 | | 25 | 28.57 | 28.83 | 28.68 | 28.96 | 28.74 | 28.92 | **29.23** | **29.16** |**29.22** | 29.48 | | 50 | 25.62 | 25.87 | 25.67 | 26.03 | - | 25.97 | **26.23** | **26.23** | **26.27** | 26.59 | **Visual Results** The left is the noisy image corrupted by AWGN, the middle is the denoised image by DnCNN, the right is the ground-truth. <img src="figs/05_25_noisy.png" width="250px"/> <img src="figs/05_25.png" width="250px"/> <img src="testsets/Set12/05.png" width="250px"/> <img src="figs/02_25_noisy.png" width="250px"/> <img src="figs/02_25.png" width="250px"/> <img src="testsets/Set12/02.png" width="250px"/> <img src="figs/102061_noisy.png" width="250px"/> <img src="figs/102061_dncnn.png" width="250px"/> <img src="figs/102061.png" width="250px"/> ## Gaussian Denoising, Single ImageSuper-Resolution and JPEG Image Deblocking via a Single (DnCNN-3) Model **Average PSNR(dB)/SSIM results of different methods for Gaussian denoising with noise level 15, 25 and 50 on BSD68 dataset, single image super-resolution with upscaling factors 2, 3 and 40 on Set5, Set14, BSD100 and Urban100 datasets, JPEG image deblocking with quality factors 10, 20, 30 and 40 on Classic5 and LIVE11 datasets.** ### Gaussian Denoising | Dataset | Noise Level | BM3D | TNRD | DnCNN-3 | |:---------:|:---------:|:---------:|:---------:|:---------:| | | 15 | 31.08 / 0.8722 | 31.42 / 0.8826 | 31.46 / 0.8826 | | BSD68 | 25 | 28.57 / 0.8017 | 28.92 / 0.8157 | 29.02 / 0.8190 | | | 50 | 25.62 / 0.6869 | 25.97 / 0.7029 | 26.10 / 0.7076 | ### Single Image Super-Resolution | Dataset | Upscaling Factor | TNRD | VDSR |DnCNN-3| |:---------:|:---------:|:---------:|:---------:|:---------:| | | 2 | 36.86 / 0.9556 | 37.56 / 0.9591 | 37.58 / 0.9590 | |Set5 | 3 | 33.18 / 0.9152 | 33.67 / 0.9220 | 33.75 / 0.