毕业设计&课设-用于图像恢复的深度CNN去噪先验学习（CVPR，2017）（Matlab）.zip

# [Learning Deep CNN Denoiser Prior for Image Restoration](http://www4.comp.polyu.edu.hk/~cslzhang/paper/IRCNN_CVPR17.pdf)  # PyTorch version [https://github.com/cszn/DPIR](https://github.com/cszn/DPIR) # Abstract Model-based optimization methods and discriminative learning methods have been the two dominant strategies for solving various inverse problems in low-level vision. Typically, those two kinds of methods have their respective merits and drawbacks, e.g., model-based optimization methods are flexible for handling different inverse problems but are usually time-consuming with sophisticated priors for the purpose of good performance; in the meanwhile, discriminative learning methods have fast testing speed but their application range is greatly restricted by the specialized task. Recent works have revealed that, with the aid of variable splitting techniques, denoiser prior can be plugged in as a modular part of model-based optimization methods to solve other inverse problems (e.g., deblurring). Such an integration induces considerable advantage when the denoiser is obtained via discriminative learning. However, the study of integration with fast discriminative denoiser prior is still lacking. To this end, this paper aims to train a set of fast and effective CNN (convolutional neural network) denoisers and integrate them into model-based optimization method to solve other inverse problems. Experimental results demonstrate that the learned set of denoisers can not only achieve promising Gaussian denoising results but also can be used as prior to deliver good performance for various low-level vision applications. # Basic Idea With the aid of variable splitting techniques, such as alternating direction method of multipliers (ADMM) method and half quadratic splitting (HQS) method, it is possible to deal with fidelity term and regularization term of general image restoration formulation separately, and particularly, the regularization term only corresponds to a denoising subproblem. Consequently, this enables an integration of any discriminative denoisers into model-based optimization methods to solve various image restoration tasks, such as - **Image Deblurring** - **Image Inpainting** - **Single Image Super-Resolution** - **Color Image Demosaicking** There is no additional training for these tasks. # Half-Quadratic Splitting (HQS) Algorithm <img src="figs/HQS.png" width="600px"/> # Image Deblurring The left is the blurred image. The right is the deblurred image by IRCNN with estimated kernels by other blind deblurring methods. - Deblur_set1 <img src="testsets/Deblur_set1/Blurry4_1.png" width="350px"/> <img src="testsets/Deblur_set1/Blurry4_1_output_ircnn.png" width="350px"/> <img src="testsets/Deblur_set1/Blurry4_1_kernel.png" width="60px"/> <img src="testsets/Deblur_set1/Blurry2_3.png" width="350px"/> <img src="testsets/Deblur_set1/Blurry2_3_output_ircnn.png" width="350px"/> <img src="testsets/Deblur_set1/Blurry2_3_kernel.png" width="60px"/> - Deblur_set2 <img src="testsets/Deblur_set2/im01_ker02.png" width="350px"/> <img src="testsets/Deblur_set2/im01_ker02_ircnn.png" width="350px"/> <img src="testsets/Deblur_set2/im01_ker02_out_kernel.png" width="60px"/> <img src="testsets/Deblur_set2/im04_ker01.png" width="350px"/> <img src="testsets/Deblur_set2/im04_ker01_ircnn.png" width="350px"/> <img src="testsets/Deblur_set2/im04_ker01_out_kernel.png" width="60px"/> - Deblur_set3 <img src="testsets/Deblur_set3/2013622235456945_blur_79.png" width="350px"/> <img src="testsets/Deblur_set3/2013622235456945_blur_79_ircnn_Isigma_1_Msigma_1.png" width="350px"/> <img src="testsets/Deblur_set3/2013622235456945_blur_79_kernel.png" width="60px"/> <img src="testsets/Deblur_set3/color_patch_blur_99.png" width="350px"/> <img src="testsets/Deblur_set3/color_patch_blur_99_ircnn_Isigma_1_Msigma_1.png" width="350px"/> <img src="testsets/Deblur_set3/color_patch_blur_99_kernel.png" width="60px"/> <img src="testsets/Deblur_set3/IMG_1240_blur.png" width="350px"/> <img src="testsets/Deblur_set3/IMG_1240_blur_ircnn_Isigma_2_Msigma_7.png" width="350px"/> <img src="testsets/Deblur_set3/IMG_1240_blur_kernel.png" width="60px"/> Use [Demo_deblur_real_application.m](Demo_deblur_real_application.m) to test IRCNN for image deblurring with estimated kernel by other blind deblurring methods. # Image Inpainting The left is the masked image. The right is the recovered image by IRCNN. - Inpaint_set1 <img src="testsets/Inpaint_set1/butterfly_color_80_masked.png" width="350px"/> <img src="testsets/Inpaint_set1/butterfly_color_80_ircnn.png" width="350px"/> <img src="testsets/Inpaint_set1/butterfly_gray_75_masked.png" width="350px"/> <img src="testsets/Inpaint_set1/butterfly_gray_75_ircnn.png" width="350px"/> <img src="testsets/Inpaint_set1/09_50_masked.png" width="350px"/> <img src="testsets/Inpaint_set1/09_50_ircnn.png" width="350px"/> - Inpaint_set2 <img src="testsets/Inpaint_set2/3ch.png" width="350px"/> <img src="testsets/Inpaint_set2/3ch_ircnn.png" width="350px"/> <img src="testsets/Inpaint_set2/3ch_mask.png" width="60px"/> <img src="testsets/Inpaint_set2/new.png" width="350px"/> <img src="testsets/Inpaint_set2/new_ircnn.png" width="350px"/> <img src="testsets/Inpaint_set2/new_mask.png" width="60px"/> Use [Demo_inpaint.m](./Demo_inpaint.m) and [Demo_inpaint_real_application.m](Demo_inpaint_real_application.m) to produce the above results. # Single Image Super-Resolution (SISR) #### IRCNN considers three degradation models for SISR. #### 1. Bicubic degradation ```matlab y = imresize(x, 1/sf, 'bicubic') % y: LR image; x: clean HR image; sf: scale factor ``` IRCNN for bicubic degradation has a comparable performance to VDSR and DnCNN. Use [Demo_SISR_YCbCr.m](./Demo_SISR_YCbCr.m) and [Demo_SISR_RGB.m](./Demo_SISR_RGB.m) to produce the SISR results. #### 2. Blurring by Gaussian kernel followed by downsampling For scale factor 3, ```matlab kernelsize = ceil(sigma*3)*2+1; % or 7 for sigma = 1.6 kernel     = fspecial('gaussian',kernelsize,sigma); blur_HR = imfilter(im,kernel,'replicate') LR = imresize(blur_HR, 1/scale, 'nearest'); ``` For scale factors 2 and 4, ```matlab kernelsize = ceil(sigma*3)*2+2; kernel = fspecial('gaussian',kernelsize,sigma); blur_HR = imfilter(im, kernel,'replicate'); LR = blur_HR(scale/2:scale:end-scale/2,scale/2:scale:end-scale/2,:); ``` Use [Demo_SISR_YCbCr.m](./Demo_SISR_YCbCr.m) and [Demo_SISR_RGB.m](./Demo_SISR_RGB.m) to produce the SISR results. #### 3. Blurring by ***arbitrary*** kernel followed by standard K-fold downsampler (matlab function '***downsample***') ```matlab blur_HR   = imfilter(im, kernel,'replicate'); % kernel can be arbitrary, such as motion blur, Gaussian blur. for i = 1:size(x,3) LR(:,:,i) = downsample(downsample(blur_HR(:,:,i),sf)',sf)'; end ``` More details on this degradation model can be found in the following paper. [1] Chan, Stanley H., Xiran Wang, and Omar A. Elgendy. "Plug-and-Play ADMM for image restoration: Fixed-point convergence and applications." IEEE Transactions on Computational Imaging 3.1 (2017): 84-98. The left is the zoomed LR image (x3) with motion blur kernel, the right is the super-resolved image (x3) by IRCNN. <img src="figs/LR1.png" width="350px"/> <img src="figs/LR1_IRCNN.png" width="350px"/> The left is the low-resolution (LR) image. The right is the super-resolved image by IRCNN. - SISR_set1, synthetic LR image SR <img src="testsets/SISR_set1/LR3_noisy_x2.png" width="160px"/> <img src="testsets/SISR_set1/LR3_noisy_x2_ircnn_x2.png" width="320px"/> <img src="testsets/SISR_set1/LR1_motion_x2.png" width="240px"/> <img src="testsets/SISR_set1/LR1_motion_x2_ircnn_x2.png" width="480px"/> - SISR_set2, real LR image SR <img src="testsets/SISR_set2/chip.png" width="109px"/> <img src="testsets/SISR_set2/chip_ircnn_x2.png" width="218px"/> <img src="testsets/SISR_set2/ch