We address the question of estimating Kullback-Leibler losses rather than squared losses in recovery problems where the noise is distributed within the exponential family. We exhibit conditions under which these losses can be unbiasedly estimated or estimated with a controlled bias. Simulations on parameter selection problems in image denoising applications with Gamma and Poisson noises illustrate the interest of Kullback-Leibler losses and the proposed estimators.
Preprint available here
Several imaging systems provide large amount of images with complex degradation models and low signal to noise ratio. Specific adapted restoration methods should be developed. With the computing power currently available, new paradigms emerge, such as the “non-local” one (Buades et al., 2005), with very good performance (see, e.g., Lebrun et al. 2012; Milanfar, 2013). However, extensions of this methods to specific image modality might not be trivial. We focus on extensions of such methods for the restoration of non-conventional image modalities (low-light imagery, coherent imagery, tomography, …): presence of complex degradation models (blur, missing data, non-Gaussian, non-stationary and correlated noise) and requirement of fast computation large volume of data.
Parameter tuning is a critical point in image restoration techniques. When degraded data are simulated form a reference image, we can compare the restored image to the reference one, and then select the set of parameters providing the best restoration quality. This tuning is less trivial in real acquisition setting for which there are no reference images. In the case of simple degradation models, statistical tools can be used to estimate the square restoration error even though the reference image is unknown, we speak about “risk estimation”. To optimize this estimate with respect to the parameters of the method leads to a near optimal calibration. The Stein’s unbiased risk estimator (SURE, Stein 1981) is one of the most famous example, successfully applied to calibrate image restoration methods under Gaussian noise degradation (see e.g., Ramani et al., 2008). We focus in developing new estimators derived from the SURE for the calibration of parametres involved in recent methods, potentially highly parameterized, for the restoration of images with complex degradation models (blur, missing data, non-Gaussian, non-stationary and correlated noise).
See:
– Stein Unbiased GrAdient estimator of the Risk
– Stein Consistent Risk Estimator for hard thresholding
– Local Behavior of Sparse Analysis Regularization
Algorithms to solve variational regularization of ill-posed inverse problems usually involve operators that depend on a collection of continuous parameters. When these operators enjoy some (local) regularity, these parameters can be selected using the so-called Stein Unbiased Risk Estimate (SURE). While this selection is usually performed by exhaustive search, we address in this work the problem of using the SURE to efficiently optimize for a collection of continuous parameters of the model. When considering non-smooth regularizers, such as the popular l1-norm corresponding to soft-thresholding mapping, the SURE is a discontinuous function of the parameters preventing the use of gradient descent optimization techniques. Instead, we focus on an approximation of the SURE based on finite differences as proposed in (Ramani et al., 2008). Under mild assumptions on the estimation mapping, we show that this approximation is a weakly differentiable function of the parameters and its weak gradient, coined the Stein Unbiased GrAdient estimator of the Risk (SUGAR), provides an asymptotically (with respect to the data dimension) unbiased estimate of the gradient of the risk. Moreover, in the particular case of soft-thresholding, the SUGAR is proved to be also a consistent estimator. The SUGAR can then be used as a basis to perform a quasi-Newton optimization. The computation of the SUGAR relies on the closed-form (weak) differentiation of the non-smooth function. We provide its expression for a large class of iterative proximal splitting methods and apply our strategy to regularizations involving non-smooth convex structured penalties. Illustrations on various image restoration and matrix completion problems are given.
Charles-Alban Deledalle, Samuel Vaiter, Gabriel Peyré and Jalal Fadili
Stein Unbiased GrAdient estimator of the Risk (SUGAR) for multiple parameter selection,
Technical report HAL, hal-00987295 (HAL)
MATLAB source codes available from GitHub.
NL-SAR (2013-2014)
Open-source software distributed under CeCILL license to perform adaptive non-local (Pol)(In)SAR filtering. Interface in command line, IDL, Matlab, Python and C dynamic library.
Download here
Open-source software distributed under CeCILL license to generate high quality LaTeX documents of any kind such as articles, letters, reports, theses, presentations or posters.
Download here