This thesis explores multi-image approaches for image as well as video restoration.
Image restoration using multiple images is studied through real raw burst denoising, where multiple images corrupted with real noise caused by the photographic acquisition process are combined to produce a single image with less noise. A popular classical algorithmic approach of the raw burst denoising literature is thoroughly explained, analyzed and reimplemented in an open source fashion.
Given the increasing ubiquity of deep neural networks and their state-of-the-art performance for image and video restoration over the last half-decade, a second learning-based and data-driven approach is proposed for multi-frame raw burst denoising. Comparing the two approaches clearly shows the appeal of properly designed convolutional neural networks for this kind of task.
Video restoration can also leverage mutual information of multiple images to produce more appealing results. This topic is explored here via a Deep Plug-and-Play (PnP) method. Such methods consist in plugging a denoising deep neural network in an optimization scheme used to solve inverse problems, e.g. using the denoising network as a replacement for the proximal operator of the data prior under a Bayesian formalism. While Plug-and-Play methods have extensively been studied for image restoration, their use in video restoration is relatively uncharted territory, and is a key focus of this thesis. We present a novel method for restoring digital videos via a Deep Plug-and-Play approach.
With it, a network trained once for denoising can be repurposed for multiple different video restoration tasks such as video deblurring, super-resolution, demosaicking and interpolation of random missing pixels. Our experiments all show a clear benefit to using a network specifically designed for video denoising, as it yields better restoration performance and better temporal stability than a single image network with similar denoising performance using the same PnP formulation. Said method compares favorably to applying other state-of-the-art PnP schemes separately on each frame of the sequence, opening new perspectives in the field of video restoration.