Antoine Monod

Antoine Monod

Multi-image restoration for computational photography and videography

Quand

5 mai 2023    
15h30 - 16h30

Salle du Conseil, Espace Turing
45 rue des Saints-Pères, Paris, 75006

Type d’évènement

This thesis explores multi-image approaches for image as well as
video restoration.

Image restoration using multiple images is studied through real raw
burst denoising, where multiple images corrupted with real noise
caused by the photographic acquisition process are combined to produce
a single image with less noise. A popular classical algorithmic
approach of the raw burst denoising literature is thoroughly
explained, analyzed and reimplemented in an open source fashion.
Given the increasing ubiquity of deep neural networks and their
state-of-the-art performance for image and video restoration over the
last half-decade, a second learning-based and data-driven approach
is proposed for multi-frame raw burst denoising. Comparing the two
approaches clearly shows the appeal of properly designed convolutional
neural networks for this kind of task.

Video restoration can also leverage mutual information of multiple
images to produce more appealing results. This topic is explored here
via a Deep Plug-and-Play (PnP) method. Such methods consist in
plugging a denoising deep neural network in an optimization scheme
used to solve inverse problems, e.g. using the denoising network as a
replacement for the proximal operator of the data prior under a
Bayesian formalism. While Plug-and-Play methods have extensively been
studied for image restoration, their use in video restoration is
relatively uncharted territory, and is a key focus of this thesis. We
present a novel method for restoring digital videos via a Deep
Plug-and-Play approach. With it, a network trained once for denoising
can be repurposed for multiple different video restoration tasks such
as video deblurring, super-resolution, demosaicking and interpolation
of random missing pixels. Our experiments all show a clear benefit to
using a network specifically designed for video denoising, as it
yields better restoration performance and better temporal stability
than a single image network with similar denoising performance using
the same PnP formulation. Said method compares favorably to applying
other state-of-the-art PnP schemes separately on each frame of the
sequence, opening new perspectives in the field of video restoration.

Vous aimerez aussi...