When a digital photograph is taken, the camera sensor records the light in the scene very differently from the way the human visual system interprets it. An image is not just passively formed, but instead constructed through a complex computational processing chain that converts the RAW sensor measurements into a color depiction a person can understand and appreciate. No matter the capture conditions, the sensing process introduces degradations that need to be corrected during this image formation chain. Among them, the most fundamental is the random noise caused by the incidence of photons and their conversion to digital values in the electronic circuits of the sensor. In this thesis, we present three new algorithmic approaches to remove such noise, operating directly on the RAW data before the image has been fully materialized.

Closely related to denoising is the problem of image fusion. A camera sensor’s photodiodes have a limited dynamic range compared to the luminance of many real-world scenes. Increasing the exposure time allows more light to be captured, revealing details in the dark but risking blur and saturation of bright areas. The ISO setting also adjusts image brightness, but higher values magnify the noise. A classical solution to extend the effective dynamic range is to fuse multiple exposures of the same scene to incorporate both dark and bright details in a single high dynamic range image. Still, small movements between shots and the inherent noise within them can produce ghosting artifacts and hinder the fusion. To address this, we propose to perform image fusion and denoising jointly, using a spatio-temporal denoiser that integrates a brightness- based weighting function to fuse multi-exposure RAW image sequences. The solution aligns a sequence after normalizing its exposures and stabilizing their noise variances, and collaboratively filters sets of similar volumetric patches with a weighted principal component analysis, producing clean fused images while suppressing ghosting.

Denoising is also interconnected with demosaicking, the process through which missing colors at the sensor—which only reads a single color channel at each position—are interpolated. Placing the denoiser early in the image processing chain preserves the noise model at the sensor, but is susceptible to aliasing. Denoising after demosaicking modifies the noise statistics, but richer color information can be used to reconstruct the image. We study the effects of both alternatives, and propose a modular patch-based spatio-temporal video denoiser that balances the usage of pre- and post-demosaicking stages by leaving a controlled fraction of noise to be removed after demosaicking. We find that correctly weighting the two stages improves image quality, with higher ISO values favoring a stronger pre-demosaicking stage.

The patch-based denoisers used throughout the thesis are built to exploit nonlocal self-similarity and are competitive in quality with deep-learning-based approaches, while adapting to any noise model at lower computational cost. As a final study, we investigate how to translate this classical collaborative nonlocal filtering paradigm into a lightweight neural network while maintaining high denoising speeds and noise model agnosticism. For that, we build a neural block that follows a three-step structure of neighbor matching, collaborative filtering in a learned domain, and aggregation, which operates on latent image features and is trainable end-to-end. The proposed block achieves higher image quality than other lightweight networks, and embedding it in a shallow multiscale scheme makes it competitive with resource-intensive alternatives. The resulting model, trained with synthetic noise on curated sensor data, demonstrates high generalization to unseen noise models. Together, the proposed models show that nonlocal filtering in the RAW domain offers efficient, practical solutions for real-world photography and videography.