|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Photographs captured by smartphones and mid-range cameras have limited spatial resolution and dynamic range, with noisy response in underexposed regions and color artefacts in saturated areas. This paper introduces the first approach (to the best of our knowledge) to the reconstruction of high-resolution, high-dynamic range color images from raw photographic bursts captured by a handheld camera with exposure bracketing. This method uses a physically-accurate model of image formation to combine an iterative optimization algorithm for solving the corresponding inverse problem with a learned image representation for robust alignment and a learned natural image prior. The proposed algorithm is fast, with low memory requirements compared to state-of-the-art learning-based approaches to image restoration, and features that are learned end to end from synthetic yet realistic data. Extensive experiments demonstrate its excellent performance with super-resolution factors of up to ×4 on real photographs taken in the wild with hand-held cameras, and high robustness to low-light conditions, noise, camera shake, and moderate object motion.
@article{lecouat22hdrsr,
author = {Bruno Lecouat and
Thomas Eboli and
Jean Ponce and
Julien Mairal},
title = {High dynamic range and super-resolution from raw image bursts},
journal = {{ACM} Transactions on Graphics},
volume = {41},
number = {4},
pages = {38:1--38:21},
year = {2022},
}
Joint HDR imaging and super-resolution ×4 with a burst taken with a hand-held Pixel4a at night, facing a spotlight. Top: The original burst. Middle: The central image in the burst (left) and the reconstructed HDR/SR image after tone mapping (right). Bottom: Six crops showing details of the original and HDR/SR images, presented respectively in the first and second rows.
Our method partly relies on exposure bracketing for restoring at the same time the saturated areas with the least exposed frames, and the highly noisy darker areas thanks to the brightest ones. An example is shown below. On the left: Three high dynamic range, high-resolution images obtained by our method from 18-image bursts taken by a handheld Pixel 4a smartphone with a ×4 super-resolution factor. We show post-processed sRGB pictures for the sake of presentation. On the right: Small crops from sample photos in the burst and our reconstruction. Note the high level of noise in the short-exposure images, in particular in the second row, and the saturated regions in the long-exposure ones. As shown by the last column of the figure, our algorithm recovers details in saturated areas and remove noise in the darkest regions. The reader is invited to zoom in on a computer screen.
Visual comparison for super-resolution only on real same-exposure raw bursts, of respectively 20 and 30 frames, with state-of-the-art competitors. We do not present HDR results in this figure. Our approach limits Moiré artefacts in the first row and reveals in general more high frequency image details in both rows. The last row shows an example requiring deghosting. The ghosted LR image on the left is obtained by averaging the whole burst to show the pedestrian’s motion. Bhat et al. [2021a] and our method effectively handle small object motions. The reader is invited to zoom in.
