Deep Joint Deinterlacing and Denoising for Single Shot Dual-ISO HDR Reconstruction


Cogalan U., AKYÜZ A. O.

IEEE TRANSACTIONS ON IMAGE PROCESSING, vol.29, pp.7511-7524, 2020 (SCI-Expanded) identifier identifier

  • Publication Type: Article / Article
  • Volume: 29
  • Publication Date: 2020
  • Doi Number: 10.1109/tip.2020.3004014
  • Journal Name: IEEE TRANSACTIONS ON IMAGE PROCESSING
  • Journal Indexes: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, PASCAL, Aerospace Database, Applied Science & Technology Source, Business Source Elite, Business Source Premier, Communication Abstracts, Compendex, Computer & Applied Sciences, EMBASE, INSPEC, MEDLINE, Metadex, zbMATH, Civil Engineering Abstracts
  • Page Numbers: pp.7511-7524
  • Keywords: Dual-ISO, HDR imaging, noise, deep learning
  • Middle East Technical University Affiliated: Yes

Abstract

HDR images have traditionally been obtained by merging multiple exposures each captured with a different exposure time. However, this approach entails longer capture times and necessitates deghosting if the captured scene contains moving objects. With the advent of modern camera sensors that can perform per-pixel exposure modulation, it is now possible to capture all of the required exposures within a single shot. The new challenge then becomes how to best combine different pixels with different exposure values into a single full-resolution and low-noise HDR image. We propose a joint multi-exposure frame deinterlacing and denoising algorithm powered by deep convolutional neural networks (DCNN). In our algorithm, we first train two DCNNs, with one tuned for reconstructing low exposures and the other for high exposures. Each DCNN takes the same mosaicked dual-ISO input image and outputs either the low exposure or high exposure depending on the type of the network. The resulting exposures can be demosaicked and converted to the desired target color space prior to HDR assembly. Our evaluations indicate that the quality of our results significantly surpasses the state-of-the-art in single-image HDR reconstruction algorithms.