ADVERSARIAL SEGMENTATION LOSS FOR SKETCH COLORIZATION


Creative Commons License

Hicsonmez S., Samet N., AKBAŞ E., DUYGULU ŞAHİN P.

2021 IEEE International Conference on Image Processing, ICIP 2021, Alaska, Amerika Birleşik Devletleri, 19 - 22 Eylül 2021, cilt.2021-September, ss.2403-2407 identifier identifier

  • Yayın Türü: Bildiri / Tam Metin Bildiri
  • Cilt numarası: 2021-September
  • Doi Numarası: 10.1109/icip42928.2021.9506637
  • Basıldığı Şehir: Alaska
  • Basıldığı Ülke: Amerika Birleşik Devletleri
  • Sayfa Sayıları: ss.2403-2407
  • Anahtar Kelimeler: sketch colorization, sketch to image translation, Generative Adversarial Networks (GAN), image segmentation, image to image translation
  • Orta Doğu Teknik Üniversitesi Adresli: Evet

Özet

© 2021 IEEEWe introduce a new method for generating color images from sketches or edge maps. Current methods either require some form of additional user-guidance or are limited to the “paired” translation approach. We argue that segmentation information could provide valuable guidance for sketch colorization. To this end, we propose to leverage semantic image segmentation, as provided by a general purpose panoptic segmentation network, to create an additional adversarial loss function. Our loss function can be integrated to any baseline GAN model. Our method is not limited to datasets that contain segmentation labels, and it can be trained for “unpaired” translation tasks. We show the effectiveness of our method on four different datasets spanning scene level indoor, outdoor, and children book illustration images using qualitative, quantitative and user study analysis. Our model improves its baseline up to 35 points on the FID metric. Our code and pretrained models can be found at https://github.com/giddyyupp/AdvSegLoss.