Skip to main content

ORIGINAL RESEARCH article

Front. Comput. Sci.
Sec. Human-Media Interaction
Volume 6 - 2024 | doi: 10.3389/fcomp.2024.1478233
This article is part of the Research Topic Artificial Intelligence: The New Frontier in Digital Humanities View all 7 articles

Inpainting with Style: Forcing style coherence to image inpainting with Deep Image Prior

Provisionally accepted
  • University of Bologna, Bologna, Italy

The final, formatted version of the article will be published soon.

    In this paper, we combine the Deep Image Prior (DIP) framework with a Style Transfer (ST) technique to propose a novel approach (called DIP-ST) for image inpainting of artworks. We specifically tackle cases where the regions to fill in are large. Hence, part of the original painting is irremediably lost, and new content must be generated. In DIP-ST, a convolutional neural network processes the damaged image while a pre-trained VGG network forces a style constraint to ensure that the inpainted regions maintain stylistic coherence with the original artwork. We evaluate our method performance to inpaint different artworks, and we compare DIP-ST to some state-of-the-art techniques. Our method provides more reliable solutions characterized by a higher fidelity to the original images, as confirmed by better values of quality assessment metrics. We also investigate the effectiveness of the style loss function in distinguishing between different artistic styles and the results show that the style loss metric accurately measures artistic similarities and differences. At last, despite the use of neural networks, DIP-ST does not require a dataset for training making it particularly suited for art restoration where relevant datasets may be scarce.

    Keywords: deep learning, Deep image prior, Style transfer, Art restoration, Image inpainting, unsupervised learning

    Received: 09 Aug 2024; Accepted: 30 Sep 2024.

    Copyright: © 2024 Morotti, Merizzi, Evangelista and Cascarano. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

    * Correspondence: Pasquale Cascarano, University of Bologna, Bologna, Italy

    Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.