High-quality deepfakes have a heart!

Clemens Seibold Eric L Wisotzky Arian Beckmann Benjamin Kossack Anna Hilsmann Peter Eisert ^*

• Fraunhofer HHI, Berlin, Germany

Deepfakes have become ubiquitous in our modern society and the number of manipulated videos and news stories on the internet are increasing. The current evolution of image generation techniques makes the detection of manipulated content through visible inspection increasingly difficult. This motivated researchers to analyze various signal properties for automatic deepfake detection. Among others, the complexity of heart rate dynamics in the video is used to distinguish between real videos and deepfakes. In particular, these signals are extracted from facial video streams through remote photoplethysmography (rPPG). The current state of the art assumption is that these subtle semantic signals get lost in the deepfake generation process and can thus be used for deepfake detection. This paper presents a study proving that this assumption is no longer valid for the current generation of deepfakes. To this end, we present a pipeline that extracts heart rate signals from the captured rPPG signals and show that current deepfake generation processes encapsulate the rPPG signal of the driving video sequence. For this, we captured a dataset of facial videos in synchrony with an electrocardiogram (ECG) as ground truth pulse signal. For deepfakes generated on this dataset, we show that (i) the deepfake video shows valid heart rate dynamic and that (ii) these match with those of the underlying source videos. Furthermore, we show that this also holds for deepfakes from a publicly available dataset.