Lightweight and Hybrid Transformer-Based Solution for Quick and Reliable DeepFake Detection

Geeta Rani ¹ Atharv Kothekar ¹ Shawn George Philip ¹ Vijaypal Singh Dhaka ^1* Ester Zumpano ² Eugenio Vocaturo ^2*

• ¹ Manipal University Jaipur, Jaipur, India
• ² Department of Computer Engineering, Modeling, Electronics and Systems, University of Calabria, Arcavacata, Calabria, Italy

Rapid advancements in artificial intelligence and generative artificial intelligence have enabled the creation of fake images and videos that appear highly realistic. According to a report published in 2022, approximately 71% of people rely on fake videos and become victims of blackmail.Moreover, these fake videos and images are used to tarnish the reputation of popular public figures. This has increased the demand for deepfake detection techniques. The accuracy of the techniques proposed in the literature so far varies with changes in fake content generation techniques. Additionally, these techniques are computationally intensive. The techniques discussed in the literature are based on convolutional neural networks, linformer models, or transformer models for deepfake detection, each with its advantages and disadvantages. In this manuscript, a hybrid architecture combining transformer and linformer models is proposed for deepfake detection. This architecture converts an image into patches and performs position encoding to retain spatial relationships between patches. Its encoder captures the contextual information from the input patches, and Gaussian Error Linear Unit resolves the vanishing gradient problem. The Linformer component reduces the size of the attention matrix. Thus, it reduces the execution time to half without compromising accuracy. Moreover, it utilizes the unique features of transformer and linformer models to enhance the robustness and generalization of deepfake detection techniques. The low computational requirement and high accuracy of 98.9% increase the real-time applicability of the model, preventing blackmail and other losses to the public.