An Explainable Self-Supervised Learning-Based Ensemble Framework for Audio Deepfake Detection
Loading...
Date
2026
Journal Title
Journal ISSN
Volume Title
Publisher
Faculty of Sciences
Abstract
Recent advances in artificial intelligence have enabled the generation of highly
realistic audio deepfakes through speech synthesis and voice conversion technologies.
Although these technologies offer numerous beneficial applications,
they also pose serious security, privacy, and misinformation risks, making reliable
audio deepfake detection an increasingly important research challenge.
To address this issue, this work proposes an audio deepfake detection framework
that combines self-supervised learning (SSL) and ensemble learning.
Three state-of-the-art SSL models: Wav2Vec 2.0, HuBERT, andWavLM, were
fine-tuned to distinguish real from fake speech. Several ensemble strategies
were investigated to exploit the complementary strengths of the individual
models. Furthermore, the Gradient Saliency explainable AI technique was employed
to enhance model interpretability and provide insights into the features
influencing the detection decisions.
The proposed framework was evaluated on the ASVspoof 2019 Logical Access
dataset and further assessed on the DeepVoice dataset to examine its generalization
capability. Experimental results demonstrated the effectiveness of SSLbased
speech representations for audio deepfake detection. Among all evaluated
methods, the stacking ensemble achieved the best performance, reaching
an accuracy of 99.15%, an F1-score of 99.14%, and an EER of 0.38% on
the ASVspoof 2019 LA dataset. Cross-dataset experiments on the DeepVoice
dataset yielded an accuracy of 81.25% and an F1-score of 88.89%, confirming
the robustness and generalization capability of the proposed approach. These
findings show that combining self-supervised speech representations with ensemble
learning provides an effective, robust, and interpretable solution for
audio deepfake detection.