DETECTING CLONED VOICES WITH CLASSICAL MACHINE LEARNING
DOI:
https://doi.org/10.28925/2663-4023.2026.35.1423Keywords:
audio deepfake, synthetic speech detection, voice cloning, spoofing countermeasures, speaker verification, support vector machine, machine learningAbstract
The rapid progress of neural text-to-speech (TTS) and voice conversion (VC) technologies has enabled highly realistic synthetic speech, significantly increasing the risk of voice spoofing and social engineering attacks. This work investigates synthetic speech detection using classical machine learning methods based on handcrafted acoustic features. For experimental evaluation, a dataset of 3,600 recordings is constructed: 400 bona fide utterances from 20 speakers and 3,200 utterances synthesized by four modern voice cloning and TTS systems — RVC, XTTS, ElevenLabs, and Tortoise (class ratio 1 : 8). Each recording is represented by a 135-dimensional vector of cepstral, spectral, and temporal features, which is fed to seven classical classifiers, including support vector machines, ensemble methods, and k-nearest neighbors. The best result is obtained with an RBF-kernel SVM, which achieves a balanced accuracy of 91.1%, correctly accepting 85% of bona fide recordings and detecting 97.2% of synthetic ones. Per-system analysis shows that the main source of errors is the false rejection of bona fide speech: for four of the seven models, only 27–64% of bona fide recordings are correctly accepted, whereas synthetic speech is detected with an accuracy of at least 85%. Speech generated by the commercial ElevenLabs system proved the most difficult to detect for linear models. The results constitute baselines for a setting in which speakers and synthesis systems are shared between the training and test sets, and provide a reference point for comparison with deep learning–based detectors.
Downloads
References
Li, M., Ahmadiadli, Y., & Zhang, X.-P. (2025). A survey on speech deepfake detection. ACM Computing Surveys, 57(7), 1–38. https://doi.org/10.1145/3714458
Mirsky, Y., & Lee, W. (2021). The creation and detection of deepfakes: A survey. ACM Computing Surveys, 54(1), 1–41. https://doi.org/10.1145/3425780
Sahidullah, M., Kinnunen, T., & Hanilçi, C. (2015). A comparison of features for synthetic speech detection. In Proceedings of Interspeech 2015 (pp. 2087–2091). https://doi.org/10.21437/Interspeech.2015-472
Lavrentyeva, G., Novoselov, S., Malykh, E., Kozlov, A., Kudashev, O., & Shchemelinin, V. (2017). Audio replay attack detection with deep learning frameworks. In Proceedings of Interspeech 2017 (pp. 82–86). https://doi.org/10.21437/Interspeech.2017-360
Tak, H., Patino, J., Todisco, M., Nautsch, A., Evans, N., & Larcher, A. (2021). End-to-end anti-spoofing with RawNet2. In ICASSP 2021 – 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (pp. 6369–6373). IEEE. https://doi.org/10.1109/ICASSP39728.2021.9414234
Jung, J.-w., Heo, H.-S., Tak, H., Shim, H.-j., Chung, J. S., Lee, B.-J., Yu, H.-J., & Evans, N. (2022). AASIST: Audio anti-spoofing using integrated spectro-temporal graph attention networks. In ICASSP 2022 – 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (pp. 6367–6371). IEEE. https://doi.org/10.1109/ICASSP43922.2022.9747766
Müller, N., Czempin, P., Diekmann, F., Froghyar, A., & Böttinger, K. (2022). Does audio deepfake detection generalize? In Proceedings of Interspeech 2022 (pp. 2783–2787). https://doi.org/10.21437/Interspeech.2022-108
Todisco, M., Wang, X., Vestman, V., Sahidullah, M., Delgado, H., Nautsch, A., Yamagishi, J., Evans, N., Kinnunen, T. H., & Lee, K. A. (2019). ASVspoof 2019: Future horizons in spoofed and fake audio detection. In Proceedings of Interspeech 2019 (pp. 1008–1012). https://doi.org/10.21437/Interspeech.2019-2249
Yamagishi, J., Wang, X., Todisco, M., Sahidullah, M., Patino, J., Nautsch, A., Liu, X., Lee, K. A., Kinnunen, T., Evans, N., & Delgado, H. (2021). ASVspoof 2021: Accelerating progress in spoofed and deepfake speech detection. In Proceedings of the 2021 Edition of the Automatic Speaker Verification and Spoofing Countermeasures Challenge (pp. 47–54). https://doi.org/10.21437/ASVSPOOF.2021-8
Yi, J., Tao, J., Fu, R., Yan, X., Wang, C., Wang, T., Zhang, C. Y., Zhang, X., Zhao, Y., Ren, Y., Xu, L., Zhou, J., Gu, H., Wen, Z., Liang, S., Lian, Z., Nie, S., & Li, H. (2023). ADD 2023: The second audio deepfake detection challenge. In Proceedings of the IJCAI 2023 Workshop on Deepfake Audio Detection and Analysis (DADA 2023) (CEUR Workshop Proceedings, Vol. 3597, pp. 125–130). https://ceur-ws.org/Vol-3597/
Brydinskyi, V., Sabodashko, D., Khoma, Y., Podpora, M., Konovalov, A., & Khoma, V. (2024). Enhancing automatic speech recognition with personalized models: Improving accuracy through individualized fine-tuning. IEEE Access, 12, 116649–116656. https://doi.org/10.1109/ACCESS.2024.3443811
Ravanelli, M., & Bengio, Y. (2018). Speaker recognition from raw waveform with SincNet. In 2018 IEEE Spoken Language Technology Workshop (SLT) (pp. 1021–1028). IEEE. https://doi.org/10.1109/SLT.2018.8639585
Zaiets, I., Brydinskyi, V., Sabodashko, D., Khoma, Y., & Ruda, K. (2024). Integrated system for speaker diarization and intruder detection using speaker embeddings. In Cybersecurity Providing in Information and Telecommunication Systems (CPITS 2024) (CEUR Workshop Proceedings, Vol. 3654, pp. 228–238). https://ceur-ws.org/Vol-3654/
Tak, H., Todisco, M., Wang, X., Jung, J.-w., Yamagishi, J., & Evans, N. (2022). Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation. In Proceedings of The Speaker and Language Recognition Workshop (Odyssey 2022) (pp. 112–119). https://doi.org/10.21437/Odyssey.2022-16
Ruda, K., Sabodashko, D., Kos, I., Opirskyy, I., & Akhmedova, A. (2025). Scalability analysis of neural network architectures for voice biometric authentication. In Cybersecurity Providing in Information and Telecommunication Systems II (CPITS-II 2025) (CEUR Workshop Proceedings, Vol. 4145, pp. 349–357). https://ceur-ws.org/Vol-4145/
Sun, C., Jia, S., Hou, S., & Lyu, S. (2023). AI-synthesized voice detection using neural vocoder artifacts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (pp. 904–912).
ElevenLabs. (n.d.). ElevenLabs: AI voice generator and text to speech. Retrieved October 9, 2026, from https://elevenlabs.io
RVC-Project. (2023). Retrieval-based-Voice-Conversion-WebUI [Computer software]. GitHub. https://github.com/RVC-Project/Retrieval-based-Voice-Conversion-WebUI
Kim, J., Kong, J., & Son, J. (2021). Conditional variational autoencoder with adversarial learning for end-to-end text-to-speech. In Proceedings of the 38th International Conference on Machine Learning (PMLR Vol. 139, pp. 5530–5540).
Betker, J. (2023). Better speech synthesis through scaling (arXiv:2305.07243). arXiv. https://doi.org/10.48550/arXiv.2305.07243
Casanova, E., Davis, K., Gölge, E., Göknar, G., Gulea, I., Hart, L., Aljafari, A., Meyer, J., Morais, R., Olayemi, S., & Weber, J. (2024). XTTS: A massively multilingual zero-shot text-to-speech model. In Proceedings of Interspeech 2024 (pp. 4978–4982). https://doi.org/10.21437/Interspeech.2024-2016
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Христина Руда, Ігор Кос, Дмитро Сабодашко, Марія Швед, Павло Глущенко
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
