DETECTION OF ANOMALIES IN THE TELECOMMUNICATIONS TRAFFIC BY STATISTICAL METHODS
DOI:
https://doi.org/10.28925/2663-4023.2021.11.183194Keywords:
anomaly detection, traffic, decision trees, fractal analysis, cluster analysis, attacks, false positivesAbstract
Anomaly detection is an important task in many areas of human life. Many statistical methods are used to detect anomalies. In this paper, statistical methods of data analysis, such as survival analysis, time series analysis (fractal), classification method (decision trees), cluster analysis, entropy method were chosen to detect anomalies. A description of the selected methods is given. To analyze anomalies, the traffic and attack implementations from an open dataset were taken. More than 3 million packets from the dataset were used to analyze the described methods. The dataset contained legitimate traffic (75%) and attacks (25%). Simulation modeling of the selected statistical methods was performed on the example of network traffic implementations of telecommunication networks of different protocols. To implement the simulation, programs were written in the Pyton programming language. DDoS attacks, UDP-flood, TCP SYN, ARP attacks and HTTP-flood were chosen as anomalies. A comparative analysis of the performance of these methods to detect anomalies (attacks) on such parameters as the probability of anomaly detection, the probability of false positive detection, the running time of each method to detect the anomaly was carried out. Experimental results showed the performance of each method. The decision tree method is the best in terms of anomaly identification probability, fewer false positives, and anomaly detection time. The entropy analysis method is slightly slower and gives slightly more false positives. Next is the cluster analysis method, which is slightly worse at detecting anomalies. Then the fractal analysis method showed a lower probability of detecting anomalies, a higher probability of false positives and a longer running time. The worst was the survival analysis method.
Downloads
References
Radivilova, T., Kirichenko, L., Tawalbeh, M., Zinchenko, P., & Bulakh, V. (2020). THE LOAD BALANCING OF SELF-SIMILAR TRAFFIC IN NETWORK INTRUSION DETECTION SYSTEMS. Cybersecurity: Education, Science, Technique, 3(7), 17–30. https://doi.org/10.28925/2663-4023.2020.7.1730
Han, J., Kamber, M., Pei, J. (2011). Data mining concepts and techniques third edition. The Morgan Kaufmann Series in Data Management Systems, 5(4), 83-124. https://doi.org/10.1016/C2009-0-61819-5
Chandola, V., Banerjee, A., Kumar, V. (2009). Anomaly Detection: A Survey. ACM Computing Survey, 41, 1–58.
Kirichenko, L., Radivilova, T., & Tkachenko, A. (2019). Comparative Analysis of Noisy Time Series Clustering. У COLINS-2019: Proceedings of the 3rd International Conference on Computational Linguistics and Intelligent Systems, Volume I: Main Conference Kharkiv, Ukraine (p. 184–196).
Madhuri, G. S. (2020). Usha Rani M. Statistical Approaches to Detect Anomalies. У Venkata Krishna P., Obaidat M. (eds) Emerging Research in Data Engineering Systems and Computer Communications. Advances in Intelligent Systems and Computing. https://doi.org/10.1007/978-981-15-0135-7_46.
Bendich, P., Chin, S. P., Clark, J., Desena, J., Harer, J., Munch, E., Newman, A., Porter, D., Rouse, D., Strawn, N., & Watkins, A. (2016). Topological and statistical behavior classifiers for tracking applications. IEEE Transactions on Aerospace and Electronic Systems, 52(6), 2644–2661. https://doi.org/10.1109/taes.2016.160405
Goldstein, M., & Uchida, S. (2016). A Comparative Evaluation of Unsupervised Anomaly Detection Algorithms for Multivariate Data. PLOS ONE, 11(4), Стаття e0152173. https://doi.org/10.1371/journal.pone.0152173
Kirichenko, L., Radivilova, T., & Bulakh, V. (2019). Machine Learning in Classification Time Series with Fractal Properties. Data, 4(5), 1-13. https://doi.org/10.3390/data4010005
Han, M. L., Kwak, B. I., & Kim, H. K. (2018). Anomaly intrusion detection method for vehicular networks based on survival analysis. Vehicular Communications, 14, 52–63. https://doi.org/10.1016/j.vehcom.2018.09.004
Pinto, J. D.(2015). Outlier Detection in Survival Analysis: Thesis to obtain the Master of Science Degree in Electrical and Computer Engineering.
Zhang, R., Zhou, M., Gong, X., He, X., Qian, W., Qin, S., & Zhou, A. (2014). Detecting anomaly in data streams by fractal model. World Wide Web, 18(5), 1419–1441. https://doi.org/10.1007/s11280-014-0296-y
Gong, X., Qian, W., Qin, S., Zhou, A. (2003). Fractal Based Anomaly Detection over Data Streams. In: Ishikawa Y., Li J., Wang W., Zhang R., Zhang W. (eds) Web Technologies and Applications. Lecture Notes in Computer Science. https://doi.org/10.1007/978-3-642-37401-2_54
Radivilova, T., Kirichenko, L., Alghawli, A. S., Ilkov, A., Tawalbeh, M., Zinchenko, P. (2020). The complex method of intrusion detection based on anomaly detection and misuse detection. У DESSERT: Proceedings of 2020 IEEE 11th International Conference on Dependable Systems, Services and Technologies (с. 133-137). https://doi.org/10.1109/DESSERT50317.2020.9125051.
Kirichenko, L., Radivilova, T., & Bulakh, V. Binary classification of fractal time series by machine learning methods. У V. Lytvynenko, S. Babichev, W. Wójcik, O. Vynokurova, S. Vyshemyrskaya & S. Radetskaya (Ред.), Lecture notes in computational intelligence and decision making (с. 701–711). Advances in Intelligent Systems and Computing.
Reif, M., Goldstein, M., Stahl, A., Breuel, T. M. (2008). Anomaly detection by combining decision trees and parametric densities. 19th International Conference on Pattern Recognition: Proceedings (с. 1-4).
Botana, I. L.-R., Eiras-Franco, C., & Alonso-Betanzos, A. (2020). Regression Tree Based Explanation for Anomaly Detection Algorithm. Proceedings, 54(1), 7. https://doi.org/10.3390/proceedings2020054007
Kirichenko, L.O., Tkachenko, A.E., Radivilova, T.A. (2019). Clustering of noisy time series. System technologies. Regional mіzhvuzіvskiy zbіrnik naukovikh prats, 3 (122), 133-139.
Alam, M. (2020). DBSCAN — a density-based unsupervised algorithm for fraud detection. Medium. https://towardsdatascience.com/dbscan-a-density-based-unsupervised-algorithm-for-fraud-detection-887c0f1016e9
Sheridan, K., Puranik, T. G., Mangortey, E., Pinon-Fischer, O. J., Kirby, M., Mavris, D. N. (2020). An application of dbscan clustering for flight anomaly detection during the approach phase. AIAA: Proceedings of Scitech 2020 Forum, (р. 1851). https://doi.org/10.2514/6.2020-1851
Saeedi Emadi, H., & Mazinani, S. M. (2017). A Novel Anomaly Detection Algorithm Using DBSCAN and SVM in Wireless Sensor Networks. Wireless Personal Communications, 98(2), 2025–2035. https://doi.org/10.1007/s11277-017-4961-1
Gu, Y., McCallum, A., Towsley, D. (2005). Detecting Anomalies in Network Traffic Using Maximum Entropy Estimation. SIGCOMM: Proceedings of the 5th ACM conference on Internet Measurement (р. 32–32).
Radivilova, T., Kirichenko, L., Alghawli, A. S. (2019). Entropy Analysis Method for Attacks Detection. PIC S&T: Proceedings of 2019 IEEE International Scientific-Practical Conference Problems of Infocommunications, Science and Technology, (р. 443-446). https://doi.org/10.1109/PICST47496.2019.9061451
Callegari, C., Giordano, S., Pagano, M. (2017). Entropy-based network anomaly Detection. ICNC: Proceedings of 2017 International Conference on Computing (р. 334-340), Networking and Communications. https://doi.org/10.1109/ICCNC.2017.7876150.
Shukla, A. S., & Maurya, R. (2018). Entropy-Based Anomaly Detection in a Network. Wireless Personal Communications, 99(4), 1487–1501. https://doi.org/10.1007/s11277-018-5288-2
UGR'16 Dataset. NESG - Home. https://nesg.ugr.es/nesg-ugr16/
Kalita, J. K., Bhuyan, M. H., & Bhattacharyya, D. K. (2017). Network Traffic Anomaly Detection and Prevention: Concepts, Techniques, and Tools. Springer.
Saad, A., Sisworahardjo, N. (2017). Data analytics-based anomaly detection in smart distribution network. ICHVEPS: Proceedings of the 2017 International Conference on High Voltage Engineering and Power Systems, IEEE.
Fernandes, G., Rodrigues, J. J. P. C., Carvalho, L. F., Al-Muhtadi, J. F., & Proença, M. L. (2018). A comprehensive survey on network anomaly detection. Telecommunication Systems, 70(3), 447–489. https://doi.org/10.1007/s11235-018-0475-8
