Yıl 2018, Cilt 22, Sayı 4, Sayfalar 1109 - 1123 2018-08-01

Anomaly Detection Using Data Mining Methods in IT Systems: A Decision Support Application
BT Sistemlerinde Veri Madenciliği Yöntemlerini Kullanarak Anomali Algılama: Karar Destek Uygulaması

Ferdi Sönmez [1] , Metin Zontul [2] , Oğuz Kaynar [3] , Hayati Tutar [4]

274 483

Although there are various studies on anomaly detection, simple and effective anomaly detection approaches are still necessary due to the lack of appropriate approaches for large-scale network environments. In the existing analysis methods, it is seen that the methods of preliminary analysis are generally used, the extrapolations and probabilities are not taken into account and the unsupervised neural network (NN) methods are not used enough. As an alternative, the use of the Self-Organizing Maps has been preferred in the study. In other studies, analysis of data obtained from network traffic is analyzed, here, analysis of other information systems data and suggestions for alternative solutions are given, too. In addition, in-memory database systems have been used in practice in order to enable faster processing in analysis studies, due to the large size of data to be analyzed in large-scale network environments. An analysis of the application log data obtained from the management tools in the information systems was carried out. After anomaly detection results obtained and the verification test results are compared, it is found out that anomaly detection process is successful by 96%. The advantage offered for the company and users at IT and security monitoring processes is to eliminate the need for pre-qualification and to reduce the heavy workload. By this way, it is thought that a significant cost item is eliminated. It is also contemplated that the security vulnerabilities and problems associated with unpredictable issues will be detected through practice and thus many attacks and problems will be prevented in advance.

 Anomali tespiti üzerine çeşitli çalışmalar olmasına rağmen, büyük ölçekli ağ ortamları için uygun yaklaşımların eksikliği nedeniyle basit ve etkili anormali tespit yaklaşımları hala gereklidir. Mevcut çalışmalardaki analiz yöntemlerinde, genellikle ön tanımlı analiz tekniklerinin kullanıldığı, kapsam dışı durum ve olasılıkların dikkate alınmadığı ve danışmansız öğrenen yapay sinir ağları (NN) metotlarının yeterince kullanılmadığı görülmektedir. Alternatif olarak bu çalışmada özörgütlemeli harita ağları kullanımı tercih edilmiştir. Diğer çalışmalarda, genellikle ağ trafiğinden elde dilen veriler analiz edilirken, bu çalışmada diğer bilişim sistemi verilerinin de analizine ve alternatif olabilecek çözüm önerilerine yer verilmektedir. Ayrıca, büyük ölçekli ağ ortamlarında analiz edilecek verilerin büyük boyutlu olmasına bağlı olarak, analiz çalışmalarında işlemlerin daha hızlı gerçekleştirilebilmesi amacıyla uygulamada Bellek İçi Veritabanı Sistemleri (BİVTS) kullanılmıştır. Bilişim sitemlerindeki yönetim araçlarından edinilen uygulama log verilerinin analizi sonrasında anomali tespitinin %96 oranında başarı ile gerçekleştiği gözlenmiştir. Çalışmanın işletme ve kullanıcılarına bilişim sistemlerini izleme ve güvenlik takibi işlemlerinde ön tanımlama ihtiyacını ortadan kaldırarak, yoğun iş yükünü azaltma yönünde fayda sağlayacağı düşünülmektedir. Böylelikle, önemli bir maliyet kaleminin ortadan kalkacağı da düşünülmektedir. Ayrıca, öngörülemeyen hususlara bağlı güvenlik açıkları ve problemlerin, uygulama ile tespiti ve böylelikle birçok saldırı ve problemin önceden engellenileceği öngörülmektedir.

  • M. V. O. Assis, J. J. P. C. Rodrigues, M. L. Proença. “A seven-dimensional flow analysis to help autonomous network management”, Information Sciences, 278, 900-913, 2014, doi: 10.1016/j.ins.2014.03.102.
  • A. Coluccia, A. D’Alconzo, F. Ricciato. “Distribution-based anomaly detection via generalized likelihood ratio test: A general Maximum Entropy approach”, Computer Networks, 57(17), ss.3446-3462, 2013, http://dx.doi.org/10.1016/j.comnet.2013.07.028.
  • F. Mata, P. Żuraniewski, M. Mandjes, M. Mellia. “Anomaly detection in diurnal data”, Computer Networks, 60, ss. 187-200, 2014.
  • M. A. Rassam, A. Zainal, M. A. Maarof. “An Efficient Distributed Anomaly Detection Model for Wireless Sensor Networks”, AASRI Procedia, 5, ss. 9-14, 2013, doi: 10.1016/j.aasri.2013.10.052.
  • S. Anil, R. Remya. “A hybrid method based on genetic algorithm, self-organised feature map, and support vector machine for better network anomaly detection", 2013 Fourth International Conference on Computing, Communications and Networking Technologies (ICCCNT), Tiruchengode, ss.1-5, 2013, doi: 10.1109/ICCCNT. 2013.6726604
  • A. Juvonen, T. Sipola, T. Hämäläinen. “Online anomaly detection using dimensionality reduction techniques for HTTP log analysis”, Computer Networks, 91(14), ss.46-56, 2015, doi: 10.1016/j.comnet. 2015.07.019.
  • I. Fronza, A. Sillitti, G. Succi, M. Terho, J. Vlasenko. “Failure prediction based on log files using Random Indexing and Support Vector Machines”, Journal of Systems and Software, 86(1), ss.2-11, 2013, doi: 10.1016/j.jss.2012.06.025.
  • D. Olszewski. “Fraud detection using self-organizing map visualizing the user profiles”, Knowledge-Based Systems, 70, 324-334, 2014, doi: 10.1016/j.knosys.2014.07.008.
  • C. Modi, D. Patel, B. Borisaniya, H. Patel, A. Patel, M. Rajarajan. “A survey of intrusion detection techniques in Cloud”, Journal of Network and Computer Applications, 36(1), ss.42-57, 2013, doi: 10.1016/j.jnca.2012.05.003.
  • A. Botta, A. Dainotti, A. Pescapé. “A tool for the generation of realistic network workload for emerging networking scenarios”, Computer Networks, 56(15), ss.3531-3547, 2012, doi: 10.1016/j. comnet.2012.02.019.
  • D. A. S. Resul, I. Turkoglu, I., M. Poyraz. “Analyzing of system errors for increasing a web server performance by using web usage mining”, IU-Journal of Electrical & Electronics Engineering, 7(2), ss.379-386, 2007.
  • S. A. Ünlü. “Ağ Üzerinden Yavaşlama Tabanlı Anomali Tespiti”, Tez Çalışması, TOBB Ekonomi Ve Teknoloji Üniversitesi, Fen Bilimleri Enstitüsü, 2011.
  • P. Ma. “Log Analysis-Based Intrusion Detection via Unsupervised Learning”, Master of Science, School of Informatics, University of Edinburgh, 2003.
  • C. Chiu, Y. Ku, T. Lie, Y. Chen. “Internet auction fraud detection using social network analysis and classification tree approaches”, Int. J. Electron. Commer, 15 (3), ss.123–147, 2011.
  • A. Li, L. Gu, K. Xu. "Fast Anomaly Detection for Large Data Centers," 2010 IEEE Global Telecommunications Conference GLOBECOM, Miami, ABD, 2010, doi: 10.1109/GLOCOM. 2010.5683551
  • Y. Kanda, K. Fukuda, T. Sugawara. "A Flow Analysis for Mining Traffic Anomalies", 2010 IEEE International Conference on Communications, Cape Town, 2010, doi: 10.1109/ ICC.2010.5502463
  • S. Molnar, Z. Moczar. "Three-Dimensional Characterization of Internet Flows," 2011 IEEE International Conference on Communications (ICC), Kyoto, 2011, doi: 10.1109/icc. 2011.5963476
  • P. P. Cortez, M. Rio, M. Rocha, P. Sousa. "Internet Traffic Forecasting using Neural Networks," The 2006 IEEE International Joint Conference on Neural Network Proceedings, Vancouver, BC, 2006, doi: 10.1109/IJCNN. 2006.247142
  • A. Lakhina, K. Papagiannaki, M. Crovella, C. Diot, E.D. Kolaczyk, N. Taft. “Structural analysis of network traffic flow”s, SIGMETRICS Perform. Eval. Rev., ss.32, 61–72, 2004.
  • C. Yoohee, K. Yihan. “Case study of an anomalous traffic detection on the aggregation points of enterprise network”, International Conference on Advanced Communication Technology (ICACT), Seul, 2011.
  • A. Chandola, V. Chandola, V. Kumar. “Anomaly Detection: A Survey”, ACM Comput. Surv., 41(3), 2009, doi: 10.1145/1541880. 1541882
  • N. Carneiro, G. Figueira, M. Costa. “A data mining based system for credit-card fraud detection in e-tail”, Decision Support Systems, 95, ss.91-101, 2017, doi: 10.1016/j.dss.2017.01.002.
  • V. Kumar. "Parallel and distributed computing for cybersecurity" IEEE Distributed Systems Online, 6(10), 2005. doi: 10.1109/MDSO. 2005.53 .
  • K. A. Smith. Introduction to neural networks and data mining for business applications. Eruditions Publishing, Melbourne, 1999.
  • D. Swagatam, D. Ajith, K. Amit, ‘‘Automatic kernel clustering with a multi-elitist particle swarm optimization algorithm’’, Pattern: Recognition Letters, 29(5), ss.688–699, 2008.
  • J.Z. Lei, A.A. Ghorbani.”Improved competitive learning neural networks for network intrusion and fraud detection”, Neurocomputing, 75 (1), 135-145, 2012, doi: 10.1016/j.neucom. 2011.02.021.
  • T. Fawcett, ROC Graphs: Notes and Practical Considerations for Data Mining Researchers, Tech. Rep. HPL-2003-4, HP Labs, 2003
  • A. Mitrokotsa, N. Komninos, C. Douligeris. “Intrusion Detection with Neural Networks and Watermarking Techniques for MANET,” IEEE computer society, ss.1-10, 2008.
  • W. Wanga, H. Wang, B. Wang, Yaping Wang, Jiajun Wang. “Energy-aware and self-adaptive anomaly detection scheme based on network tomography in mobile ad hoc networks,” Information Sciences 220, ss.580–602, 2013.
  • S. Haykin, Neural Networks and Learning Machine, 3E, Pearson Education Inc., New Jersey, 2009.
  • L. Cao. “Support vector machines experts for time series forecasting”, Neurocomputing, 51, ss.321-329, doi:10.1016/S0925-2312(02) 00577-5.
  • F. Sönmez, Ş. Bülbül. “Intelligent Software Model Design for Estimating Deposit Banks Profitability with Soft Computing Techniques”, Neural Network World, ss.319-345, 2015, doi: 10.14311/NNW.2015.25.017.
  • D. Altaş, A. M. Çilingirtürk, V. Gülpınar. “Analyzing the process of the artificial neural networks by the help of the social network analysis”, New Knowledge Journal of Science. 2(2), ss.80–91, 2013.
  • B. Yıldız, S. Akkoç. “Banka Finansal Başarısızlıklarının Sinirsel Bulanık Ağ Yöntemi ile Öngörüsü”, BDDK Bankacılık ve Finansal Piyasalar, 3(1), ss.9-36, 2009.
  • L. A. Zadeh. “The Roles of Fuzzy Logic and Soft Computing in the Conception, Design and Deployment of Intelligent Systems”, BT Technology Journal, 14(4), ss.32-36, 1994.
  • T. K. Kohonen. “The self-organizing map”, Proceedings of the IEEE, 78 (9), ss.1464–1480, 1990.
  • T. K. Kohonen, S. Kaski, K. Lagus, J. Saloj¨arvi, J. Honkela, V. Paatero, A. Saarela. “Self Organization of a Massive Document Collection”, IEEE Transactions on Neural Networks, 11(3), ss.574–585, 2000.
  • Bullinaria, J. A. Introduction to neural networks. University of Birmingham, UK, 2004.
  • L. Yang, Z. Ouyang, Y. Shi. “A Modified Clustering Method Based on Self-Organizing Maps and Its Applications”, Procedia Computer Science, 9, ss.1371-1379, 2012, doi: doi.org/10.1016/j.procs.2012. 04.151.
  • J. A. Kangas, T. K. Kohonen, J. T. Jorma. “Variants of self-organizing maps”, IEEE transactions on neural networks, 1(1), ss.93-99, 1990.
  • N. R Pal, J. C. Bezdek, E. C. K. Tsao. “Generalized clustering networks and Kohonen's self-organizing scheme”, IEEE transactions on Neural Networks, 4 (4), ss.549-557,1993.
  • B. Hammer, T. Villmann. “Generalized relevance learning vector quantization”, Neural Networks, 15(8–9), 1059-1068, 2002, doi: 10.1016/S0893-6080(02)00079-5.
  • T. M. Martinetz, S. G.Berkovich, K. J. Schulten. “Neural-gas network for vector quantization and its application to time-series prediction", Neural Networks, IEEE Transactions on, 4(4), ss.558-569, 1993, doi: 10.1109/72.238311.
  • G. M. Afify, A. E. Bastawissy, O. M. Hegazy. “A hybrid filtering approach for storage optimization in main-memory cloud database”, Egyptian Informatics Journal, 16(3), ss.329-337, 2015, doi: 10.1016/j.eij.2015.06.007.
  • A. T. Kabakus, R. Kara. “A performance evaluation of in-memory databases”, Journal of King Saud University - Computer and Information Sciences, 29(4), ss.520-525, 2017, doi:10.1016/j.jksuci.2016.06.007.
  • T. Lahiri, M. A. Neimat, S. Folkman. “Oracle TimesTen: An In-Memory Database for Enterprise Applications”, IEEE Data Eng. Bull., 36(2), ss.6-13, 2013.
  • P. Jaroslav. “NoSQL databases: a step to database scalability in web environment”, International Journal of Web Information Systems, 9(1), ss.69-82, 2013.
  • P. Chao, D. He, S. Sadiq, K. Zheng, X. Zhou. "A performance study on large-scale data analytics using disk-based and in-memory database systems," 2017 IEEE International Conference on Big Data and Smart Computing (BigComp), Jeju, ss. 247-254, 2017, doi: 10.1109/BIGCOMP. 2017.7881706
  • Y. Wang, G. Zhong, L. Kun, L. Wang, H. Kai, F. Guo. "The Performance Survey of in Memory Database", 2015 IEEE 21st International Conference on Parallel and Distributed Systems (ICPADS), Melbourne, Avustralya, ss.815-820, 2015, doi: 10.1109/ICPADS.2015.109.
  • S.-Y. Huang, R.-H. Tsaih, F. Yu. “Topological pattern discovery and feature extraction for fraudulent financial reporting”, Expert Syst. Appl. 41 (9) , ss.4360–4372, 2014.
  • P. C. González, J.D. Velásquez. “Characterization and detection of taxpayers with false invoices using data mining techniques”, Expert Syst. Appl., 40 (5), ss.1427–1436, 2013.
  • S. Jha, M. Guillen, J.C. Westland. “Employing transaction aggregation strategy to detect credit card fraud”, Expert Syst. Appl., 39, ss.12650–12657, 2012.
  • D. Olszewski. “A probabilistic approach to fraud detection in telecommunications”, Knowledge-Based Systems, 26, ss.246–258, 2012.
  • V. D. Kumar, S. Radhakrishnan. "Intrusion detection in MANET using Self Organizing Map (SOM)", 2014 International Conference on Recent Trends in Information Technology, Chennai, 2014, doi: 10.1109/ICRTIT.2014.6996118.
  • W. Khreich, E. Granger, A. Miri, R. Sabourin. “Iterative Boolean combination of classifiers in the ROC space: An application to anomaly detection with HMMs”, Pattern Recognition, 43 (8), ss.2732-2752, 2010, doi: 10.1016/j.patcog.2010.03.006.
  • G. Kim, S. Lee, S. Kim. “A novel hybrid intrusion detection method integrating anomaly detection with misuse detection”, Expert Systems with Applications, 41(4), ss.1690-1700, 2014.
Konular Bilgisayar Mühendisliği ve Bilişim
Yayımlanma Tarihi August 2018
Dergi Bölümü Araştırma Makalesi
Yazarlar

Orcid: orcid.org/0000-0002-5761-3866
Yazar: Ferdi Sönmez (Sorumlu Yazar)
Kurum: İSTANBUL AREL ÜNİVERSİTESİ
Ülke: Turkey


Orcid: orcid.org/0000-0002-5761-3866
Yazar: Metin Zontul
Kurum: İSTANBUL AYDIN ÜNİVERSİTESİ

Orcid: orcid.org/0000-0002-5761-3866
Yazar: Oğuz Kaynar
Kurum: CUMHURİYET ÜNİVERSİTESİ

Orcid: orcid.org/0000-0002-5761-3866
Yazar: Hayati Tutar
Kurum: Türk Telekom

Bibtex @araştırma makalesi { saufenbilder365931, journal = {Sakarya University Journal of Science}, issn = {1301-4048}, eissn = {2147-835X}, address = {Sakarya Üniversitesi}, year = {2018}, volume = {22}, pages = {1109 - 1123}, doi = {10.16984/saufenbilder.365931}, title = {Anomaly Detection Using Data Mining Methods in IT Systems: A Decision Support Application}, key = {cite}, author = {Kaynar, Oğuz and Zontul, Metin and Tutar, Hayati and Sönmez, Ferdi} }
APA Sönmez, F , Zontul, M , Kaynar, O , Tutar, H . (2018). Anomaly Detection Using Data Mining Methods in IT Systems: A Decision Support Application. Sakarya University Journal of Science, 22 (4), 1109-1123. DOI: 10.16984/saufenbilder.365931
MLA Sönmez, F , Zontul, M , Kaynar, O , Tutar, H . "Anomaly Detection Using Data Mining Methods in IT Systems: A Decision Support Application". Sakarya University Journal of Science 22 (2018): 1109-1123 <http://www.saujs.sakarya.edu.tr/issue/31264/365931>
Chicago Sönmez, F , Zontul, M , Kaynar, O , Tutar, H . "Anomaly Detection Using Data Mining Methods in IT Systems: A Decision Support Application". Sakarya University Journal of Science 22 (2018): 1109-1123
RIS TY - JOUR T1 - Anomaly Detection Using Data Mining Methods in IT Systems: A Decision Support Application AU - Ferdi Sönmez , Metin Zontul , Oğuz Kaynar , Hayati Tutar Y1 - 2018 PY - 2018 N1 - doi: 10.16984/saufenbilder.365931 DO - 10.16984/saufenbilder.365931 T2 - Sakarya University Journal of Science JF - Journal JO - JOR SP - 1109 EP - 1123 VL - 22 IS - 4 SN - 1301-4048-2147-835X M3 - doi: 10.16984/saufenbilder.365931 UR - http://dx.doi.org/10.16984/saufenbilder.365931 Y2 - 2018 ER -
EndNote %0 Sakarya University Journal of Science Anomaly Detection Using Data Mining Methods in IT Systems: A Decision Support Application %A Ferdi Sönmez , Metin Zontul , Oğuz Kaynar , Hayati Tutar %T Anomaly Detection Using Data Mining Methods in IT Systems: A Decision Support Application %D 2018 %J Sakarya University Journal of Science %P 1301-4048-2147-835X %V 22 %N 4 %R doi: 10.16984/saufenbilder.365931 %U 10.16984/saufenbilder.365931
ISNAD Sönmez, Ferdi , Zontul, Metin , Kaynar, Oğuz , Tutar, Hayati . "BT Sistemlerinde Veri Madenciliği Yöntemlerini Kullanarak Anomali Algılama: Karar Destek Uygulaması". Sakarya University Journal of Science 22 / 4 (Ağustos 2018): 1109-1123. http://dx.doi.org/10.16984/saufenbilder.365931