End-to-end attack detection based on ML and spark

Kumbhar, Ajay Ashok (2022) End-to-end attack detection based on ML and spark. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (670kB) | Preview
Preview	PDF (Configuration manual) Download (743kB) | Preview

Abstract

The main area of concern in the IT environment is the security. There are various attack occurs on the end devices and the networking devices. However, in the current market there are various methodology is present to identify and prevent the attack. Research implemented numerous prevention technique based on the machine learning and the deep learning, however, the accuracy and the prediction rate of some algorithm was good and accurate. Moreover, prediction of attack on the end device is crucial part because most of the attack occurs on the edge devices such as firewall and router. There is so many research is occurred on the intrusion detection system and machine learning is the proposed model from various researcher. Machine learning is the superior and accurately detect or predict the input. To build the machine learning dataset is required and machine learning train by using the available dataset. Apache spark is the huge data processing framework which helps to process the data which is generated in the device. The uses of spark are more beneficial in real time environment where traffic is recorded continuously in the machine.

The aim of this project is to build the end-to-end model to detect the attack. In the proposed system Spark, Random Forest, Decision tree and Binary classification has been implemented. The implemented project is used to detect the attack on the end device and provide the real time notification to the administrator department for the further action. Spark is the most important part of this project which is used to process the larger amount of dataset quickly. Frontend socket is implemented to transfer the captured traffic to centralised server for the prediction of the attack and based on the result of machine learning is generate the log file for specific attacked traffic. Evaluation has been performed for the proposed model and generated values are recorded for the future and to understand the best model. After training and testing we have gathered 97 % accuracy for the Binary classification and 99 percent for the Random Forest and Decision Tree.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Moldovan, Arghir-Nicolae UNSPECIFIED
Uncontrolled Keywords:	K-Nearest Neighbour; Transmission control Protocol; User Datagram Protocol
Subjects:	Q Science > QA Mathematics > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science Q Science > QA Mathematics > Computer software > Computer Security T Technology > T Technology (General) > Information Technology > Computer software > Computer Security Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Machine learning
Divisions:	School of Computing > Master of Science in Cyber Security
Depositing User:	Tamara Malone
Date Deposited:	28 Apr 2023 16:02
Last Modified:	28 Apr 2023 16:02
URI:	https://norma.ncirl.ie/id/eprint/6528

Actions (login required)

View Item