Transformers for Malware Detection through Machine Learning & Deep Learning

Angala, Vamshi Krishna (2024) Transformers for Malware Detection through Machine Learning & Deep Learning. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (3MB) | Preview
Preview	PDF (Configuration Manual) Download (765kB) | Preview

Abstract

The application of transformers, a prominent deep learning architecture in natural language processing (NLP), extends beyond traditional text-based tasks to encompass malware detection based on raw byte sequences. In this project, we explore the adaptation of transformers for byte-level sequence analysis in the realm of malware detection. Executable files, viewed as sequences of bytes, present an opportunity to leverage transformers by treating each byte as a token in the input sequence. Despite the computational and data-intensive nature of transformers, a breakthrough methodology is introduced in our recent research paper, ”Certified Robustness of Static Deep Learning-based Malware Detectors against Patch and Append Attacks,” presented at AISEC’23. This innovative approach addresses the challenges posed by the immense size of byte sequences—often in the order of millions of bytes—by strategically dividing executable files into manageable chunks of 500 bytes. Each chunk is then independently classified, and the final detection score is derived from the ratio of malicious chunks to the total number of chunks. This novel approach not only renders transformers feasible for malware detection but also introduces a robustness certification mechanism against diverse attacks employed by malware authors to elude detection. The groundbreaking shift from processing massive byte sequences to the analysis of smaller, more manageable chunks opens new avenues for enhancing the efficiency and scalability of malware detection using transformer-based models.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Trinh, Anh Duong UNSPECIFIED
Uncontrolled Keywords:	Deep Learning Architecture; NLP; Malware Detection
Subjects:	Q Science > QA Mathematics > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science Q Science > QA Mathematics > Computer software > Computer Security T Technology > T Technology (General) > Information Technology > Computer software > Computer Security Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Machine learning
Divisions:	School of Computing > Master of Science in Artificial Intelligence
Depositing User:	Ciara O'Brien
Date Deposited:	30 May 2025 14:04
Last Modified:	30 May 2025 14:04
URI:	https://norma.ncirl.ie/id/eprint/7713

Actions (login required)

View Item