Optimizing Data Storage through Neural Network Based Adaptive Compression

Noyola Sanchez, Carlos Alberto (2024) Optimizing Data Storage through Neural Network Based Adaptive Compression. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (876kB) | Preview
Preview	PDF (Configuration Manual) Download (676kB) | Preview

Abstract

Nowadays digital information is growing exponentially, with large amounts of information being generated and stored every second. This exponential growth presents significant challenges for data storage infrastructure. This is increasing the cost for storing data which impacts negatively on all kinds of businesses. Traditional compression algorithms are effective for specific data types but often fail when they are applied to mixed data types. Each file type—audio, video, text, or executable—presents unique byte-level patterns, which can be helpful in finding the best compression method.

This research addressed the problem of inefficient compression methods when an individual method is applied to mixed data types. This can lead to inefficient data storage and increase the costs. While data is growing, businesses need to find an efficient way to store data without sacrificing performance and quality.

To tackle this problem, a CNN based solution is proposed to identify and analyse patterns to byte level in files. By converting file data into grayscale images, a CNN can be trained to detect patterns that helps to identify the right compression method that offers the highest compression ratio from a predefined list of categories.

Results show that the CNN was able to infer the compression mechanism for different files by analysing patterns in the grayscale images, generated from the mixed data files. The experiments run showed no noticeable differences when the dimensions of the images were changed or when more layers were used. A 96% of accuracy was obtained in overall for the experiments performed.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Sahni, Vikas UNSPECIFIED
Subjects:	Q Science > QA Mathematics > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Cloud computing Q Science > QA Mathematics > Algebra > Algorithms > Computer algorithms Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Machine learning
Divisions:	School of Computing > Master of Science in Cloud Computing
Depositing User:	Ciara O'Brien
Date Deposited:	04 Jul 2025 09:16
Last Modified:	04 Jul 2025 09:16
URI:	https://norma.ncirl.ie/id/eprint/8043

Actions (login required)

View Item