Leveraging Self-Attention in Transformer Models for Improved Retail Demand Forecasting

Sardena, Rohith (2025) Leveraging Self-Attention in Transformer Models for Improved Retail Demand Forecasting. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration Manual) Download (2MB) | Preview

Abstract

Inventory management and operation planning in retail sector involve accurate demand forecasting as a key element of the process. Linear models and tree-based models are typical examples of traditional forecasting models that may not work well in capturing complex, non-linear, and time-evolving demands that retail demand exhibits especially when it is affected by promotions, seasonal trends, and other external market factors. The study examines the possibility of deep learning models based on Transformers, which will use self-attention mechanisms, to improve the accuracy and scalability of retail demand forecasting systems. Model development was done using a detailed retail dataset with historical sales, inventory, pricing, discounts, promotions, weather conditions, and other contextual characteristics. A number of models were applied and tested in the study: Linear Regression, Decision Tree, Random Forest, Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), and Transformer. Mean Absolute Error (MAE), Root Mean Squared Error (RMSE) and Median Absolute Error (MedianAE) were used to evaluate performance. The classical models were moderate in their performance, and Linear Regression and Random Forest injected MAEs of 7.47 and 7.55 respectively, and RMSEs of more than 8.6. Deep learning models, on the contrary, showed a dramatic increase. Transformer model performed the best and its MAE was 0.81, RMSE was 0.99, and MedianAE was 0.74, which are better than LSTM (MAE: 0.83) and GRU (MAE: 0.83). The results verify that self-attention mechanism of Transformer is capable of learning long-range dependencies and complex temporal interactions, giving significant improvement on the baseline models. This study shows the ability of Transformers to be an efficient and scalable method of real-time retail demand forecasting. Further development will consider multi-step forecasting, probabilistic additions and deployment in other retail areas.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Kelly, John UNSPECIFIED
Subjects:	Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Machine learning H Social Sciences > HD Industries. Land use. Labor > Specific Industries > Retail Industry
Divisions:	School of Computing > Master of Science in Data Analytics
Depositing User:	Ciara O'Brien
Date Deposited:	26 Aug 2026 11:34
Last Modified:	26 Aug 2026 11:34
URI:	https://norma.ncirl.ie/id/eprint/9664

Actions (login required)

View Item