Tools
Tools
Ahmed, Mohammed Sameer (2025) Evaluating Deep Learning Models for Cross-domain Propaganda Detection. Masters thesis, Dublin, National College of Ireland.
Preview |
PDF (Master of Science)
Download (727kB) | Preview |
Preview |
PDF (Configuration Manual)
Download (392kB) | Preview |
Abstract
Propaganda is a major issue in a world generating data and information exponentially. It has a huge impact on our societies and potential to influence people. With advances in technology, the propaganda techniques can be detected and tackled, and previous works have given some excellent results in doing so. While these studies have given applaudable results they are restricted to one single domain of information. Very little work has been done to generalize the propaganda detection and to address this gap this study evaluates performance for four deep learning models (LSTM, BERT, XLNet and GPT-2). Each model is trained on one domain of information and tested to detect propaganda in information form other domains. The results reveal that these models perform detection significantly better in same domain than a different domain. The Accuracy and F1 are high even with limited resources but this drops sharply in a cross-domain setup. Transformer-based models, specifically BERT (Accuracy: 93%, F1: 98% on speech articles) and XLNet (Accuracy: 97%, F1: 98% on news articles), outperform the traditional LSTM (Accuracy: 91%, F1: 92% on speeches) but still struggle to get acceptable scores in cross-domain detection. GPT-2 (Accuracy 90%, F1: 90% on tweets) a generative model, included for comparison did not perform any better than BERT and XLNet. These results suggest that the current models are not adaptable for reliable propaganda detection in different domains. This study therefore lays out a framework and baseline for future studies in this area by highlighting the limitation in a real-world cross-domain setup.
Actions (login required)
 |
View Item |