Comparative Analysis of Transformer Models for Multi-Class Text Classification

Neelakanti, Sharanya (2024) Comparative Analysis of Transformer Models for Multi-Class Text Classification. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration Manual) Download (739kB) | Preview

Abstract

This paper makes a comparative evaluation of five state-of-the-art transformer models in multi-class emotion recognition: BERT, GPT-3.5, RoBERTa, XLNet, and DistilBERT. Motivated by the demand for detecting emotions with accuracy in so many applications today, this research aimed at comparing these models on accuracy, precision, recall, and F1 score on classifying texts into multiple categories of emotions.

The research employed the usage of the GoEmotions dataset, which is a dataset containing 58,000 Reddit comments with 27 different annotated emotions and consolidated into three major classes, i.e., positive, neutral, and negative. The methodology in this research undertook preprocessing for the dataset, model implementation, and fine-tuning, ending up at the point of developing a comprehensive evaluation framework.

Key findings were that there did exist a performance hierarchy and, quite unexpectedly, DistilBERT outstripped all larger models, scoring 95.88%. Following were RoBERTa, XLNet, BERT, and GPT-3.5, performing in descending order. For all models, in comparison to the neutral or negative ones, recognizing positive emotions was easier. A remarkable exception was GPT-3.5, which, though doing splendidly elsewhere in NLP applications, underperformed in the given task.

This paper aids in adding to this literature by disputing the commonly held belief that improvements in NLP tasks are made when the model's size is increased and focusing on the compression methods of models. The findings have implications for academic research in NLP and for the practical applications of emotion recognition systems, mainly scenarios related to high computational efficiency.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Haque, Rejwanul UNSPECIFIED
Subjects:	Q Science > QA Mathematics > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science P Language and Literature > P Philology. Linguistics > Computational linguistics. Natural language processing
Divisions:	School of Computing > Master of Science in Artificial Intelligence
Depositing User:	Ciara O'Brien
Date Deposited:	18 Jun 2025 14:11
Last Modified:	18 Jun 2025 14:11
URI:	https://norma.ncirl.ie/id/eprint/7921

Actions (login required)

View Item