Detecting Customer Dissatisfaction in Support Chats Using AI-Based Sentiment Analysis

Lemos, Diego Amaxmandro De Oliveira (2025) Detecting Customer Dissatisfaction in Support Chats Using AI-Based Sentiment Analysis. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration Manual) Download (766kB) | Preview

Abstract

Early detection of a customer’s dissatisfaction during support chat conversations is essential to improve clients retention and experience. Despite of the advances of Artificial intelligence (AI), identify customer’s dissatisfaction in a short and informal conversation is still a challenge, especially under weak supervision, where labelled data is scarce. This research examine whether AI-based sentiment analysis is capable of detect effectively a dissatisfaction in support chats before an escalation occurs. Weak supervision labelling throughout lexicon-based and transformer-based sentiment models were used on a public available dataset with over 3 million interaction between support agents and customers on Twitter. We have developed and evaluated three modelling pipeline, one with traditional machine learning model such as Logistic Regression, Random Forest and Support Vector Machine, they were trained on PCA-reduced BERT embeddings, second we used the same models but now they were trained on TF-IDF feature derived from raw text, and the last, a Bidirectional Long Short-Term Memory (BiLSTM) network was trained on tokenised text sequences. The results showed that BiLSTM outperformed all other traditional approaches reaching a precision of 84.45%, demonstrating the effectiveness of the sequential deep learning on informal conversational data. Even though the traditional machine learning using TF-IDF showed competitive results, their performance dropped dramatically when using embeddings with PCA reduction, highlighting the risks of dimensionality reduction in preserving semantic data. These findings emphasise the importance of sequential based models and context-awareness for dissatisfaction detection in customer support environment. Future researches should focus on real time implementations and multilingual capabilities to extend its applicability.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Anand, Devanshu UNSPECIFIED
Subjects:	Q Science > QH Natural history > QH301 Biology > Methods of research. Technique. Experimental biology > Data processing. Bioinformatics > Artificial intelligence Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Artificial intelligence P Language and Literature > P Philology. Linguistics > Computational linguistics. Natural language processing H Social Sciences > HF Commerce > Marketing > Consumer Behaviour H Social Sciences > HF Commerce > Customer Service Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Machine learning
Divisions:	School of Computing > Master of Science in Artificial Intelligence
Depositing User:	Ciara O'Brien
Date Deposited:	02 Jun 2026 11:11
Last Modified:	02 Jun 2026 11:11
URI:	https://norma.ncirl.ie/id/eprint/9330

Actions (login required)

View Item