Voice to Value: Multimodal Emotion Analysis for Smarter Customer Interactions

Ramapatnam, Bharathi (2025) Voice to Value: Multimodal Emotion Analysis for Smarter Customer Interactions. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (2MB) | Preview
Preview	PDF (Configuration Manual) Download (740kB) | Preview

Abstract

Sentiment analysis has been a major focus in natural language processing, traditionally centered on text. However, in voice-driven interactions across healthcare, education, and customer service, text alone often misses emotional cues present in speech. This research introduces a multimodal sentiment analysis framework that independently analyzes speech emotion and text sentiment to capture a fuller emotional profile. Deep learning models—including 1D and 2D Convolutional Neural Networks (CNNs), Random Forest, and Recurrent Neural Networks (RNNs)—are applied to the RAVDESS and EMOGATOR datasets. Audio features are extracted using Mel-frequency cepstral coefficients (MFCCs), while textual sentiment is analyzed using transformer-based models. The 2D CNN outperforms baseline models, achieving 71% accuracy on RAVDESS and 63% on EMOGATOR, compared to 61% from Random Forest. AssemblyAI’s speech-to-text API enables accurate transcription, with the full system integrated into a web application for real-time emotion tracking. A total of 13 emotion classes are classified. Results show that parallel analysis of audio and text provides complementary insights into user sentiment. This approach offers practical benefits for customer relationship management (CRM) systems and healthcare platforms, enabling more responsive and personalized services based on emotional understanding.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Chikkankod, Arjun UNSPECIFIED
Subjects:	Q Science > QH Natural history > QH301 Biology > Methods of research. Technique. Experimental biology > Data processing. Bioinformatics > Artificial intelligence Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Artificial intelligence P Language and Literature > P Philology. Linguistics > Computational linguistics. Natural language processing H Social Sciences > HF Commerce > Marketing > Consumer Behaviour B Philosophy. Psychology. Religion > Psychology > Emotions
Divisions:	School of Computing > Master of Science in Data Analytics
Depositing User:	Ciara O'Brien
Date Deposited:	03 Jul 2026 09:41
Last Modified:	03 Jul 2026 09:41
URI:	https://norma.ncirl.ie/id/eprint/9453

Actions (login required)

View Item