Large Language Model Powered Chatbot for Comprehensive Citizens Information Services in Ireland

Zanello Klostermann, Reinaldo (2023) Large Language Model Powered Chatbot for Comprehensive Citizens Information Services in Ireland. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (3MB) | Preview
Preview	PDF (Configuration manual) Download (4MB) | Preview

Abstract

This study explores the development and evaluation of a large language model (LLM)powered chatbot that employs GPT-4 to improve public information accessibility through WhatsApp. The chatbot uses the Citizens Information Ireland website as a data source. Creating the evaluation dataset is an automated process that ensures that the sample is unbiased and diverse. The chatbot was evaluated using ROUGE, BLEU, BERTScore, and UNIEVAL metrics based on 114 distinct question-answer pairs. A comprehensive analysis of the chatbot's performance was obtained based on mean scores and standard deviations. It demonstrates its ability to provide relevant, accurate, and context-appropriate responses. With a particular focus on improving public information accessibility, this study provides valuable contributions to the literature on LLM-enabled customer support, demonstrating the broader commercial potential of LLM-powered chatbots.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Milosavljevic, Vladimir UNSPECIFIED
Subjects:	H Social Sciences > HV Social pathology. Social and public welfare Q Science > QA Mathematics > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science Q Science > QH Natural history > QH301 Biology > Methods of research. Technique. Experimental biology > Data processing. Bioinformatics > Artificial intelligence Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Artificial intelligence
Divisions:	School of Computing > Master of Science in Data Analytics
Depositing User:	Tamara Malone
Date Deposited:	09 Jan 2025 16:09
Last Modified:	09 Jan 2025 16:09
URI:	https://norma.ncirl.ie/id/eprint/7293

Actions (login required)

View Item