Fine-Tuning Bert Model for Intent Classification task using GAN

Kule, Sayali Suresh (2022) Fine-Tuning Bert Model for Intent Classification task using GAN. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration manual) Download (3MB) | Preview

Abstract

The intelligent chatbot systems are the result of natural language processing and neural network algorithm. A chatbot is nothing but a virtual human-like assistant, that helps in completing a task or provides required service in no time through a conversation. The task-oriented dialog systems need to classify the query intent correctly to avoid incorrect responses. Even though the issue of a low data regime is common, this study proposes the use of the pre-trained transformer model BERT which is fine-tuned under adversarial settings for the intent classification task. This unique method makes use of a few shots of supervised data and the GAN model allows the adaptation of unsupervised data which is available easily. With the experimental comparison and validation, the performance of the model was found to improve when fewer examples were trained for longer epochs. The size of annotated examples can be reduced up to 60, still obtaining an accuracy of 94.8% and an F1 score of 0.95. This approach successfully allowed us to achieve high performance even with low resources.

Item Type:	Thesis (Masters)
Subjects:	Q Science > QA Mathematics > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science Q Science > QH Natural history > QH301 Biology > Methods of research. Technique. Experimental biology > Data processing. Bioinformatics > Artificial intelligence Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Artificial intelligence P Language and Literature > P Philology. Linguistics > Computational linguistics. Natural language processing
Divisions:	School of Computing > Master of Science in Data Analytics
Depositing User:	Tamara Malone
Date Deposited:	21 Feb 2023 17:33
Last Modified:	02 Mar 2023 09:40
URI:	https://norma.ncirl.ie/id/eprint/6207

Actions (login required)

View Item