Comparative Analysis of Parameter-Efficient Fine-Tuning and Traditional Continual Learning Methods

Hernandez Lopez, Jesus Moises (2025) Comparative Analysis of Parameter-Efficient Fine-Tuning and Traditional Continual Learning Methods. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration Manual) Download (771kB) | Preview

Abstract

Continual learning allows models to adapt to new data without being fully retrained. This often produce catastrophic forgetting, where performance on earlier tasks drops when new ones are learned. This thesis studies this problem in class-incremental text classification and a complementary vision experiment. Basically, it compares traditional continual learning baselines with parameter-efficient finetuning (PEFT) methods such as adapters and prompt tuning combined with and without replay or knowledge distillation.

The experiments indicate that replay-based PEFT methods retain earlier tasks more effectively and show more stable behaviour over task sequences than regularisation-based approaches and full fine-tuning, while requiring significantly fewer trainable parameters. A complementary study in a vision setting with a Vision Transformer (ViT) shows similar trends, suggesting that these advantages are not limited to text classification. Overall, the results highlight that focusing only on average accuracy can mask substantial degradation on early tasks, and that retention and computational cost should be considered jointly when evaluating continual learning methods.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Sahni, Vikas UNSPECIFIED
Subjects:	Q Science > QA Mathematics > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science Q Science > QH Natural history > QH301 Biology > Methods of research. Technique. Experimental biology > Data processing. Bioinformatics > Artificial intelligence Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Artificial intelligence P Language and Literature > P Philology. Linguistics > Computational linguistics. Natural language processing
Divisions:	School of Computing > Master of Science in Artificial Intelligence
Depositing User:	Ciara O'Brien
Date Deposited:	02 Sep 2026 09:35
Last Modified:	02 Sep 2026 09:35
URI:	https://norma.ncirl.ie/id/eprint/9759

Actions (login required)

View Item