Evaluating Trade-offs Between Performance and Resource Utilization in Retrieval-Augmented Generation and Finetuning for Domain-Specific Language Models

Ramani, Aryan (2025) Evaluating Trade-offs Between Performance and Resource Utilization in Retrieval-Augmented Generation and Finetuning for Domain-Specific Language Models. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration Manual) Download (820kB) | Preview

Abstract

This report investigates the trade-offs between Retrieval Augmented Generation (RAG) and fine-tuned Language Models for a domain-specific question answering task. A high-performing large language model (LLM) delivers practical value only when it can be deployed within the constraints of the target environment. We focus on NASA technical reports to generate a dataset, a perfect fit for jargon-rich domain. First, we extract the content of the PDFs and generate corresponding questions. Secondly, we conduct experiments using simple RAG architecture and LLMs finetuned exclusively on our dataset. Lastly, we evaluate both the approaches based on the standard performance metrics as well as the computation resources consumed to produce the associated results. The performance analysis aspect of our research highlights the strength and weaknesses of each method in handling domain queries. Whereas resource monitoring ensure that the techniques are efficient and scalable enough to be utilized in real-life practices given the resource constraint environments. Based on the results, we recommend the more effect approach while providing practical guidelines for choosing retrieval-based and finetuning-based strategies, balancing performance with the available resources.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Shahid, Abdul UNSPECIFIED
Subjects:	Q Science > QH Natural history > QH301 Biology > Methods of research. Technique. Experimental biology > Data processing. Bioinformatics > Artificial intelligence Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Artificial intelligence P Language and Literature > P Philology. Linguistics > Computational linguistics. Natural language processing
Divisions:	School of Computing > Master of Science in Artificial Intelligence
Depositing User:	Ciara O'Brien
Date Deposited:	04 Jun 2026 15:05
Last Modified:	04 Jun 2026 15:05
URI:	https://norma.ncirl.ie/id/eprint/9341

Actions (login required)

View Item