Sentimental Effects of Temperature Setting After QLoRA Fine-Tuning in LLAMA 2 7B and 13B models

Yildiz, Melih (2024) Sentimental Effects of Temperature Setting After QLoRA Fine-Tuning in LLAMA 2 7B and 13B models. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration manual) Download (334kB) | Preview

Abstract

This research investigates the impact of temperature settings and model size on the performance of AI language models, specifically focusing on LLAMA 2 7B and 13B models post-QLoRA fine-tuning. The study is driven by the need to understand how varying conditions affect these models’ abilities to replicate known answers. The methodology includes generating data and question-answer pairs using GPT-4, fine-tuning LLAMA 2 models, and then evaluating their performance using various statistical analyses. The research’s novel contribution lies in its comprehensive approach to evaluating AI language models under different conditions. It highlights the complexities in optimizing these models and underscores the importance of considering multiple factors in their deployment and operationalization. The findings provide valuable insights for future research, particularly in exploring diverse datasets and conditions to enhance model robustness and generalizability.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Syed, Muslim Jameel UNSPECIFIED
Subjects:	Q Science > QA Mathematics > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science Q Science > QH Natural history > QH301 Biology > Methods of research. Technique. Experimental biology > Data processing. Bioinformatics > Artificial intelligence Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Artificial intelligence P Language and Literature > P Philology. Linguistics > Computational linguistics. Natural language processing
Divisions:	School of Computing > Master of Science in Artificial Intelligence
Depositing User:	Tamara Malone
Date Deposited:	07 Apr 2025 11:00
Last Modified:	07 Apr 2025 11:00
URI:	https://norma.ncirl.ie/id/eprint/7378

Actions (login required)

View Item