Evaluating Large Language Models for Sustainability-Aware Code Selection Using Pairwise Efficiency Comparison

Pereira, Cedric John Savio (2025) Evaluating Large Language Models for Sustainability-Aware Code Selection Using Pairwise Efficiency Comparison. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (997kB) | Preview
Preview	PDF (Configuration Manual) Download (1MB) | Preview

Abstract

The purpose of this study is to determine whether large language models (LLMs) can accurately determine which of two semantically comparable programs has a more energy-efficient implementation. We build pairwise comparisons and extract ground-truth labels from execution time, memory use, and code size using a filtered subset of Project CodeNet. A controlled A/B prompt is used to assess the performance of four code-oriented LLMs using accuracy, ranking metrics, and Energy Gap (EGAP). The findings demonstrate that while pairwise accuracy is close to chance for all models, agreement behavior and sensitivity to efficiency variations vary significantly. The results point out current shortcomings in the efficiency reasoning of LLMs and suggest ways to enhance sustainability-aware code analysis.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Razzaq, Abdul UNSPECIFIED
Subjects:	Q Science > QA Mathematics > Computer software T Technology > T Technology (General) > Information Technology > Computer software Q Science > QH Natural history > QH301 Biology > Methods of research. Technique. Experimental biology > Data processing. Bioinformatics > Artificial intelligence Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Artificial intelligence H Social Sciences > HC Economic History and Conditions > Natural resources > Power resources > Energy consumption
Divisions:	School of Computing > Master of Science in Data Analytics
Depositing User:	Ciara O'Brien
Date Deposited:	08 Sep 2026 11:35
Last Modified:	08 Sep 2026 11:35
URI:	https://norma.ncirl.ie/id/eprint/9894

Actions (login required)

View Item