AI-Driven Unified QA Framework for Serverless APIs

Sundar, Adhithya Narendran (2025) AI-Driven Unified QA Framework for Serverless APIs. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration Manual) Download (971kB) | Preview

Abstract

Serverless computing drastically simplifies the delivery of cloud applications but because of its short-lived execution model, cold starts, and limited visibility the quality assurance for the same becomes difficult. Traditional CI/CD pipelines execute static test suites that in turn makes it difficult or unable to respond to runtime problems, Hence this research is the development of an AI-Driven Unified QA Framework for Serverless APIs that unifies and integrates functional (PyTest), behavioural (Robot Framework), and performance testing (Locust) under a single, adaptive workflow driven by an intelligent QA Controller. Two AWS Lambda workloads, a CRUD API and a dockerised machine learning inference API, were instrumented with custom CloudWatch metrics, such as RequestsProcessed, ColdStartCount, InferenceLatency and ModelColdStartCount. These metrics have been exported to Prometheus and visualised with Grafana and Streamlit dashboard. The Adaptive AI - QA Controller employed a combination of rules and a simple regression model to identify the anomalies and automatically trigger specific retesting within the CI/CD pipeline. Experiments demonstrated that the adaptive pipeline was able to keep all functional and behavioural tests passing and reduce the end-to-end CI/CD time by 15 - 20 % Under load, the adaptive mode also resulted in a consistent 4-5 reduction of mean inference latency and the regression-based anomaly detector led to a 23 %reduction of false positives and a 30 % faster reaction than simple threshold checks. These are some of the improvements that show how combining observability data with automated, AI-guided testing produces a more reliable and self-correcting QA process for serverless systems. The framework demonstrates that continuous, metrics-driven adaptation can improve the stability of performance and minimize the time spent on manual QA and build confidence in deployment. It also forms a practical basis for future work in test selection using reinforcement learning, multi-cloud observability and more autonomous DevOps-based pipelines.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Heeney, Sean UNSPECIFIED
Uncontrolled Keywords:	Serverless Computing; AWS Lambda; CI/CD Pipelines; Cloud Metrics; CloudWatch; Prometheus; Grafana; ML Inference API; Adaptive AI QA controller Adaptive Testing; Anomaly Detection; PyTest; Robot Framework; Locust
Subjects:	Q Science > QH Natural history > QH301 Biology > Methods of research. Technique. Experimental biology > Data processing. Bioinformatics > Artificial intelligence Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Artificial intelligence T Technology > T Technology (General) > Information Technology > Cloud computing Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Machine learning
Divisions:	School of Computing > Master of Science in Cloud Computing
Depositing User:	Ciara O'Brien
Date Deposited:	01 Sep 2026 11:31
Last Modified:	01 Sep 2026 11:31
URI:	https://norma.ncirl.ie/id/eprint/9740

Actions (login required)

View Item