Improving Kubernetes Autoscaling Through a Heuristic Resource-Aware Scheduler for ML Applications

Konda, Saiteja (2025) Improving Kubernetes Autoscaling Through a Heuristic Resource-Aware Scheduler for ML Applications. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration Manual) Download (1MB) | Preview

Abstract

Machine learning workloads operating in Kubernetes environments often experience performance inefficiencies due to unpredictable CPU bursts, delayed scaling responses, and static scheduling behaviour, making traditional mechanisms such as the default Kubernetes scheduler and Horizontal Pod Auto-scaler inadequate for rapidly fluctuating compute demands. Existing solutions rely heavily on averaged metrics and generic rule-based placement, limiting their suitability for real-time ML workloads. To address these challenges, this work introduces a Prometheus-driven heuristic-based custom scheduler with integrated autoscaling logic, designed to make informed scheduling decisions using real-time CPU and memory metrics. The system was implemented within a Kubernetes cluster using Docker-containerized ML workloads, incorporating node scoring, threshold-based scale-out decisions, and real-time metric evaluation to support dynamic resource allocation. A 60% CPU threshold was defined so that when usage exceeded this limit, the scheduler intelligently created additional replicas. Grafana dashboards were used to visualize resource patterns. Experiments were conducted over 7-minute, 15-minute, and 30-minute intervals, and results consistently showed more responsive scaling behaviour, smoother CPU utilization trends, and improved pod stability compared to the default scheduler. The evaluation indicates that the proposed scheduler offers improved responsiveness, enhanced resource utilization efficiency, and more stable workload handling compared to traditional Kubernetes autoscaling and scheduling mechanisms.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Kazmi, Aqeel UNSPECIFIED
Uncontrolled Keywords:	Kubernetes; Custom Scheduler; Prometheus; Auto-scaling; ML Workloads
Subjects:	T Technology > T Technology (General) > Information Technology > Cloud computing Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Machine learning
Divisions:	School of Computing > Master of Science in Cloud Computing
Depositing User:	Ciara O'Brien
Date Deposited:	31 Aug 2026 15:34
Last Modified:	31 Aug 2026 15:34
URI:	https://norma.ncirl.ie/id/eprint/9715

Actions (login required)

View Item