Tools
Tools
Vora, Vyom Mehul (2025) Low-Latency Cold Start Optimization for Serverless Machine Learning Inference. Masters thesis, Dublin, National College of Ireland.
Preview |
PDF (Master of Science)
Download (1MB) | Preview |
Preview |
PDF (Configuration Manual)
Download (2MB) | Preview |
Abstract
Serverless computing offers cost-efficient computation especially in the case of machine learning models but it has severe drawbacks, one of them being cold start problem. Model’s initialization takes 5-15 seconds which can cause delays in inference. This research proposes a generalized framework that addresses the cold start problem through prewarming of serverless containers based on their usage pattern. Unlike existing solutions which are platform specific, our framework combines multi-cloud solution. The framework is evaluated on three serverless functions AWS Lambda, Azure Functions, Google Cloud Run using three diverse ML models including natural language processing model DistilBert, image classification model Resnet-50 and Random Forest Classifier. Experimental results shows an overall 30.9% reduction in cold start timings from 45.9% to 15.1% with prewarming containers and model achieving 65% prediction accuracy. Platform specific reductions were 44.4% on AWS, 23.1% on Azure and 11.1% on GCP. This research also contributes to reducing gap between platform specific optimizations and the need for generalized framework. The framework also eliminates the need for platform based solutions for machine learning model deployment and serving inference.
| Item Type: | Thesis (Masters) |
|---|---|
| Supervisors: | Name Email Kazmi, Aqeel UNSPECIFIED |
| Subjects: | T Technology > T Technology (General) > Information Technology > Cloud computing Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Machine learning |
| Divisions: | School of Computing > Master of Science in Cloud Computing |
| Depositing User: | Ciara O'Brien |
| Date Deposited: | 01 Sep 2026 12:00 |
| Last Modified: | 01 Sep 2026 12:00 |
| URI: | https://norma.ncirl.ie/id/eprint/9747 |
Actions (login required)
 |
View Item |