Distributed Multi-Cloud Application Observability For Advanced Logging Using KEK stack and Docker

Gurnani, Kunal (2024) Distributed Multi-Cloud Application Observability For Advanced Logging Using KEK stack and Docker. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration Manual) Download (487kB) | Preview

Abstract

The quick development of data-escalated applications has elevated the interest for scalable and reliable answers for oversee and visualize real-time data streams. This undertaking tends to these challenges by carrying out a completely containerized architecture to streamline the integration, processing, and visualization of log data utilizing Docker and Docker Compose. The essential inspiration originates from the complexities associations face in deploying distributed systems, especially the requirement for efficient, scalable, and effectively sensible data pipelines. The goal was to plan and convey a vigorous architecture including Kafka for real-time log streaming, Elasticsearch for indexing, and Kibana for visualization, upheld by custom Python scripts for log generation and ingestion. The execution utilized Docker Compose to coordinate multiple services in a brought together climate, guaranteeing consistent integration and worked on sending. Custom Docker pictures were made for the Kafka producer and consumer scripts, empowering robotized log ingestion into Elasticsearch. The results exhibited effective real-time log ingestion and visualization, highlighting the capability of containerized answers for work on the arrangement and scaling of complicated distributed systems. This task gives a reusable and scalable framework for associations looking to use containerization for real-time data processing and visualization, eventually lessening functional above and further developing framework productivity.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Deshmukh, Sudarshan UNSPECIFIED
Uncontrolled Keywords:	Multi-cloud; observability; logging; elastic search; apache kafka; kibana; docker
Subjects:	Q Science > QA Mathematics > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Cloud computing
Divisions:	School of Computing > Master of Science in Cloud Computing
Depositing User:	Ciara O'Brien
Date Deposited:	15 Jul 2025 09:19
Last Modified:	15 Jul 2025 09:19
URI:	https://norma.ncirl.ie/id/eprint/8101

Actions (login required)

View Item