Hybrid Machine Learning Approach Towards Anomaly Detection and Data Quality Assessment Of IoT Weather Data

Abhaykumar Kulkarni, Saket (2024) Hybrid Machine Learning Approach Towards Anomaly Detection and Data Quality Assessment Of IoT Weather Data. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration Manual) Download (939kB) | Preview

Abstract

The growing rate of climate change requires high standards of analysis of weather data. This research focuses on climate science and its data quality assessment though clustering and anomaly detection using traditional machine learning with a combination of PCA for dimensional reduction. By using IoT weather data from almost 1200 cities around the world, the hybrid model is built to detect anomalies and classify data according to its quality. The final percentage of anomalies found in the data are between 14-15 % of the entire data with an ROC of 0.70 and the anomalies are visualised which shows clear separation between the values. The hybrid model performed better compared to individual models with evaluations metrics of Homogeneity: 0.702, Completeness: 0.527, V-Measure: 0.602, Adjusted Rand Index: 0.713. The following model is a strong base to prove that there is space for traditional machine learning in this field rather that heavy deep learning models which require much higher computational power. The pre-trained models perform well and are a the base for future development in clustering analysis of weather data.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Singh, Jaswinder UNSPECIFIED
Uncontrolled Keywords:	Machine Learning; Climate Science; Clustering; Anomalies; Data Quality
Subjects:	G Geography. Anthropology. Recreation > GE Environmental Sciences Q Science > QA Mathematics > Electronic computers. Computer science T Technology > T Technology (General) > Information Technology > Electronic computers. Computer science T Technology > TK Electrical engineering. Electronics. Nuclear engineering > Telecommunications > Computer networks > Internet of things Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Machine learning
Divisions:	School of Computing > Master of Science in Data Analytics
Depositing User:	Ciara O'Brien
Date Deposited:	06 Aug 2025 14:40
Last Modified:	06 Aug 2025 14:40
URI:	https://norma.ncirl.ie/id/eprint/8448

Actions (login required)

View Item