Real-Time Multimodal AI for Continuous Mental Health Monitoring

Amate, Rajat Ranjit (2025) Real-Time Multimodal AI for Continuous Mental Health Monitoring. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration Manual) Download (2MB) | Preview

Abstract

Growing evidence links subtle changes in facial affect, speech prosody and ocular physiology with depression, anxiety, and cognitive fatigue. We present a preliminary multimodal pipeline that integrates facial emotion recognition, speech emotion analysis, and blink-based fatigue detection for mental health monitoring applications. Our implementation combines a MobileNet-v2 model trained on the MELD dataset for facial emotion recognition, a CNN-BiLSTM architecture trained on RAVDESS for speech emotion recognition, and MediaPipe FaceMesh for blink detection and fatigue assessment. The facial emotion component achieves real-time processing capabilities with seven emotion classes, while the speech emotion recognition system demonstrates 86.5% validation accuracy on eight RAVDESS emotion categories. We integrate these components with a text analysis pipeline using transformer-based models to create a foundation for comprehensive mental state monitoring. This preliminary study establishes the technical feasibility of real-time multimodal emotion recognition on commodity hardware and identifies key challenges for future clinical deployment.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Thomas, Lavish UNSPECIFIED
Subjects:	Q Science > QH Natural history > QH301 Biology > Methods of research. Technique. Experimental biology > Data processing. Bioinformatics > Artificial intelligence Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Artificial intelligence B Philosophy. Psychology. Religion > Psychology > Emotions R Medicine > RA Public aspects of medicine > RA790 Mental Health
Divisions:	School of Computing > Master of Science in Artificial Intelligence
Depositing User:	Ciara O'Brien
Date Deposited:	11 Aug 2026 14:44
Last Modified:	11 Aug 2026 14:44
URI:	https://norma.ncirl.ie/id/eprint/9494

Actions (login required)

View Item