Beyond Text: How Images Compensate for Missing Quantitative Data in Multimodal Nutrition Estimation

Chen, Bintong (2025) Beyond Text: How Images Compensate for Missing Quantitative Data in Multimodal Nutrition Estimation. Masters thesis, Dublin, National College of Ireland.

Preview	PDF (Master of Science) Download (1MB) | Preview
Preview	PDF (Configuration Manual) Download (883kB) | Preview

Abstract

Motivated by practical challenges in obtaining precise ingredient mass data for nutritional estimation, this study investigates how images can compensate for missing quantitative data within a multimodal framework. Three parallel pipelines, text-only, image-only, and multimodal, are implemented using domain-specific models: DistilBERT for text and ResNet18 for images. This study also explores various multimodal fusion architectures to identify the most effective strategy to combine data modalities. Key findings reveal that when precise mass data is available, text-only models achieve the highest accuracy. However, in its absence, multimodal models with gated fusion (per-feature weights) compensate for missing quantitative data effectively, reducing calorie estimation errors by 2.0% MAE. Furthermore, the results highlight fat estimation as the most challenging task and emphasise the importance of tailored fusion architectures in nutritional estimation. Overall, these insights demonstrate the potential of multimodal systems to enhance both the flexibility and accuracy of dietary monitoring.

Item Type:	Thesis (Masters)
Supervisors:	Name Email Shahid, Abdul UNSPECIFIED
Subjects:	Q Science > QH Natural history > QH301 Biology > Methods of research. Technique. Experimental biology > Data processing. Bioinformatics > Artificial intelligence Q Science > Q Science (General) > Self-organizing systems. Conscious automata > Artificial intelligence Q Science > QP Physiology > Nutrition R Medicine > RA Public aspects of medicine > RA0421 Public health. Hygiene. Preventive Medicine > Personal Health and Hygiene
Divisions:	School of Computing > Master of Science in Artificial Intelligence
Depositing User:	Ciara O'Brien
Date Deposited:	04 Jul 2026 13:40
Last Modified:	04 Jul 2026 13:40
URI:	https://norma.ncirl.ie/id/eprint/9470

Actions (login required)

View Item