Modalities Missing in Multimodal Sentiment Analysis

Abstract

In recent years, sentiment analysis has been widely applied in various fields such as smart home, intelligent healthcare, and education. Multiple technologies such as eye movement recognition and text analysis are employed as means to determine the emotional state of users. However, during this process, factors such as sensor failures, privacy restrictions, and environmental disturbances can all lead to the problem of modal absence in sentiment analysis. Traditional multimodal approaches often fail to handle the correlations between different modalities during feature extraction, and when integrating features, they often neglect multi-scale emotional cues. This research aims to summarize typical multimodal feature fusion models. Based on four optimized models developed in recent years, it aims to summarize and outline the technical breakthroughs in the emerging model architectures in addressing the issue of modality absence in multimodal sentiment analysis, and also makes assumptions and prospects for future multimodal sentiment analysis in dealing with modality absence.