Depression Detection Method Based on Multimodal Emotion Recognition

Abstract

Depression screening traditionally relies on subjective questionnaires, which face limitations such as low efficiency and high risk of misdiagnosis. To address this, this study proposes a real-time depression detection system based on multimodal emotion recognition, integrating speech and micro-expression features. The system leverages prosodic and acoustic biomarkers in speech, along with micro-expression analysis via the FACS framework, to enhance objectivity and reliability. Two representative models are compared in this study: the MFM-Att model, which adopts multi-level attention to achieve comprehensive multimodal fusion, and the Q-learning model, which applies reinforcement learning to support adaptive and lightweight detection. The experiments demonstrate that multimodal fusion methods outperform single-modality approaches, with the MFM-Att model showing higher accuracy and the Q-learning model exhibiting better real-time adaptability. To further balance accuracy and efficiency, the optimized framework integrates MobileNet-ViT with adaptive weight adjustment. Overall, this work underscores the potential of multimodal emotion recognition in early depression screening and provides technical support for scalable clinical applications and personalized mental health interventions.