Advanced Search
LIU Yunxiang, ZHANG Kexin. Bimodal cross-corpus speech emotion recognition[J]. Journal of Technology, 2024, 24(1): 77-84. DOI: 10.3969/j.issn.2096-3424.2024.01.008
Citation: LIU Yunxiang, ZHANG Kexin. Bimodal cross-corpus speech emotion recognition[J]. Journal of Technology, 2024, 24(1): 77-84. DOI: 10.3969/j.issn.2096-3424.2024.01.008

Bimodal cross-corpus speech emotion recognition

  • In the field of speech emotion recognition(SER), a heterogeneity gap exists between different modalities and most cross-corpus SER only uses the audio modality. These issues were addressed simultaneously. YouTube datasets were selected as source data and an interactive emotional dyadic motion capture database (IEMOCAP) as target data. The Opensmile toolbox was used to extract speech features from both source and target data, then the extracted speech features were input into Convolutional Neural Network (CNN) and bidirectional long short-term memory network (BLSTM) to extract higher-level speech features with the text mode as the translation of speech signals. Firstly, BLSTM was adopted to extract the text features from text information vectorized by Bidirectional Encoder Representation from Transformers (BERT), then modality-invariance loss was designed to form a common representation space for the two modalities. To solve the problem of cross-corpus SER, a common subspace of source data and target data were learned by optimizing Linear Discriminant analysis (LDA), Maximum Mean Discrepancy (MMD) and Graph Embedding (GE) and Label Smoothing Regularization (LSR) jointly. To preserve emotion-discriminative features, emotion-aware center loss was combined with MMD+GE+LDA+LSR. The SVM classifier was designed as a final emotion classification for migrating common subspaces. The experimental results on IEMOCAP showed that this method outperformed other state-of-art cross-corpus and bimodal SER.
  • loading

Catalog

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return
    摩臣5娱乐专业提供⬇️:摩臣5娱乐🏋🏼‍♀️🫶🏻、摩臣5🎤、摩臣5平台等服务,提供最新官网平台、地址、注册、登陆、登录、入口、全站、网站、网页、网址、娱乐、手机版、app、下载、欧洲杯、欧冠、nba、世界杯、英超等,界面美观优质完美,安全稳定,服务一流,摩臣5娱乐欢迎您。 摩臣5娱乐官網xml地圖