Multimodal Deep Learning

← Back to document page

To train a multimodal model, a direct approach is to train a RBM over the concatenated audio and video data (Figure 2c). While this approach jointly mod-els the distribution of the audio and video data, it is limited as a shallow model. In particular, since the cor-relations between the audio and video data are highly

  Model, Approach, Learning, Deep, Multimodal, Multimodal deep learning