Espnet实践 - Xiamen University

1 espnet .. espnet . 2018 espnet espnet end-to-end speech processing toolkit . ASR TTS . [ espnet ]. 1 Kaldi . 2 Pytorch Chainer Python E2E . [ espnet ]. CTC. Attention RNN-T. Transformer espnet .. 1. git espnet Kaldi . 2. cuda CUDADNN . 3. Pytorch cuda .. 1. cuda ~/.bashrc cuda . source ~/.bashrc . 2. Kaldi 13 . 3. Kaldi espnet . espnet tools . make KALDI=/path_to_kaldi make check_install . 4. espnet egs . espnet . RNN-LM. Transformer . CTC . fbank . : ( | ) | .. (transcriptions). 02 04. [ id words ] = log ( | ) 1 log | .. Kaldi fbank . 01 80 03 tokens . FBank 3 pitch Beam Search .. Transformer CTC . RNN . argmax = Y log ( | ) + 1 log | + log .. 1.. 0o8o export/data/data_record/wav 62 350~400 . 350 400 ID S005.

2 2 . a) transcript b) wav wav . train: 53 dev: 6 test: 3 .. 2.. spk2utt utt2spk text . aishell recipe data_prep .. 3.. espnet 80 FBank 3 pitch 83 . Kaldi compute-cmvn-stats CMVN .. 4.. espnet text .. <unk> 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15.. 5.. espnet text . json input output . input shape output .. +CTC . espnet Attention .. Transformer . RNN-T . +Attention CTC . Attention . espnet . # network architecture # encoder related etype: vggblstm # encoder architecture type elayers: 3. eunits: 1024. CTC eprojs: 1024. subsample: "1_2_2_1_1" # skip every n frame from input to nth layers # decoder related dlayers: 2. dunits: 1024. espnet . # network architecture # encoder related | 1 , 2 , , , 1 , 2 , , 1.

3 Etype: vggblstm # encoder architecture type elayers: 3. eunits: 1024. eprojs: 1024 Softmax subsample: "1_2_2_1_1" # skip every n frame from input to nth layers . # decoder related dlayers: 2. dunits: 1024 Decoder Attention .. 1 1. # attention related . atype: location Attention adim: 1024. aconv-chans: 10.. 1 1 .. aconv-filts: 100 .. Encoder # hybrid CTC/attention mtlalpha: 1 . espnet . # network architecture # encoder related elayers: 12. eunits: 2048. # decoder related dlayers: 6. dunits: 2048. Transformer # attention related adim: 256. aheads: 4.. # hybrid CTC/attention mtlalpha: Attention Is All you Need . espnet . , | 1 , 2 , , , 1 , 2 , , 1. # network architecture ## encoder related etype: blstmp Softmax elayers: 4.

4 Eunits: 320. eprojs: 320 , . subsample: "1_2_2_1_1". RNN-T dropout-rate: Joint Network ## decoder related dtype: lstm . dlayers: 1. dec-embed-dim: 300 Prediction Encoder dunits: 300 Network ## joint network related joint-dim: 300.. 1 . espnet . # network architecture , | 1 , 2 , , , 1 , 2 , , 1. ## encoder related etype: transformer transformer-input-layer: vgg2l Softmax elayers: 8. eunits: 320. dropout-rate: , . ## decoder related dtype: transformer/lstm Joint Network Transformer-T dlayers: 2. dec-embed-dim: 300 . dunits: 300. dropout-rate-decoder: Prediction Encoder ## attention related Network adim: 320. aheads: 4 . ## joint network related 1 . joint-dim: 300.. 1. espnet . a) Transformer # network architecture # encoder related elayers: 4.

5 Eunits: 1024. # decoder related dlayers: 4. dunits: 1024. # attention related adim: 128. aheads: 4. # hybrid CTC/attention mtlalpha: . b) # rnnlm related layer: 2. unit: 650. opt: sgd # or adam batchsize: 32 # batch size in LM training epoch: 20 # if the data size is large, we can reduce this patience: 3. maxlen: 100.. c) batchsize: 0. beam-size: 20. penalty: maxlenratio: minlenratio: ctc-weight: lm-weight: . 2.. RNNLM epoch .. 3.. shell espnet . 4 . a) . b) . c) . d) .. espnet CPU . GPU decode api=v2 .. a) b) c) .. espnet .. 1. Transformer+CTC > Transformer >= Attention+CTC > Attention >= (Transformer-T) > RNN-T. > CTC. 2. Attention+CTC < Attention < Transformer+CTC < Transformer (Transformer-T) <= RNN-T. < CTC.

6 espnet . Transformer/CTC Attention/CTC Kaldi dev test .. (dev/test) .. espnet .