Transcription of A Gift From Knowledge Distillation: Fast Optimization ...
{{id}} {{{paragraph}}}
A Gift from Knowledge Distillation: Fast Optimization , Network Minimization and transfer LearningJunho Yim1 Donggyu Joo1 Jihoon Bae2 Junmo Kim11 School of Electrical Engineering, KAIST, South Korea2 Electronics and Telecommunications Research Institute{ , jdg105, introduce a novel technique for Knowledge transfer ,where Knowledge from a pretrained deep neural network(DNN) is distilled and transferred to another DNN. As theDNN maps from the input space to the output space throughmany layers sequentially, we define the distilled knowledgeto be transferred in terms of flow between layers, which iscalculated by computing the inner product between featuresfrom two layers.}
work and a network using various knowledge transfer tech-niques. The third task was transfer learning. Although a new task may provide only a small dataset, transfer learning can take advantage of a deep and heavy DNN pretrained with a huge dataset [2]. Because our proposed method has the advantage of being able to transfer the distilled knowledge
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document:
{{id}} {{{paragraph}}}