Transcription of Involution: Inverting the Inherence of Convolution for ...
{{id}} {{{paragraph}}}
Example: biology
Involution: Inverting the Inherence of Convolution for ...
Involution: Inverting the Inherence of Convolution for visual RecognitionDuo Li1 Jie Hu2 Changhu Wang2 Xiangtai Li3Qi She2 Lei Zhu3 Tong Zhang1 Qifeng Chen1 The Hong Kong University of Science and Technology1 ByteDance AI Lab2 Peking has been the core ingredient of modern neu-ral networks, triggering the surge of deep learning in vi-sion. In this work, we rethink the inherent principles ofstandard Convolution for vision tasks, specifically spatial-agnostic and channel-specific. Instead, we present a novelatomic operation for deep neural networks by invertingthe aforementioned design principles of Convolution , coinedas involution. We additionally demystify the recent pop-ular self-attention operator and subsume it into our invo-lution family as an over-complicated instantiation.
ness for visual recognition as an alternative, breaking through existing inductive biases of convolution. 2.We bridge the emerging philosophy of incorporating self-attention into the learning procedure of visual rep-resentation. In this context, the desiderata of com-posing pixel pairs for relation modeling is challenged.
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: