Transcription of Transformer Interpretability Beyond Attention Visualization
{{id}} {{{paragraph}}}
Example: marketing
Transformer Interpretability Beyond Attention Visualization
Transformer Interpretability Beyond Attention VisualizationHila Chefer1 Shir Gur1 Lior Wolf1,21 The School of Computer Science, Tel Aviv University2 Facebook AI Research (FAIR)AbstractSelf- Attention techniques, and specifically Transformers,are dominating the field of text processing and are becom-ing increasingly popular in computer vision classificationtasks. In order to visualize the parts of the image thatled to a certain classification, existing methods either relyon the obtained Attention maps or employ heuristic prop-agation along the Attention graph. In this work, we pro-pose a novel way to compute relevancy for Transformernetworks. The method assigns local relevance based onthe Deep Taylor Decomposition principle and then prop-agates these relevancy scores through the layers. Thispropagation involves Attention layers and skip connections,which challenge existing methods.
Transformer Interpretability Beyond Attention Visualization Hila Chefer1 Shir Gur1 Lior Wolf1,2 1The School of Computer Science, Tel Aviv University 2Facebook AI Research (FAIR) Abstract Self-attention techniques, and specifically Transformers, are dominating the field of text processing and are becom-
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: