Transcription of Transformer Interpretability Beyond Attention Visualization
{{id}} {{{paragraph}}}
Transformer Interpretability Beyond Attention VisualizationHila Chefer1 Shir Gur1 Lior Wolf1,21 The School of Computer Science, Tel Aviv University2 Facebook AI Research (FAIR)AbstractSelf- Attention techniques, and specifically transformers ,are dominating the field of text processing and are becom-ing increasingly popular in computer vision classificationtasks. In order to visualize the parts of the image thatled to a certain classification, existing methods either relyon the obtained Attention maps or employ heuristic prop-agation along the Attention graph.
Transformer networks necessitates tools for the visualiza-tion of their decision process. Such a visualization can aid indebuggingthemodels,helpverifythatthemodelsarefair and unbiased, and enable downstream tasks. The main building block of Transformer networks are self-attention layers [29, 7], which assign a pairwise atten-
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document:
{{id}} {{{paragraph}}}