Transformer Interpretability Beyond Attention Visualization
Transformer Interpretability Beyond Attention VisualizationHila Chefer1Shir Gur1Lior Wolf1,21The School of Computer Science, Tel Aviv University2Facebook AI Research (FAIR)AbstractSelf- Attention techniques, and specifically transformers ,are dominating the field of text processing and are becom-ing increasingly popular in computer vision classificationtasks. In order to visualize the parts of the image thatled to a certain classification, existing methods either relyon the obtained Attention maps or employ heuristic prop-agation along the Attention graph.
Transformer networks necessitates tools for the visualiza-tion of their decision process. Such a visualization can aid indebuggingthemodels,helpverifythatthemodelsarefair and unbiased, and enable downstream tasks. The main building block of Transformer networks are self-attention layers [29, 7], which assign a pairwise atten-
Download Transformer Interpretability Beyond Attention Visualization
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: