Example: quiz answers
Self-Attention with Relative Position Representations

Self-Attention with Relative Position Representations

Back to document page

Position encodings based on sinusoids of vary-ing frequency are added to encoder and decoder input elements prior to the first layer. In contrast to learned, absolute position representations, the authors hypothesized that sinusoidal position en-codings would help the model to generalize to se-quence lengths unseen during training by allowing

  Position

Download Self-Attention with Relative Position Representations


Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Other abuse

Advertisement

Related search queries