End-to-End Object Detection with Transformers arXiv:2005 ...

Transformers were rst used in auto-regressive models, following early sequence-to-sequence models [44], generating output tokens one by one. However, the pro-hibitive inference cost (proportional to output length, and hard to batch) lead to the development of parallel sequence generation, in the domains of audio [29],

  Sequence, Length