Transformer-XL: Attentive Language Models beyond a Fixed-Length Context
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, and Ruslan Salakhutdinov submitted 'Transformer-XL: Attentive Language Models beyond a Fixed-Length Context' on January 9, 2019. The paper proposes Transformer-XL, a neural architecture using a segment-level recurrence mechanism and a positional encoding scheme to model dependency beyond a fixed-length context.
Historical classification
This verified record belongs to LINEAiGE's artificial-intelligence history layer beginning in 1956.
Why it matters
A documented language-model architecture milestone introducing Transformer-XL for learning dependencies beyond a fixed-length context.
People, organizations & technologies
PEOPLE
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, Ruslan Salakhutdinov
TECHNOLOGIES
Transformer-XL, Transformers, Language modeling, Segment-level recurrence
Evidence & verification
STATE
verified
CONFIDENCE
high
LAST VERIFIED
2026-09-12T04:05:45+08:00
The original arXiv record establishes the authors, January 9 2019 submission date, and the paper's description of Transformer-XL, including segment-level recurrence and a positional encoding scheme. The ACL Anthology independently corroborates the paper, authors, technical description, July 2019 conference publication, and pages 2978-2988. The day-precision date is attached only to the arXiv submission milestone and is not asserted as a universal architecture-invention, release, or system-birth date. No geography, institutional affiliation, ancestry, influence, causality, or relationships are inferred.
Sources
Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
arXiv
PUBLISHED
2019-01-09
Transformer-XL: Attentive Language Models beyond a Fixed-Length Context
Association for Computational Linguistics
Trace the chronology
Record identity
LINEAiGE IDtransformer-xl-2019