LINEAiGE
AI HISTORY

Transformer-XL: Attentive Language Models beyond a Fixed-Length Context

Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, and Ruslan Salakhutdinov submitted 'Transformer-XL: Attentive Language Models beyond a Fixed-Length Context' on January 9, 2019. The paper proposes Transformer-XL, a neural architecture using a segment-level recurrence mechanism and a positional encoding scheme to model dependency beyond a fixed-length context.

January 9, 2019AI HISTORYVERIFIED

Historical classification

This verified record belongs to LINEAiGE's artificial-intelligence history layer beginning in 1956.

Why it matters

A documented language-model architecture milestone introducing Transformer-XL for learning dependencies beyond a fixed-length context.

People, organizations & technologies

PEOPLE
Zihang Dai, Zhilin Yang, Yiming Yang, Jaime Carbonell, Quoc V. Le, Ruslan Salakhutdinov

TECHNOLOGIES
Transformer-XL, Transformers, Language modeling, Segment-level recurrence

Evidence & verification

STATE
verified

CONFIDENCE
high

LAST VERIFIED
2026-09-12T04:05:45+08:00

The original arXiv record establishes the authors, January 9 2019 submission date, and the paper's description of Transformer-XL, including segment-level recurrence and a positional encoding scheme. The ACL Anthology independently corroborates the paper, authors, technical description, July 2019 conference publication, and pages 2978-2988. The day-precision date is attached only to the arXiv submission milestone and is not asserted as a universal architecture-invention, release, or system-birth date. No geography, institutional affiliation, ancestry, influence, causality, or relationships are inferred.

Sources

SOURCE 1 · PRIMARY · CLAIM-EVIDENCE

Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context

arXiv

PUBLISHED
2019-01-09

SOURCE 2 · PRIMARY · CORROBORATION

Transformer-XL: Attentive Language Models beyond a Fixed-Length Context

Association for Computational Linguistics

Trace the chronology

Record identity

LINEAiGE ID
transformer-xl-2019