Improving language understanding with unsupervised learning
OpenAI published Improving language understanding with unsupervised learning on June 11, 2018, describing and releasing a scalable, task-agnostic language system that combined Transformer models with unsupervised pre-training followed by supervised fine-tuning across diverse language tasks.
Historical classification
This verified record belongs to LINEAiGE's artificial-intelligence history layer beginning in 1956.
Why it matters
A documented OpenAI language-model research and release milestone for generative pre-training with a Transformer-based model before the later GPT-2 publication.
People, organizations & technologies
PEOPLE
Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever
ORGANIZATIONS
OpenAI
TECHNOLOGIES
Transformer, Unsupervised pre-training, Generative pre-training
Evidence & verification
STATE
verified
CONFIDENCE
high
LAST VERIFIED
2026-09-11T13:02:10+08:00
OpenAI's first-party June 11, 2018 publication establishes the dated public research/release milestone and explicitly describes a scalable task-agnostic system combining Transformers and unsupervised pre-training, with model release. The linked OpenAI technical report identifies Radford, Narasimhan, Salimans, and Sutskever as authors and documents the generative pre-training/fine-tuning method. This record does not retroactively assert that OpenAI used the product/model name GPT-1 on that date, and it does not infer geography, ancestry, influence, causality, or relationships.
Sources
Improving language understanding with unsupervised learning
OpenAI
PUBLISHED
2018-06-11
Improving Language Understanding by Generative Pre-Training
OpenAI
Trace the chronology
Record identity
LINEAiGE IDopenai-generative-pretraining-2018