Improving language understanding with unsupervised learning

OpenAI published Improving language understanding with unsupervised learning on June 11, 2018, describing and releasing a scalable, task-agnostic language system that combined Transformer models with unsupervised pre-training followed by supervised fine-tuning across diverse language tasks.