Better Language Models and Their Implications (GPT-2)

OpenAI published Better Language Models and Their Implications on February 14, 2019, introducing GPT-2 as a large transformer-based language model with 1.5 billion parameters and describing an initial staged-release approach in which a much smaller model and technical paper were released while the full trained model was withheld.