OpenAI publishes the GPT-1 paper on pre-training

Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever published “Improving Language Understanding by Generative Pre-Training” on 11 June 2018, describing a transformer pretrained on unlabeled text and then fine-tuned for particular tasks.

Why it mattered The paper set the pretrain-then-fine-tune pattern, and the term generative pre-trained transformer, that OpenAI carried through every later GPT model.