NLP

3 entries between September 2016 and October 2018.

Every entry

  1. Google publishes BERT, a bidirectional language model

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova published BERT on 11 October 2018, a transformer that pretrains representations of text by reading in both directions, then fine-tuned for particular tasks. Google released the code and weights the following month.

  2. OpenAI publishes the GPT-1 paper on pre-training

    Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever published “Improving Language Understanding by Generative Pre-Training” on 11 June 2018, describing a transformer pretrained on unlabeled text and then fine-tuned for particular tasks.

  3. Google puts neural machine translation into production

    Google switched Google Translate’s Chinese-to-English pair to GNMT, an end-to-end deep LSTM system described in a companion paper by Yonghui Wu and 30 co-authors. Google said it cut translation errors by 55 to 85 percent on several language pairs.