The year foundation models multiplied and Europe proposed binding rules

OpenAI extended GPT-3 into images with DALL-E and CLIP in January, then into code with Codex, which was already supplying GitHub’s Copilot. The Beijing Academy of Artificial Intelligence announced Wu Dao 2.0 at 1.75 trillion parameters, and Microsoft and Nvidia trained a 530-billion-parameter model. DeepMind and EMBL-EBI opened a public database of AlphaFold structures covering the human proteome. Stanford researchers named the class of system a foundation model. The European Commission proposed the AI Act, the first binding framework of its kind. Seven former OpenAI staff founded Anthropic, and Frances Haugen told the Senate that Facebook’s ranking systems caused harm the company had measured.

14 entries recorded

November

  1. Meta shuts down Facebook’s face recognition system

    Meta said on 2 November 2021 that it would shut down Facebook’s face recognition system, which suggested photo tags and told people when they appeared in others’ uploads, and delete the templates of more than a billion people who had opted in.

October

  1. Microsoft and Nvidia unveil a 530-billion-parameter model

    Microsoft and Nvidia announced Megatron-Turing NLG on 11 October 2021, a 530-billion-parameter transformer language model. It was trained with DeepSpeed and Megatron-LM on 560 Nvidia DGX A100 servers, drawing largely on the Pile and Common Crawl web data.

  2. Frances Haugen testifies to Congress on Facebook’s algorithms

    Frances Haugen, a former Facebook product manager, told a United States Senate subcommittee on 5 October 2021 that the company’s content-ranking systems were dangerous and that its own research showed as much.

August

  1. Stanford researchers coin the term foundation model

    More than 150 Stanford researchers, organized as the new Center for Research on Foundation Models, published “On the Opportunities and Risks of Foundation Models” on 16 August 2021, covering models such as BERT, GPT-3 and DALL-E and the risks they carry.

  2. OpenAI announces Codex, a code-generating version of GPT-3

    OpenAI announced Codex on 10 August 2021 and opened a private beta. The model, a version of GPT-3 fine-tuned on public source code, turned plain-English instructions into working code across several programming languages in live demonstrations.

July

  1. DeepMind and EMBL-EBI publish the AlphaFold protein database

    DeepMind and EMBL’s European Bioinformatics Institute opened the AlphaFold Protein Structure Database on 22 July 2021, with more than 350,000 predicted structures covering the roughly 20,000 proteins of the human proteome and 20 other organisms used in research.

June

  1. GitHub launches a technical preview of Copilot

    GitHub opened a technical preview of Copilot on 29 June 2021, a Visual Studio Code extension built with OpenAI that suggests whole lines and functions as a developer types. GitHub said the model was trained on billions of lines of public code.

  2. BAAI unveils Wu Dao 2.0, a 1.75-trillion-parameter model

    The Beijing Academy of Artificial Intelligence unveiled Wu Dao 2.0 on 1 June 2021, a multimodal pretrained model it said used 1.75 trillion parameters, about ten times the count in GPT-3. BAAI said it trained on 1.2 terabytes of English and Chinese text.

May

  1. Anthropic emerges with $124 million and a safety focus

    Anthropic, founded by seven former OpenAI employees including Dario Amodei, previously vice president of research, and his sister Daniela Amodei, disclosed on 28 May 2021 that it had raised $124 million.

April

  1. European Commission proposes the EU Artificial Intelligence Act

    The European Commission published its proposal for the Artificial Intelligence Act on 21 April 2021, sorting systems into risk tiers from unacceptable to minimal, setting mandatory duties for high-risk uses and transparency rules for chatbots and deepfake generators.

March

  1. Bender, Gebru and coauthors publish “Stochastic Parrots”

    Emily M. Bender, Timnit Gebru, Angelina McMillan-Major and a coauthor writing as Shmargaret Shmitchell published “On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?” at the ACM FAccT conference on 3 March 2021.

  2. National Security Commission delivers its final AI report

    The National Security Commission on Artificial Intelligence, chaired by Eric Schmidt with vice chair Robert Work, delivered a final report of more than 700 pages to Congress on 1 March 2021, arguing the United States was not prepared to compete with China on AI.

January

  1. OpenAI unveils CLIP, a zero-shot image classifier

    OpenAI released CLIP on 5 January 2021, a model trained on 400 million image and text pairs to place both in a shared representation. It could sort images into categories it had never been trained on by matching them against written descriptions of the candidate labels.

  2. OpenAI unveils DALL-E, a text-to-image generator

    OpenAI introduced DALL-E on 5 January 2021, a 12-billion-parameter version of GPT-3 trained to generate images from text captions. It combined unrelated concepts, rendered text inside images and produced anthropomorphized objects and animals from written prompts.