Foundation model
A large model trained once at great expense, then adapted by many others for many purposes.
The term was coined by Stanford researchers in 2021 to describe systems like GPT-3: trained once on broad data, then fine-tuned, prompted, or built upon for countless downstream uses. Lawmakers borrowed the concept when writing AI rules, since regulating each application separately looked impractical. The economics follow from the definition: few can afford to build one, many can afford to adapt one.
Where it mattered in the record
-
Meta releases Llama 3.1 with a 405-billion-parameter model
Meta released Llama 3.1 on 23 July 2024, including a 405-billion-parameter model it called the largest openly available foundation model to date, with a 128,000-token context window. Meta said it matched GPT-4o and Claude 3.5 Sonnet on knowledge, math, and tool-use benchmarks.
-
Stanford researchers coin the term foundation model
More than 150 Stanford researchers, organized as the new Center for Research on Foundation Models, published “On the Opportunities and Risks of Foundation Models” on 16 August 2021, covering models such as BERT, GPT-3 and DALL-E and the risks they carry.