Parameter

One of the adjustable numbers inside a model. Modern language models have billions of them.

A parameter is a dial. During training the system turns billions of them a little at a time, keeping the settings that make its answers better and discarding the ones that make them worse. Counts are quoted as a rough measure of size: GPT-3 was described as having 175 billion parameters. Size is not the same as usefulness. A smaller model trained on better data often does the job of a larger one at a fraction of the running cost.

Where it mattered in the record

  1. Reflection AI launches Beam

    On 5 October 2026 Reflection AI introduced Beam, its 501B-parameter open-weight model with 23B active parameters and 1M tokens of effective context. Beam stays in final red-teaming; weights will follow this month.

  2. Aleph Alpha releases open-weight model Kolibri

    On 3 October 2026 Aleph Alpha released Kolibri, an English-German mixture-of-experts Transformer with 78.1B total parameters and 3.46B active, under Apache 2.0 on Hugging Face. Context reaches 1M tokens, and Merlin-Arthur withholds answers the context does not support.

  3. Alibaba announces Qwen3.8-Max, its largest model to date

    Alibaba announced Qwen3.8-Max on 3 August 2026 with 2.4 trillion parameters, about 95 billion active at a time through a sparse mixture of experts, a context window up to 1 million tokens, and native text, image and video input. Weights were scheduled to follow a week later.

  4. DeepSeek publishes its V4 model series under an MIT license

    DeepSeek published a preview of its V4 series on 26 April 2026, comprising V4-Pro at 1.6 trillion total parameters with 49 billion active and V4-Flash at 284 billion with 13 billion active, released under the MIT license with checkpoints on Hugging Face.

  5. Alibaba splits Qwen into open and proprietary tiers

    Alibaba released Qwen3.6 in April 2026, a 35-billion-parameter mixture-of-experts model with 3 billion active parameters, under the Apache 2.0 license. It kept the more capable Qwen3.6-Plus and Qwen3.5-Omni proprietary, reachable only through Qwen’s apps and Alibaba Cloud.

  6. Mistral releases Small 4 and joins Nvidia’s Nemotron Coalition

    Mistral AI released Mistral Small 4 on 16 March 2026, a 119-billion-parameter mixture-of-experts model with 6 billion active parameters, a 256,000-token context window and an Apache 2.0 license. It joined Nvidia’s Nemotron Coalition the same day.

  7. Meta releases Llama 3 under an open license

    Meta released the first Llama 3 models, at 8 billion and 70 billion parameters, on 18 April 2024 under a license permitting most commercial use. Meta said the 70B model beat comparable models, including Claude 3 Sonnet, on human evaluations across twelve use cases.

  8. Meta releases Llama 2 with weights free for commercial use

    Meta released Llama 2 on 18 July 2023 at 7, 13 and 70 billion parameters, with the weights and code free for research and for most commercial use. Microsoft was named Meta’s preferred partner, and the models were distributed through Azure, Amazon Web Services and Hugging Face.

Every term in the glossary