OpenAI unveils CLIP, a zero-shot image classifier

OpenAI released CLIP on 5 January 2021, a model trained on 400 million image and text pairs to place both in a shared representation. It could sort images into categories it had never been trained on by matching them against written descriptions of the candidate labels.

Why it mattered CLIP’s shared image-text representations were later used to condition image generation in DALL-E 2, linking its classification work to a text-to-image system.