Google introduces PaLM, a 540-billion-parameter model

Google Research introduced the Pathways Language Model on 4 April 2022, a 540-billion-parameter dense transformer trained across 6,144 TPU v4 chips using Google’s Pathways system. Google reported the best few-shot scores it had published on most of the benchmarks tested.

Why it mattered The results tied scale to Google’s Pathways training system, and fed the chain-of-thought prompting work and the PaLM 2 and Bard systems that followed it.