← All Programmes

Emergent Abilities in Large Language Models

Machine Learning

Investigating capabilities that appear suddenly at specific model scales, including multi-step reasoning, instruction following, and few-shot learning. Questions whether these are genuine phase transitions or measurement artifacts.

Core Beliefs

  • Certain capabilities only manifest above critical model scales
  • Sharp performance transitions may indicate qualitative changes in learned representations
  • Metric choice significantly affects whether abilities appear emergent or continuous

Methods

  • Scaling law analysis across model sizes
  • Benchmark performance tracking at different scales
  • Probing classifiers for intermediate representations
64Papers
15Contributors
521Subscribers
Jan 2023Founded

Evolution Timeline

4 EVENTS
Paper Added

Paper added: Emergent Abilities Mirage

Are Emergent Abilities of Large Language Models a Mirage?

Rylan Schaeffer, Brando Miranda, Sanmi Koyejo (2023)

We show that emergent abilities may be artifacts of evaluation metrics. Continuous metrics reveal smooth improvements rather than sudden phase transitions.

Curated by @scaling_skeptic
Controversy

Controversy: Does scaling continue to work?

Debate over whether recent models show diminishing returns from scale or if we need new architectures

Scaling Laws and the Limits of Transformer Language Models

Jason Wei, Yi Tay, Quoc Le (2024)

We examine whether scaling laws continue to hold for modern large language models. Our analysis reveals potential saturation effects in certain capabilities while others continue to improve predictably with scale.

Curated by @deepmind_research
Paper Added

Paper added: Chinchilla Scaling Laws

Training Compute-Optimal Large Language Models

Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch (2022)

We investigate optimal model sizes and training dataset sizes. Current models are significantly undertrained, suggesting scaling should balance model size and training tokens.

Curated by @deepmind_research
Founded

Programme founded

Programme created to investigate limits and breakdown of neural scaling laws

Emergent Abilities of Large Language Models

Jason Wei, Yi Tay, Rishi Bommasani (2022)

We document dozens of examples where large language models display abilities not present in smaller models. These emergent abilities appear unpredictably as model scale increases, raising fundamental questions about scaling.

Curated by @scaling_skeptic