2020 · Scaling laws
Scaling Laws for Neural Language Models
How model size, training data and compute relate to language-model performance. A foundation for understanding scaling as an empirical relationship, with limits.
Paper & contextSam McCandlish
Scaling laws
Studying predictable relationships between resources and learning.
McCandlish studies quantitative patterns in neural-network training and model behavior. This selection starts with scaling laws and large language models, then follows work on alignment, evaluation, and the predictability of learning systems.
Selected work
10 papers
2020 · LLMs
Language Models are Few-Shot Learners
GPT-3 investigates how a large language model can perform tasks from instructions and examples in its prompt, without task-specific weight updates.
Paper & context2022 · Alignment
Constitutional AI: Harmlessness from AI Feedback
Constitutional AI explores training a more helpful and harmless assistant with written principles and AI-generated feedback.
Paper & context2021 · AI safety
A General Language Assistant as a Laboratory for Alignment
Selected research in ai safety, scaling laws. Read the full paper, including the methods, experiments, and reported results.
Paper & context2023 · Scaling laws
Specific versus General Principles for Constitutional AI
Selected research in scaling laws, ai safety. Read the full paper, including the methods, experiments, and reported results.
Paper & context2023 · Scaling laws
Towards Understanding Sycophancy in Language Models
Selected research in scaling laws, ai safety. Read the full paper, including the methods, experiments, and reported results.
Paper & context2023 · Scaling laws
Studying Large Language Model Generalization with Influence Functions
Selected research in scaling laws. Read the full paper, including the methods, experiments, and reported results.
Paper & context2023 · Scaling laws
Measuring Faithfulness in Chain-of-Thought Reasoning
Selected research in scaling laws. Read the full paper, including the methods, experiments, and reported results.
Paper & context2023 · Scaling laws
Question Decomposition Improves the Faithfulness of Model-Generated Reasoning
Selected research in scaling laws. Read the full paper, including the methods, experiments, and reported results.
Paper & context2023 · Scaling laws
Towards Measuring the Representation of Subjective Global Opinions in Language Models
Selected research in scaling laws, ai safety. Read the full paper, including the methods, experiments, and reported results.
Paper & contextAn independent editorial profile. Inclusion does not imply Council membership or endorsement. Research is collaborative; coauthorship does not imply sole credit.
Selection & sources