2014 · Optimization
Adam: A Method for Stochastic Optimization
Adam combines adaptive learning rates with moving averages of gradients, becoming a widely used optimizer for neural networks.
Paper & contextJimmy Ba
Making optimization and attention more efficient.
Ba studies optimization, representation learning, and efficient neural networks. Adam and layer normalization are central starting points in this selection, followed by work on training, evaluating, and adapting language models.
Selected work
10 papers
2016 · Representation learning
Layer Normalization
Layer normalization normalizes a network's activations within an individual training example rather than across a batch.
Paper & context2026 · Efficient AI
EMA Policy Gradient: Taming Reinforcement Learning for LLMs with EMA Anchor and Top-k KL
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2024 · Efficient AI
Report Cards: Qualitative Evaluation of Language Models Using Natural Language Summaries
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2024 · Efficient AI
Decomposed Prompting to Answer Questions on a Course Discussion Board
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2024 · Efficient AI
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2023 · Efficient AI
Using Large Language Models for Hyperparameter Optimization
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2023 · Efficient AI
OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2023 · Efficient AI
Identifying the Risks of LM Agents with an LM-Emulated Sandbox
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2023 · Efficient AI
STEVE-1: A Generative Model for Text-to-Behavior in Minecraft
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & contextAn independent editorial profile. Inclusion does not imply Council membership or endorsement. Research is collaborative; coauthorship does not imply sole credit.
Selection & sources