2022 · Scaling laws
Training Compute-Optimal Large Language Models
The Chinchilla study revisits how to divide a training budget between model size and data. It argues that many contemporary large models were undertrained.
Paper & contextArthur Mensch
Language models
Studying compute-efficient training and sparse language models.
Mensch’s coauthored research spans compute-efficient language-model training and multimodal models. Chinchilla and Mixtral are starting points for this selection, followed by work on open models, coding, speech, and reasoning.
Selected work
10 papers
2024 · Open source
Mixtral of Experts
Mixtral of Experts explores a sparse mixture-of-experts architecture, using a subset of experts for each token.
Paper & context2023 · Language models
Mistral 7B
Selected research in language models. Read the full paper, including the methods, experiments, and reported results.
Paper & context2022 · Language models
Flamingo: a Visual Language Model for Few-Shot Learning
Selected research in language models, computer vision. Read the full paper, including the methods, experiments, and reported results.
Paper & context2025 · Language models
Devstral: Fine-tuning Language Models for Coding Agent Applications
Selected research in language models. Read the full paper, including the methods, experiments, and reported results.
Paper & context2025 · Language models
Voxtral
Selected research in language models. Read the full paper, including the methods, experiments, and reported results.
Paper & context2025 · Language models
Magistral
Selected research in language models. Read the full paper, including the methods, experiments, and reported results.
Paper & context2024 · Language models
Pixtral 12B
Selected research in language models. Read the full paper, including the methods, experiments, and reported results.
Paper & context2023 · AI for science
Gemini: A Family of Highly Capable Multimodal Models
Selected research in ai for science, reinforcement learning. Read the full paper, including the methods, experiments, and reported results.
Paper & context2023 · Language models
Three ways to improve feature alignment for open vocabulary detection
Selected research in language models. Read the full paper, including the methods, experiments, and reported results.
Paper & contextAn independent editorial profile. Inclusion does not imply Council membership or endorsement. Research is collaborative; coauthorship does not imply sole credit.
Selection & sources