2023 · Fine-tuning
QLoRA: Efficient Finetuning of Quantized LLMs
QLoRA investigates fine-tuning quantized models with low-rank adapters to reduce memory requirements.
Paper & contextTim Dettmers
Efficient AI
Reducing the memory cost of large-model training and inference.
Dettmers studies how to reduce the memory and computing requirements of large models. The selection includes 8-bit optimization, LLM.int8(), and QLoRA, alongside research on practical deployment, reproducibility, and resource costs.
Selected work
10 papers
2022 · Efficient AI
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
LLM.int8() reduces language-model memory requirements while treating unusually large feature values with higher precision.
Paper & context2021 · Efficient AI
8-bit Optimizers via Block-wise Quantization
8-bit optimizers compress the running statistics used during training, reducing memory spent on optimizer state.
Paper & context2026 · Efficient AI
ReproRepo: Scaling Reproducibility Audits with GitHub Repository Issues
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2026 · Efficient AI
AdaExplore: Failure-Driven Adaptation and Diversity-Preserving Search for Efficient Kernel Generation
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2026 · Efficient AI
SERA: Soft-Verified Efficient Repository Agents
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2025 · Efficient AI
Holistically Evaluating the Environmental Impact of Creating Language Models
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2024 · Efficient AI
OLMoE: Open Mixture-of-Experts Language Models
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2024 · Efficient AI
Scaling Retrieval-Based Language Models with a Trillion-Token Datastore
Selected research in efficient ai, language models. Read the full paper, including the methods, experiments, and reported results.
Paper & context2023 · Efficient AI
Distributed Inference and Fine-tuning of Large Language Models Over The Internet
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & contextAn independent editorial profile. Inclusion does not imply Council membership or endorsement. Research is collaborative; coauthorship does not imply sole credit.
Selection & sources