2022 · Chips
FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness
FlashAttention reorganizes attention computation to reduce costly reads and writes between GPU memory levels.
Paper & contextTri Dao
Reducing memory traffic and computation in sequence models.
Dao studies the relationship between learning algorithms and the hardware that runs them. FlashAttention and Mamba anchor this selection, connecting memory-efficient attention with sequence models designed to use computation and memory differently.
Selected work
10 papers
2023 · Architecture
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Mamba investigates selective state-space models as an alternative approach to sequence modeling with efficient scaling.
Paper & context2023 · Chips
FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning
FlashAttention-2 further improves GPU attention efficiency through work partitioning and parallelism.
Paper & context2026 · Efficient AI
DiG-bench: Discovery in Games
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2026 · Efficient AI
Dion3: Full-Stack Orthogonal Updates
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2026 · Efficient AI
Thought-Level Beam Search for Reasoning
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2026 · Efficient AI
Addressing the Orchestration Gap in Generalist Robots via Physical Agency
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2026 · Efficient AI
SonicSampler: Unified Tile-Aware Kernels for LLM Sampling and Speculative Verification
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2026 · Efficient AI
CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & context2026 · Efficient AI
Search Your Block Floating Point Scales!
Selected research in efficient ai. Read the full paper, including the methods, experiments, and reported results.
Paper & contextAn independent editorial profile. Inclusion does not imply Council membership or endorsement. Research is collaborative; coauthorship does not imply sole credit.
Selection & sources