Explore university research and the professors behind it. Original papers, connected ideas, room to learn.
Academia includes work connected to the academic researchers in this collection. Collaborations can span universities and companies; see each original manuscript for affiliations at publication.
229 papers
Direct Preference Optimization derives a preference-learning objective that avoids training a separate reward model in the studied setup.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon and colleaguesDenoising diffusion probabilistic models learn to generate data by reversing a noise-adding process. This became a foundation of modern image generation.
Jonathan Ho, Ajay Jain, Pieter AbbeelResidual networks introduce skip connections that make very deep networks easier to optimize for visual recognition.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian SunGenerative adversarial networks train a generator and a discriminator together. Their competing objectives can learn to synthesize data.
Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu and colleaguesVariational autoencoders provide a tractable approach to learning probabilistic latent representations and generating data.
Diederik P Kingma, Max WellingAdam combines adaptive learning rates with moving averages of gradients, becoming a widely used optimizer for neural networks.
Diederik P. Kingma, Jimmy BaDropout regularizes neural networks by randomly omitting units during training, reducing dependence on particular combinations of features.
Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever and colleaguesQLoRA investigates fine-tuning quantized models with low-rank adapters to reduce memory requirements.
Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke ZettlemoyerFlashAttention reorganizes attention computation to reduce costly reads and writes between GPU memory levels.
Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra and colleaguesFlashAttention-2 further improves GPU attention efficiency through work partitioning and parallelism.
Tri DaoMamba investigates selective state-space models as an alternative approach to sequence modeling with efficient scaling.
Albert Gu, Tri DaoSparsely gated mixture-of-experts layers explore conditional computation using a learned routing network.
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis and colleaguesToolformer explores teaching language models to invoke external tools with a self-supervised training approach.
Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu and colleaguesAttention Sinks studies why a few initial tokens can preserve streaming language-model performance over long sequences, enabling efficient generation with limited memory.
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han and colleaguesBIG-bench collects a broad set of language-model evaluation tasks to study capabilities and limitations beyond a single benchmark.
Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb and colleaguesRT-2 transfers knowledge from vision-language models into robotic control by representing actions in a compatible format.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar and colleaguesOpen X-Embodiment combines robotic data across embodiments to study generalization and reusable robot policies.
Embodiment Collaboration, Abby O'Neill, Abdul Rehman, Abhinav Gupta and colleaguesRT-1 uses a robotics transformer trained on diverse real-world demonstrations to map camera images and instructions to robot actions, studying generalization and control at scale.
Anthony Brohan, Noah Brown, Justice Carbajal, Yevgen Chebotar and colleaguesFourier Neural Operators learn mappings between function spaces and can approximate solutions to families of partial differential equations.
Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu and colleaguesAWQ uses activation information to choose weight quantization strategies that preserve model performance under compression.
Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang and colleaguesKnowledge distillation trains a smaller model to reproduce the informative predictions of a larger teacher or ensemble.
Geoffrey Hinton, Oriol Vinyals, Jeff DeanCapsule networks represent object properties with vectors and route information according to agreement between parts and wholes.
Sara Sabour, Nicholas Frosst, Geoffrey E HintonGLOM is a conceptual proposal for representing image-dependent part-whole hierarchies inside a network with a fixed architecture.
Geoffrey HintonSelected research in representation learning. Read the full paper, including the methods, experiments, and reported results.
Geoffrey HintonSelected research in representation learning. Read the full paper, including the methods, experiments, and reported results.
Kevin Clark, Kelvin Guu, Ming-Wei Chang, Panupong Pasupat and colleaguesSelected research in representation learning. Read the full paper, including the methods, experiments, and reported results.
Laura Culp, Sara Sabour, Geoffrey E. HintonSelected research in representation learning. Read the full paper, including the methods, experiments, and reported results.
Ting Chen, Ruixiang Zhang, Geoffrey HintonSelected research in representation learning, deep learning. Read the full paper, including the methods, experiments, and reported results.
Yoshua Bengio, Stephen Clare, Carina Prunkl, Maksym Andriushchenko and colleaguesSelected research in deep learning. Read the full paper, including the methods, experiments, and reported results.
Dzmitry Bahdanau, Kyunghyun Cho, Yoshua BengioSelected research in deep learning. Read the full paper, including the methods, experiments, and reported results.
Yoshua Bengio, Aaron Courville, Pascal VincentSelected research in deep learning. Read the full paper, including the methods, experiments, and reported results.
Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero and colleaguesSelected research in deep learning. Read the full paper, including the methods, experiments, and reported results.
Qinghua Lu, Yoshua Bengio