Kushal Tirumala

23 papers A* 6Journal 16Unranked 1
YearRankTypeTitle / Venue / Authors
2025 J jnl
CAT: Content-Adaptive Image Tokenization.
CoRR
Junhong Shen, Kushal Tirumala, Michihiro Yasunaga, Ishan Misra, Luke Zettlemoyer, Lili Yu, Chunting Zhou
2025 J jnl
Text Quality-Based Pruning for Efficient Training of Language Models.
J. Data-centric Mach. Learn. Res.
Vasu Sharma, Karthik Padthe, Newsha Ardalani, Kushal Tirumala, Russell Howes, Hu Xu, Po-Yao Huang, Daniel Li Chen, Armen Aghajanyan, Gargi Ghosh, Luke Zettlemoyer
2025 A* conf
The Unreasonable Ineffectiveness of the Deeper Layers.
ICLR
Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, Daniel A. Roberts
2025 A* conf
Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model.
ICLR
Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, Omer Levy
2024 J jnl
An Introduction to Vision-Language Modeling.
CoRR
Florian Bordes, Richard Yuanzhe Pang, Anurag Ajay, Alexander C. Li, Adrien Bardes, Suzanne Petryk, Oscar Mañas, Zhiqiu Lin, Anas Mahmoud, Bargav Jayaraman, Mark Ibrahim, Melissa Hall, Yunyang Xiong, Jonathan Lebensold, Candace Ross, Srihari Jayakumar, Chuan Guo, Diane Bouchacourt, Haider Al-Tahan, Karthik Padthe, Vasu Sharma, Hu Xu, Xiaoqing Ellen Tan, Megan Richards, Samuel Lavoie, Pietro Astolfi, Reyhane Askari Hemmat, Jun Chen, Kushal Tirumala, Rim Assouel, Mazda Moayeri, Arjang Talattof, Kamalika Chaudhuri, Zechun Liu, Xilun Chen, Quentin Garrido, Karen Ullrich, Aishwarya Agrawal, Kate Saenko, Asli Celikyilmaz, Vikas Chandra
2024 J jnl
Brevity is the soul of wit: Pruning long files for code generation.
CoRR
Aaditya K. Singh, Yu Yang, Kushal Tirumala, Mostafa Elhoushi, Ari S. Morcos
2024 A* conf
Effective pruning of web-scale datasets based on complexity of concept clusters.
ICLR
Amro Abbas, Evgenia Rusak, Kushal Tirumala, Wieland Brendel, Kamalika Chaudhuri, Ari S. Morcos
2024 J jnl
Effective pruning of web-scale datasets based on complexity of concept clusters.
CoRR
Amro Abbas, Evgenia Rusak, Kushal Tirumala, Wieland Brendel, Kamalika Chaudhuri, Ari S. Morcos
2024 J jnl
Text Quality-Based Pruning for Efficient Training of Language Models.
CoRR
Vasu Sharma, Karthik Padthe, Newsha Ardalani, Kushal Tirumala, Russell Howes, Hu Xu, Po-Yao Huang, Shang-wen Li, Armen Aghajanyan, Gargi Ghosh, Luke Zettlemoyer
2024 J jnl
The Unreasonable Ineffectiveness of the Deeper Layers.
CoRR
Andrey Gromov, Kushal Tirumala, Hassan Shapourian, Paolo Glorioso, Daniel A. Roberts
2024 J jnl
Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model.
CoRR
Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, Omer Levy
2024 J jnl
When Worse is Better: Navigating the compression-generation tradeoff in visual tokenization.
CoRR
Vivek Ramanujan, Kushal Tirumala, Armen Aghajanyan, Luke Zettlemoyer, Ali Farhadi
2023 A* conf
D4: Improving LLM Pretraining via Document De-Duplication and Diversification.
NeurIPS
Kushal Tirumala, Daniel Simig, Armen Aghajanyan, Ari Morcos
2023 J jnl
D4: Improving LLM Pretraining via Document De-Duplication and Diversification.
CoRR
Kushal Tirumala, Daniel Simig, Armen Aghajanyan, Ari S. Morcos
2023 J jnl
Decoding Data Quality via Synthetic Corruptions: Embedding-guided Pruning of Code Data.
CoRR
Yu Yang, Aaditya K. Singh, Mostafa Elhoushi, Anas Mahmoud, Kushal Tirumala, Fabian Gloeckle, Baptiste Rozière, Carole-Jean Wu, Ari S. Morcos, Newsha Ardalani
2023 J jnl
SemDeDup: Data-efficient learning at web-scale through semantic deduplication.
CoRR
Amro Abbas, Kushal Tirumala, Daniel Simig, Surya Ganguli, Ari S. Morcos
2022 — conf
Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks.
ACL (demo)
Tristan Thrush, Kushal Tirumala, Anmol Gupta, Max Bartolo, Pedro Rodriguez, Tariq Kane, William Gaviria Rojas, Peter Mattson, Adina Williams, Douwe Kiela
2022 J jnl
Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks.
CoRR
Tristan Thrush, Kushal Tirumala, Anmol Gupta, Max Bartolo, Pedro Rodriguez, Tariq Kane, William Gaviria Rojas, Peter Mattson, Adina Williams, Douwe Kiela
2022 A* conf
Investigating Generalization by Controlling Normalized Margin.
ICML
Alexander R. Farhang, Jeremy D. Bernstein, Kushal Tirumala, Yang Liu, Yisong Yue
2022 J jnl
Investigating Generalization by Controlling Normalized Margin.
CoRR
Alexander R. Farhang, Jeremy Bernstein, Kushal Tirumala, Yang Liu, Yisong Yue
2022 A* conf
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models.
NeurIPS
Kushal Tirumala, Aram H. Markosyan, Luke Zettlemoyer, Armen Aghajanyan
2022 J jnl
Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models.
CoRR
Kushal Tirumala, Aram H. Markosyan, Luke Zettlemoyer, Armen Aghajanyan
2020 J jnl
Ensemble Machine Learning Methods for Modeling COVID19 Deaths.
CoRR
Rahil Bathwal, Pavan Chitta, Kushal Tirumala, Vignesh Varadarajan