Rohan Anil

45 papers A* 9A 1Journal 31Unranked 4
YearRankTypeTitle / Venue / Authors
2025 J jnl
Fast and Simplex: 2-Simplicial Attention in Triton.
CoRR
Aurko Roy, Timothy Chou, Sai Surya Duvvuri, Sijia Chen, Jiecao Yu, Xiaodong Wang, Manzil Zaheer, Rohan Anil
2024 J jnl
A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs.
CoRR
Ankit Singh Rawat, Veeranjaneyulu Sadhanala, Afshin Rostamizadeh, Ayan Chakrabarti, Wittawat Jitkrittum, Vladimir Feinberg, Seungyeon Kim, Hrayr Harutyunyan, Nikunj Saunshi, Zachary Nado, Rakesh Shivanna, Sashank J. Reddi, Aditya Krishna Menon, Rohan Anil, Sanjiv Kumar
2024 A* conf
Combining Axes Preconditioners through Kronecker Approximation for Deep Learning.
ICLR
Sai Surya Duvvuri, Devvrit, Rohan Anil, Cho-Jui Hsieh, Inderjit S. Dhillon
2024 J jnl
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.
CoRR
Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy P. Lillicrap, Jean-Baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, Ioannis Antonoglou, Rohan Anil, Sebastian Borgeaud, Andrew M. Dai, Katie Millican, Ethan Dyer, Mia Glaese, Thibault Sottiaux, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, James Molloy, Jilin Chen, Michael Isard, Paul Barham, Tom Hennigan, Ross McIlroy, Melvin Johnson, Johan Schalkwyk, Eli Collins, Eliza Rutherford, Erica Moreira, Kareem Ayoub, Megha Goel, Clemens Meyer, Gregory Thornton, Zhen Yang, Henryk Michalewski, Zaheer Abbas, Nathan Schucher, Ankesh Anand, Richard Ives, James Keeling, Karel Lenc, Salem Haykal, Siamak Shakeri, Pranav Shyam, Aakanksha Chowdhery, Roman Ring, Stephen Spencer, Eren Sezener, et al.
2024 J jnl
Learning from straggler clients in federated learning.
CoRR
Andrew Hard, Antonious M. Girgis, Ehsan Amid, Sean Augenstein, Lara McConnaughey, Rajiv Mathews, Rohan Anil
2024 J jnl
Michelangelo: Long Context Evaluations Beyond Haystacks via Latent Structure Queries.
CoRR
Kiran Vodrahalli, Santiago Ontanon, Nilesh Tripuraneni, Kelvin Xu, Sanil Jain, Rakesh Shivanna, Jeffrey Hui, Nishanth Dikkala, Mehran Kazemi, Bahare Fatemi, Rohan Anil, Ethan Dyer, Siamak Shakeri, Roopali Vij, Harsh Mehta, Vinay V. Ramasesh, Quoc Le, Ed H. Chi, Yifeng Lu, Orhan Firat, Angeliki Lazaridou, Jean-Baptiste Lespiau, Nithya Attaluri, Kate Olszewska
2023 A* conf
A Computationally Efficient Sparsified Online Newton Method.
NeurIPS
Devvrit, Sai Surya Duvvuri, Rohan Anil, Vineet Gupta, Cho-Jui Hsieh, Inderjit S. Dhillon
2023 J jnl
A Computationally Efficient Sparsified Online Newton Method.
CoRR
Devvrit, Sai Surya Duvvuri, Rohan Anil, Vineet Gupta, Cho-Jui Hsieh, Inderjit S. Dhillon
2023 J jnl
Benchmarking Neural Network Training Algorithms.
CoRR
George E. Dahl, Frank Schneider, Zachary Nado, Naman Agarwal, Chandramouli Shama Sastry, Philipp Hennig, Sourabh Medapati, Runa Eschenhagen, Priya Kasimbeg, Daniel Suo, Juhan Bae, Justin Gilmer, Abel L. Peirson, Bilal Khan, Rohan Anil, Mike Rabbat, Shankar Krishnan, Daniel Snider, Ehsan Amid, Kongtao Chen, Chris J. Maddison, Rakshith Vasudev, Michal Badura, Ankush Garg, Peter Mattson
2023 J jnl
Heterogeneous Federated Learning Using Knowledge Codistillation.
CoRR
Jared Lichtarge, Ehsan Amid, Shankar Kumar, Tien-Ju Yang, Rohan Anil, Rajiv Mathews
2023 J jnl
Layerwise Bregman Representation Learning of Neural Networks with Applications to Knowledge Distillation.
Trans. Mach. Learn. Res.
Ehsan Amid, Rohan Anil, Christopher Fifty, Manfred K. Warmuth
2023 J jnl
PaLM 2 Technical Report.
CoRR
Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernández Ábrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan A. Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christopher A. Choquette-Choo, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du, Ethan Dyer, Vladimir Feinberg, Fangxiaoyu Feng, Vlad Fienber, Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, et al.
2023 A* conf
Sketchy: Memory-efficient Adaptive Regularization with Frequent Directions.
NeurIPS
Vladimir Feinberg, Xinyi Chen, Y. Jennifer Sun, Rohan Anil, Elad Hazan
2023 J jnl
Sketchy: Memory-efficient Adaptive Regularization with Frequent Directions.
CoRR
Vladimir Feinberg, Xinyi Chen, Y. Jennifer Sun, Rohan Anil, Elad Hazan
2022 A* conf
Knowledge distillation: A good teacher is patient and consistent.
CVPR
Lucas Beyer, Xiaohua Zhai, Amélie Royer, Larisa Markeeva, Rohan Anil, Alexander Kolesnikov
2022 — conf
Large-Scale Differentially Private BERT.
EMNLP (Findings)
Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, Pasin Manurangsi
2022 J jnl
Layerwise Bregman Representation Learning with Applications to Knowledge Distillation.
CoRR
Ehsan Amid, Rohan Anil, Christopher Fifty, Manfred K. Warmuth
2022 J jnl
Learning from Randomly Initialized Neural Network Features.
CoRR
Ehsan Amid, Rohan Anil, Wojciech Kotlowski, Manfred K. Warmuth
2022 A conf
LocoProp: Enhancing BackProp via Local Loss Optimization.
AISTATS
Ehsan Amid, Rohan Anil, Manfred K. Warmuth
2022 J jnl
N-Grammer: Augmenting Transformers with latent n-grams.
CoRR
Aurko Roy, Rohan Anil, Guangda Lai, Benjamin Lee, Jeffrey Zhao, Shuyuan Zhang, Shibo Wang, Ye Zhang, Shen Wu, Rigel Swavely, Tao Yu, Phuong Dao, Christopher Fifty, Zhifeng Chen, Yonghui Wu
2022 — conf
On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Models.
ORSUM@RecSys
Rohan Anil, Sandra Gadanho, Da Huang, Nijith Jacob, Zhuoshu Li, Dong Lin, Todd Phillips, Cristina Pop, Kevin Regan, Gil I. Shamir, Rakesh Shivanna, Qiqi Yan
2022 J jnl
On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Models.
CoRR
Rohan Anil, Sandra Gadanho, Da Huang, Nijith Jacob, Zhuoshu Li, Dong Lin, Todd Phillips, Cristina Pop, Kevin Regan, Gil I. Shamir, Rakesh Shivanna, Qiqi Yan
2022 J jnl
Step-size Adaptation Using Exponentiated Gradient Updates.
CoRR
Ehsan Amid, Rohan Anil, Christopher Fifty, Manfred K. Warmuth
2021 J jnl
A Large Batch Optimizer Reality Check: Traditional, Generic Optimizers Suffice Across Batch Sizes.
CoRR
Zachary Nado, Justin Gilmer, Christopher J. Shallue, Rohan Anil, George E. Dahl
2021 A* conf
Efficiently Identifying Task Groupings for Multi-Task Learning.
NeurIPS
Chris Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, Chelsea Finn
2021 J jnl
Efficiently Identifying Task Groupings for Multi-Task Learning.
CoRR
Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, Chelsea Finn
2021 J jnl
Knowledge distillation: A good teacher is patient and consistent.
CoRR
Lucas Beyer, Xiaohua Zhai, Amélie Royer, Larisa Markeeva, Rohan Anil, Alexander Kolesnikov
2021 J jnl
Large-Scale Differentially Private BERT.
CoRR
Rohan Anil, Badih Ghazi, Vineet Gupta, Ravi Kumar, Pasin Manurangsi
2021 J jnl
LocoProp: Enhancing BackProp via Local Loss Optimization.
CoRR
Ehsan Amid, Rohan Anil, Manfred K. Warmuth
2020 J jnl
Disentangling Adaptive Gradient Methods from Learning Rates.
CoRR
Naman Agarwal, Rohan Anil, Elad Hazan, Tomer Koren, Cyril Zhang
2020 J jnl
Measuring and Harnessing Transference in Multi-Task Learning.
CoRR
Christopher Fifty, Ehsan Amid, Zhe Zhao, Tianhe Yu, Rohan Anil, Chelsea Finn
2020 J jnl
Second Order Optimization Made Practical.
CoRR
Rohan Anil, Vineet Gupta, Tomer Koren, Kevin Regan, Yoram Singer
2020 A* conf
Stochastic Optimization with Laggard Data Pipelines.
NeurIPS
Naman Agarwal, Rohan Anil, Tomer Koren, Kunal Talwar, Cyril Zhang
2020 J jnl
Stochastic Optimization with Laggard Data Pipelines.
CoRR
Naman Agarwal, Rohan Anil, Tomer Koren, Kunal Talwar, Cyril Zhang
2019 J jnl
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling.
CoRR
Jonathan Shen, Patrick Nguyen, Yonghui Wu, Zhifeng Chen, Mia Xu Chen, Ye Jia, Anjuli Kannan, Tara N. Sainath, Yuan Cao, Chung-Cheng Chiu, Yanzhang He, Jan Chorowski, Smit Hinsu, Stella Laurenzo, James Qin, Orhan Firat, Wolfgang Macherey, Suyog Gupta, Ankur Bapna, Shuyuan Zhang, Ruoming Pang, Ron J. Weiss, Rohit Prabhavalkar, Qiao Liang, Benoit Jacob, Bowen Liang, HyoukJoong Lee, Ciprian Chelba, Sébastien Jean, Bo Li, Melvin Johnson, Rohan Anil, Rajat Tibrewal, Xiaobing Liu, Akiko Eriguchi, Navdeep Jaitly, Naveen Ari, Colin Cherry, Parisa Haghani, Otavio Good, Youlong Cheng, Raziel Alvarez, Isaac Caswell, Wei-Ning Hsu, Zongheng Yang, Kuan-Chieh Wang, Ekaterina Gonina, Katrin Tomanek, Ben Vanik, Zelin Wu, Llion Jones, Mike Schuster, Yanping Huang, Dehao Chen, Kazuki Irie, George F. Foster, John Richardson, Klaus Macherey, Antoine Bruguier, Heiga Zen, Colin Raffel, Shankar Kumar, Kanishka Rao, David Rybach, Matthew Murray, Vijayaditya Peddinti, Maxim Krikun, Michiel Bacchiani, Thomas B. Jablin, Robert Suderman, Ian Williams, Benjamin Lee, Deepti Bhatia, Justin Carlson, Semih Yavuz, Yu Zhang, Ian McGraw, Max Galkin, Qi Ge, Golan Pundak, Chad Whipkey, Todd Wang, Uri Alon, Dmitry Lepikhin, Ye Tian, Sara Sabour, William Chan, Shubham Toshniwal, Baohua Liao, Michael Nirschl, Pat Rondon
2019 A* conf
Memory Efficient Adaptive Optimization.
NeurIPS
Rohan Anil, Vineet Gupta, Tomer Koren, Yoram Singer
2019 J jnl
Memory-Efficient Adaptive Optimization for Large-Scale Learning.
CoRR
Rohan Anil, Vineet Gupta, Tomer Koren, Yoram Singer
2019 A* conf
Robust Bi-Tempered Logistic Loss Based on Bregman Divergences.
NeurIPS
Ehsan Amid, Manfred K. Warmuth, Rohan Anil, Tomer Koren
2019 J jnl
Robust Bi-Tempered Logistic Loss Based on Bregman Divergences.
CoRR
Ehsan Amid, Manfred K. Warmuth, Rohan Anil, Tomer Koren
2019 A* conf
TF-Ranking: Scalable TensorFlow Library for Learning-to-Rank.
KDD
Rama Kumar Pasumarthi, Sebastian Bruch, Xuanhui Wang, Cheng Li, Michael Bendersky, Marc Najork, Jan Pfeifer, Nadav Golbandi, Rohan Anil, Stephan Wolf
2018 — conf
Large scale distributed neural network training through online distillation.
ICLR (Poster)
Rohan Anil, Gabriel Pereyra, Alexandre Passos, Róbert Ormándi, George E. Dahl, Geoffrey E. Hinton
2018 J jnl
Large scale distributed neural network training through online distillation.
CoRR
Rohan Anil, Gabriel Pereyra, Alexandre Passos, Róbert Ormándi, George E. Dahl, Geoffrey E. Hinton
2018 J jnl
TF-Ranking: Scalable TensorFlow Library for Learning-to-Rank.
CoRR
Rama Kumar Pasumarthi, Xuanhui Wang, Cheng Li, Sebastian Bruch, Michael Bendersky, Marc Najork, Jan Pfeifer, Nadav Golbandi, Rohan Anil, Stephan Wolf
2016 — conf
Wide & Deep Learning for Recommender Systems.
DLRS@RecSys
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, Hemal Shah
2016 J jnl
Wide & Deep Learning for Recommender Systems.
CoRR
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, Rohan Anil, Zakaria Haque, Lichan Hong, Vihan Jain, Xiaobing Liu, Hemal Shah