Sashank J. Reddi

95 papers A* 31A 7Journal 48Unranked 8

Year	Rank	Type	Title / Venue / Authors
2025	A*	`conf`	Bipartite Ranking From Multiple Labels: On Loss Versus Label Aggregation. ICML Michal Lukasik, Lin Chen, Harikrishna Narasimhan, Aditya Krishna Menon, Wittawat Jitkrittum, Felix X. Yu, Sashank J. Reddi, Gang Fu, MohammadHossein Bateni, Sanjiv Kumar
2025	J	`jnl`	Bipartite Ranking From Multiple Labels: On Loss Versus Label Aggregation. CoRR Michal Lukasik, Lin Chen, Harikrishna Narasimhan, Aditya Krishna Menon, Wittawat Jitkrittum, Felix X. Yu, Sashank J. Reddi, Gang Fu, MohammadHossein Bateni, Sanjiv Kumar
2025	A*	`conf`	Efficient stagewise pretraining via progressive subnetworks. ICLR Abhishek Panigrahi, Nikunj Saunshi, Kaifeng Lyu, Sobhan Miryoosefi, Sashank J. Reddi, Satyen Kale, Sanjiv Kumar
2025	A*	`conf`	Reasoning with Latent Thoughts: On the Power of Looped Transformers. ICLR Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, Sashank J. Reddi
2025	J	`jnl`	Reasoning with Latent Thoughts: On the Power of Looped Transformers. CoRR Nikunj Saunshi, Nishanth Dikkala, Zhiyuan Li, Sanjiv Kumar, Sashank J. Reddi
2025	A*	`conf`	Structured Preconditioners in Adaptive Optimization: A Unified Analysis. ICML Shuo Xie, Tianhao Wang, Sashank J. Reddi, Sanjiv Kumar, Zhiyuan Li
2024	J	`jnl`	A Little Help Goes a Long Way: Efficient LLM Training by Leveraging Small LMs. CoRR Ankit Singh Rawat, Veeranjaneyulu Sadhanala, Afshin Rostamizadeh, Ayan Chakrabarti, Wittawat Jitkrittum, Vladimir Feinberg, Seungyeon Kim, Hrayr Harutyunyan, Nikunj Saunshi, Zachary Nado, Rakesh Shivanna, Sashank J. Reddi, Aditya Krishna Menon, Rohan Anil, Sanjiv Kumar
2024	A*	`conf`	Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning? ICML Khashayar Gatmiry, Nikunj Saunshi, Sashank J. Reddi, Stefanie Jegelka, Sanjiv Kumar
2024	J	`jnl`	Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning? CoRR Khashayar Gatmiry, Nikunj Saunshi, Sashank J. Reddi, Stefanie Jegelka, Sanjiv Kumar
2024	J	`jnl`	Efficient Document Ranking with Learnable Late Interactions. CoRR Ziwei Ji, Himanshu Jain, Andreas Veit, Sashank J. Reddi, Sadeep Jayasumana, Ankit Singh Rawat, Aditya Krishna Menon, Felix X. Yu, Sanjiv Kumar
2024	J	`jnl`	Efficient Stagewise Pretraining via Progressive Subnetworks. CoRR Abhishek Panigrahi, Nikunj Saunshi, Kaifeng Lyu, Sobhan Miryoosefi, Sashank J. Reddi, Satyen Kale, Sanjiv Kumar
2024	J	`jnl`	Landscape-Aware Growing: The Power of a Little LAG. CoRR Stefani Karp, Nikunj Saunshi, Sobhan Miryoosefi, Sashank J. Reddi, Sanjiv Kumar
2024	J	`jnl`	On the Inductive Bias of Stacking Towards Improving Reasoning. CoRR Nikunj Saunshi, Stefani Karp, Shankar Krishnan, Sobhan Miryoosefi, Sashank J. Reddi, Sanjiv Kumar
2024	J	`jnl`	On the Role of Depth and Looping for In-Context Learning with Task Diversity. CoRR Khashayar Gatmiry, Nikunj Saunshi, Sashank J. Reddi, Stefanie Jegelka, Sanjiv Kumar
2024	A*	`conf`	Simplicity Bias via Global Convergence of Sharpness Minimization. ICML Khashayar Gatmiry, Zhiyuan Li, Sashank J. Reddi, Stefanie Jegelka
2024	J	`jnl`	Simplicity Bias via Global Convergence of Sharpness Minimization. CoRR Khashayar Gatmiry, Zhiyuan Li, Sashank J. Reddi, Stefanie Jegelka
2023	J	`jnl`	Depth Dependence of μP Learning Rates in ReLU MLPs. CoRR Samy Jelassi, Boris Hanin, Ziwei Ji, Sashank J. Reddi, Srinadh Bhojanapalli, Sanjiv Kumar
2023	A*	`conf`	Differentially Private Adaptive Optimization with Delayed Preconditioners. ICLR Tian Li, Manzil Zaheer, Ken Liu, Sashank J. Reddi, Hugh Brendan McMahan, Virginia Smith
2023	A*	`conf`	Efficient Training of Language Models using Few-Shot Learning. ICML Sashank J. Reddi, Sobhan Miryoosefi, Stefani Karp, Shankar Krishnan, Satyen Kale, Seungyeon Kim, Sanjiv Kumar
2023	J	`jnl`	The Inductive Bias of Flatness Regularization for Deep Matrix Factorization. CoRR Khashayar Gatmiry, Zhiyuan Li, Ching-Yao Chuang, Sashank J. Reddi, Tengyu Ma, Stefanie Jegelka
2023	A*	`conf`	The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformers. ICLR Zonglin Li, Chong You, Srinadh Bhojanapalli, Daliang Li, Ankit Singh Rawat, Sashank J. Reddi, Ke Ye, Felix Chern, Felix X. Yu, Ruiqi Guo, Sanjiv Kumar
2023	A*	`conf`	What is the Inductive Bias of Flatness Regularization? A Study of Deep Matrix Factorization Models. NeurIPS Khashayar Gatmiry, Zhiyuan Li, Tengyu Ma, Sashank J. Reddi, Stefanie Jegelka, Ching-Yao Chuang
2022	J	`jnl`	Differentially Private Adaptive Optimization with Delayed Preconditioners. CoRR Tian Li, Manzil Zaheer, Ken Ziyu Liu, Sashank J. Reddi, H. Brendan McMahan, Virginia Smith
2022	J	`jnl`	FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clients. CoRR Jianyu Wang, Hang Qi, Ankit Singh Rawat, Sashank J. Reddi, Sagar Waghmare, Felix X. Yu, Gauri Joshi
2022	A*	`conf`	In defense of dual-encoders for neural ranking. ICML Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Seungyeon Kim, Sashank J. Reddi, Sanjiv Kumar
2022	J	`jnl`	Large Models are Parsimonious Learners: Activation Sparsity in Trained Transformers. CoRR Zonglin Li, Chong You, Srinadh Bhojanapalli, Daliang Li, Ankit Singh Rawat, Sashank J. Reddi, Ke Ye, Felix Chern, Felix X. Yu, Ruiqi Guo, Sanjiv Kumar
2022	J	`jnl`	On the Algorithmic Stability and Generalization of Adaptive Optimization Methods. CoRR Han Nguyen, Hai Pham, Sashank J. Reddi, Barnabás Póczos
2022	J	`jnl`	Private Adaptive Optimization with Side Information. CoRR Tian Li, Manzil Zaheer, Sashank J. Reddi, Virginia Smith
2022	A*	`conf`	Private Adaptive Optimization with Side information. ICML Tian Li, Manzil Zaheer, Sashank J. Reddi, Virginia Smith
2022	A*	`conf`	Robust Training of Neural Networks Using Scale Invariant Architectures. ICML Zhiyuan Li, Srinadh Bhojanapalli, Manzil Zaheer, Sashank J. Reddi, Sanjiv Kumar
2022	J	`jnl`	Robust Training of Neural Networks using Scale Invariant Architectures. CoRR Zhiyuan Li, Srinadh Bhojanapalli, Manzil Zaheer, Sashank J. Reddi, Sanjiv Kumar
2021	J	`jnl`	A Field Guide to Federated Optimization. CoRR Jianyu Wang, Zachary Charles, Zheng Xu, Gauri Joshi, H. Brendan McMahan, Blaise Agüera y Arcas, Maruan Al-Shedivat, Galen Andrew, Salman Avestimehr, Katharine Daly, Deepesh Data, Suhas N. Diggavi, Hubert Eichner, Advait Gadhikar, Zachary Garrett, Antonious M. Girgis, Filip Hanzely, Andrew Hard, Chaoyang He, Samuel Horváth, Zhouyuan Huo, Alex Ingerman, Martin Jaggi, Tara Javidi, Peter Kairouz, Satyen Kale, Sai Praneeth Karimireddy, Jakub Konecný, Sanmi Koyejo, Tian Li, Luyang Liu, Mehryar Mohri, Hang Qi, Sashank J. Reddi, Peter Richtárik, Karan Singhal, Virginia Smith, Mahdi Soltanolkotabi, Weikang Song, Ananda Theertha Suresh, Sebastian U. Stich, Ameet Talwalkar, Hongyi Wang, Blake E. Woodworth, Shanshan Wu, Felix X. Yu, Honglin Yuan, Manzil Zaheer, Mi Zhang, Tong Zhang, Chunxiang Zheng, Chen Zhu, Wennan Zhu
2021	A*	`conf`	A statistical perspective on distillation. ICML Aditya Krishna Menon, Ankit Singh Rawat, Sashank J. Reddi, Seungyeon Kim, Sanjiv Kumar
2021	A*	`conf`	Adaptive Federated Optimization. ICLR Sashank J. Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konecný, Sanjiv Kumar, Hugh Brendan McMahan
2021	A*	`conf`	Breaking the centralized barrier for cross-device federated learning. NeurIPS Sai Praneeth Karimireddy, Martin Jaggi, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, Ananda Theertha Suresh
2021	J	`jnl`	Disentangling Sampling and Labeling Bias for Learning in Large-Output Spaces. CoRR Ankit Singh Rawat, Aditya Krishna Menon, Wittawat Jitkrittum, Sadeep Jayasumana, Felix X. Yu, Sashank J. Reddi, Sanjiv Kumar
2021	A*	`conf`	Disentangling Sampling and Labeling Bias for Learning in Large-output Spaces. ICML Ankit Singh Rawat, Aditya Krishna Menon, Wittawat Jitkrittum, Sadeep Jayasumana, Felix X. Yu, Sashank J. Reddi, Sanjiv Kumar
2021	J	`jnl`	Distilling Double Descent. CoRR Andrew Cotter, Aditya Krishna Menon, Harikrishna Narasimhan, Ankit Singh Rawat, Sashank J. Reddi, Yichen Zhou
2021	A*	`conf`	Efficient Training of Retrieval Models using Negative Cache. NeurIPS Erik Lindgren, Sashank J. Reddi, Ruiqi Guo, Sanjiv Kumar
2021	A*	`conf`	Federated Composite Optimization. ICML Honglin Yuan, Manzil Zaheer, Sashank J. Reddi
2021	A	`conf`	RankDistil: Knowledge Distillation for Ranking. AISTATS Sashank J. Reddi, Rama Kumar Pasumarthi, Aditya Krishna Menon, Ankit Singh Rawat, Felix X. Yu, Seungyeon Kim, Andreas Veit, Sanjiv Kumar
2020	J	`jnl`	$O(n)$ Connections are Expressive Enough: Universal Approximability of Sparse Transformers. CoRR Chulhee Yun, Yin-Wen Chang, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
2020	J	`jnl`	Adaptive Federated Optimization. CoRR Sashank J. Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konecný, Sanjiv Kumar, H. Brendan McMahan
2020	J	`jnl`	Adaptive Sampling Distributed Stochastic Variance Reduced Gradient for Heterogeneous Distributed Datasets. CoRR Ilqar Ramazanli, Han Nguyen, Hai Pham, Sashank J. Reddi, Barnabás Póczos
2020	A*	`conf`	Are Transformers universal approximators of sequence-to-sequence functions? ICLR Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
2020	A*	`conf`	Can gradient clipping mitigate label noise? ICLR Aditya Krishna Menon, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
2020	J	`jnl`	Doubly-stochastic mining for heterogeneous retrieval. CoRR Ankit Singh Rawat, Aditya Krishna Menon, Andreas Veit, Felix X. Yu, Sashank J. Reddi, Sanjiv Kumar
2020	J	`jnl`	Federated Composite Optimization. CoRR Honglin Yuan, Manzil Zaheer, Sashank J. Reddi
2020	A*	`conf`	Large Batch Optimization for Deep Learning: Training BERT in 76 minutes. ICLR Yang You, Jing Li, Sashank J. Reddi, Jonathan Hseu, Sanjiv Kumar, Srinadh Bhojanapalli, Xiaodan Song, James Demmel, Kurt Keutzer, Cho-Jui Hsieh
2020	A*	`conf`	Learning to Learn by Zeroth-Order Oracle. ICLR Yangjun Ruan, Yuanhao Xiong, Sashank J. Reddi, Sanjiv Kumar, Cho-Jui Hsieh
2020	A*	`conf`	Low-Rank Bottleneck in Multi-head Attention Models. ICML Srinadh Bhojanapalli, Chulhee Yun, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
2020	J	`jnl`	Low-Rank Bottleneck in Multi-head Attention Models. CoRR Srinadh Bhojanapalli, Chulhee Yun, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
2020	J	`jnl`	Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning. CoRR Sai Praneeth Karimireddy, Martin Jaggi, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, Ananda Theertha Suresh
2020	A*	`conf`	O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformers. NeurIPS Chulhee Yun, Yin-Wen Chang, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
2020	A*	`conf`	SCAFFOLD: Stochastic Controlled Averaging for Federated Learning. ICML Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, Ananda Theertha Suresh
2020	A*	`conf`	Why are Adaptive Methods Good for Attention Models? NeurIPS Jingzhao Zhang, Sai Praneeth Karimireddy, Andreas Veit, Seungyeon Kim, Sashank J. Reddi, Sanjiv Kumar, Suvrit Sra
2020	J	`jnl`	Why distillation helps: a statistical perspective. CoRR Aditya Krishna Menon, Ankit Singh Rawat, Sashank J. Reddi, Seungyeon Kim, Sanjiv Kumar
2019	J	`jnl`	AdaCliP: Adaptive Clipping for Private SGD. CoRR Venkatadheeraj Pichapati, Ananda Theertha Suresh, Felix X. Yu, Sashank J. Reddi, Sanjiv Kumar
2019	J	`jnl`	Are Transformers universal approximators of sequence-to-sequence functions? CoRR Chulhee Yun, Srinadh Bhojanapalli, Ankit Singh Rawat, Sashank J. Reddi, Sanjiv Kumar
2019	A*	`conf`	Breaking the Glass Ceiling for Embedding-Based Classifiers for Large Output Spaces. NeurIPS Chuan Guo, Ali Mousavi, Xiang Wu, Daniel Niels Holtmann-Rice, Satyen Kale, Sashank J. Reddi, Sanjiv Kumar
2019	A*	`conf`	Escaping Saddle Points with Adaptive Gradient Methods. ICML Matthew Staib, Sashank J. Reddi, Satyen Kale, Sanjiv Kumar, Suvrit Sra
2019	J	`jnl`	Escaping Saddle Points with Adaptive Gradient Methods. CoRR Matthew Staib, Sashank J. Reddi, Satyen Kale, Sanjiv Kumar, Suvrit Sra
2019	J	`jnl`	Learning to Learn by Zeroth-Order Oracle. CoRR Yangjun Ruan, Yuanhao Xiong, Sashank J. Reddi, Sanjiv Kumar, Cho-Jui Hsieh
2019	J	`jnl`	On the Convergence of Adam and Beyond. CoRR Sashank J. Reddi, Satyen Kale, Sanjiv Kumar
2019	J	`jnl`	SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning. CoRR Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank J. Reddi, Sebastian U. Stich, Ananda Theertha Suresh
2019	A	`conf`	Stochastic Negative Mining for Learning with Large Output Spaces. AISTATS Sashank J. Reddi, Satyen Kale, Felix X. Yu, Daniel Niels Holtmann-Rice, Jiecao Chen, Sanjiv Kumar
2019	J	`jnl`	Why ADAM Beats SGD for Attention Models. CoRR Jingzhao Zhang, Sai Praneeth Karimireddy, Andreas Veit, Seungyeon Kim, Sashank J. Reddi, Sanjiv Kumar, Suvrit Sra
2018	A	`conf`	A Generic Approach for Escaping Saddle points. AISTATS Sashank J. Reddi, Manzil Zaheer, Suvrit Sra, Barnabás Póczos, Francis R. Bach, Ruslan Salakhutdinov, Alexander J. Smola
2018	A*	`conf`	Adaptive Methods for Nonconvex Optimization. NeurIPS Manzil Zaheer, Sashank J. Reddi, Devendra Singh Sachan, Satyen Kale, Sanjiv Kumar
2018	—	`conf`	On the Convergence of Adam and Beyond. ICLR Sashank J. Reddi, Satyen Kale, Sanjiv Kumar
2018	J	`jnl`	Stochastic Negative Mining for Learning with Large Output Spaces. CoRR Sashank J. Reddi, Satyen Kale, Felix X. Yu, Daniel N. Holtmann-Rice, Jiecao Chen, Sanjiv Kumar
2017	J	`jnl`	A Generic Approach for Escaping Saddle points. CoRR Sashank J. Reddi, Manzil Zaheer, Suvrit Sra, Barnabás Póczos, Francis R. Bach, Ruslan Salakhutdinov, Alexander J. Smola
2017	—	`—`	New Optimization Methods for Modern Machine Learning. Sashank J. Reddi
2016	J	`jnl`	AIDE: Fast and Communication Efficient Distributed Optimization. CoRR Sashank J. Reddi, Jakub Konecný, Peter Richtárik, Barnabás Póczos, Alexander J. Smola
2016	J	`jnl`	Fast Incremental Method for Nonconvex Optimization. CoRR Sashank J. Reddi, Suvrit Sra, Barnabás Póczos, Alexander J. Smola
2016	J	`jnl`	Fast Stochastic Methods for Nonsmooth Nonconvex Optimization. CoRR Sashank J. Reddi, Suvrit Sra, Barnabás Póczos, Alexander J. Smola
2016	—	`conf`	Fast incremental method for smooth nonconvex optimization. CDC Sashank J. Reddi, Suvrit Sra, Barnabás Póczos, Alexander J. Smola
2016	J	`jnl`	Fast stochastic optimization on Riemannian manifolds. CoRR Hongyi Zhang, Sashank J. Reddi, Suvrit Sra
2016	—	`conf`	Proximal Stochastic Methods for Nonsmooth Nonconvex Finite-Sum Optimization. NIPS Sashank J. Reddi, Suvrit Sra, Barnabás Póczos, Alexander J. Smola
2016	—	`conf`	Riemannian SVRG: Fast Stochastic Optimization on Riemannian Manifolds. NIPS Hongyi Zhang, Sashank J. Reddi, Suvrit Sra
2016	J	`jnl`	Stochastic Frank-Wolfe Methods for Nonconvex Optimization. CoRR Sashank J. Reddi, Suvrit Sra, Barnabás Póczos, Alexander J. Smola
2016	—	`conf`	Stochastic Frank-Wolfe methods for nonconvex optimization. Allerton Sashank J. Reddi, Suvrit Sra, Barnabás Póczos, Alexander J. Smola
2016	A*	`conf`	Stochastic Variance Reduction for Nonconvex Optimization. ICML Sashank J. Reddi, Ahmed Hefny, Suvrit Sra, Barnabás Póczos, Alexander J. Smola
2016	J	`jnl`	Stochastic Variance Reduction for Nonconvex Optimization. CoRR Sashank J. Reddi, Ahmed Hefny, Suvrit Sra, Barnabás Póczos, Alexander J. Smola
2016	—	`conf`	Variance Reduction in Stochastic Gradient Langevin Dynamics. NIPS Kumar Avinava Dubey, Sashank J. Reddi, Sinead A. Williamson, Barnabás Póczos, Alexander J. Smola, Eric P. Xing
2015	J	`jnl`	Adaptivity and Computation-Statistics Tradeoffs for Kernel and Distance based High Dimensional Two Sample Testing. CoRR Aaditya Ramdas, Sashank J. Reddi, Barnabás Póczos, Aarti Singh, Larry A. Wasserman
2015	A	`conf`	Communication Efficient Coresets for Empirical Loss Minimization. UAI Sashank J. Reddi, Barnabás Póczos, Alexander J. Smola
2015	A	`conf`	Large-scale randomized-coordinate descent methods with non-separable linear constraints. UAI Sashank J. Reddi, Ahmed Hefny, Carlton Downey, Avinava Dubey, Suvrit Sra
2015	—	`conf`	On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants. NIPS Sashank J. Reddi, Ahmed Hefny, Suvrit Sra, Barnabás Póczos, Alexander J. Smola
2015	J	`jnl`	On Variance Reduction in Stochastic Gradient Descent and its Asynchronous Variants. CoRR Sashank J. Reddi, Ahmed Hefny, Suvrit Sra, Barnabás Póczos, Alexander J. Smola
2015	A	`conf`	On the High Dimensional Power of a Linear-Time Two Sample Test under Mean-shift Alternatives. AISTATS Sashank J. Reddi, Aaditya Ramdas, Barnabás Póczos, Aarti Singh, Larry A. Wasserman
2014	J	`jnl`	Kernel MMD, the Median Heuristic and Distance Correlation in High Dimensions. CoRR Sashank J. Reddi, Aaditya Ramdas, Barnabás Póczos, Aarti Singh, Larry A. Wasserman
2014	J	`jnl`	On the High-dimensional Power of Linear-time Kernel Two-Sample Testing under Mean-difference Alternatives. CoRR Aaditya Ramdas, Sashank J. Reddi, Barnabás Póczos, Aarti Singh, Larry A. Wasserman
2014	A	`conf`	k-NN Regression on Functional Data with Incomplete Observations. UAI Sashank J. Reddi, Barnabás Póczos
2013	—	`conf`	Scale Invariant Conditional Dependence Measures. ICML (3) Sashank J. Reddi, Barnabás Póczos