Frederik Kunstner

18 papers A* 7A 1Journal 10
YearRankTypeTitle / Venue / Authors
2025 J jnl
Scaling Laws for Gradient Descent and Sign Descent for Linear Bigram Models under Zipf's Law.
CoRR
Frederik Kunstner, Francis Bach
2024 A* conf
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models.
NeurIPS
Frederik Kunstner, Alan Milligan, Robin Yadav, Mark Schmidt, Alberto Bietti
2024 J jnl
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models.
CoRR
Frederik Kunstner, Robin Yadav, Alan Milligan, Mark Schmidt, Alberto Bietti
2023 J jnl
Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be.
CoRR
Frederik Kunstner, Jacques Chen, Jonathan Wilder Lavington, Mark Schmidt
2023 A* conf
Noise Is Not the Main Factor Behind the Gap Between Sgd and Adam on Transformers, But Sign Descent Might Be.
ICLR
Frederik Kunstner, Jacques Chen, Jonathan Wilder Lavington, Mark Schmidt
2023 A* conf
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking.
NeurIPS
Frederik Kunstner, Victor Sanches Portella, Mark Schmidt, Nicholas J. A. Harvey
2023 J jnl
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking.
CoRR
Frederik Kunstner, Victor S. Portella, Mark Schmidt, Nick Harvey
2022 A* conf
Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent (Extended Abstract).
IJCAI
Frederik Kunstner, Raunak Kumar, Mark Schmidt
2021 J jnl
Convergence Rates for the MAP of an Exponential Family and Stochastic Mirror Descent - an Open Problem.
CoRR
Rémi Le Priol, Frederik Kunstner, Damien Scieur, Simon Lacoste-Julien
2021 A conf
Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent.
AISTATS
Frederik Kunstner, Raunak Kumar, Mark Schmidt
2020 J jnl
Adaptive Gradient Methods Converge Faster with Over-Parameterization (and you can do a line-search).
CoRR
Sharan Vaswani, Frederik Kunstner, Issam H. Laradji, Si Yi Meng, Mark Schmidt, Simon Lacoste-Julien
2020 A* conf
BackPACK: Packing more into Backprop.
ICLR
Felix Dangel, Frederik Kunstner, Philipp Hennig
2020 J jnl
Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent.
CoRR
Frederik Kunstner, Raunak Kumar, Mark Schmidt
2019 J jnl
BackPACK: Packing more into backprop.
CoRR
Felix Dangel, Frederik Kunstner, Philipp Hennig
2019 J jnl
Limitations of the Empirical Fisher Approximation.
CoRR
Frederik Kunstner, Lukas Balles, Philipp Hennig
2019 A* conf
Limitations of the empirical Fisher approximation for natural gradient descent.
NeurIPS
Frederik Kunstner, Philipp Hennig, Lukas Balles
2018 A* conf
SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient.
NeurIPS
Aaron Mishkin, Frederik Kunstner, Didrik Nielsen, Mark Schmidt, Mohammad Emtiyaz Khan
2018 J jnl
SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient.
CoRR
Aaron Mishkin, Frederik Kunstner, Didrik Nielsen, Mark Schmidt, Mohammad Emtiyaz Khan