James Martens

49 papers A* 15A 1B 1Journal 24Unranked 6
YearRankTypeTitle / Venue / Authors
2026 J jnl
Orthogonal Self-Attention.
CoRR
Leo Zhang, James Martens
2025 J jnl
Cutting the Skip: Training Residual-Free Transformers.
CoRR
Yiping Ji, James Martens, Jianqiao Zheng, Ziqin Zhou, Peyman Moghadam, Xinyu Zhang, Hemanth Saratchandran, Simon Lucey
2025 J jnl
Optimizers Qualitatively Alter Solutions And We Should Leverage This.
CoRR
Razvan Pascanu, Clare Lyle, Ionut-Vlad Modoranu, Naima Elosegui Borras, Dan Alistarh, Petar Velickovic, Sarath Chandar, Soham De, James Martens
2024 — conf
Disentangling the Causes of Plasticity Loss in Neural Networks.
CoLLAs
Clare Lyle, Zeyu Zheng, Khimya Khetarpal, Hado van Hasselt, Razvan Pascanu, James Martens, Will Dabney
2024 J jnl
Disentangling the Causes of Plasticity Loss in Neural Networks.
CoRR
Clare Lyle, Zeyu Zheng, Khimya Khetarpal, Hado van Hasselt, Razvan Pascanu, James Martens, Will Dabney
2024 A* conf
Normalization and effective learning rates in reinforcement learning.
NeurIPS
Clare Lyle, Zeyu Zheng, Khimya Khetarpal, James Martens, Hado Philip van Hasselt, Razvan Pascanu, Will Dabney
2024 J jnl
Normalization and effective learning rates in reinforcement learning.
CoRR
Clare Lyle, Zeyu Zheng, Khimya Khetarpal, James Martens, Hado van Hasselt, Razvan Pascanu, Will Dabney
2023 A* conf
Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation.
ICLR
Bobby He, James Martens, Guodong Zhang, Aleksandar Botev, Andrew Brock, Samuel L. Smith, Yee Whye Teh
2023 J jnl
Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation.
CoRR
Bobby He, James Martens, Guodong Zhang, Aleksandar Botev, Andrew Brock, Samuel L. Smith, Yee Whye Teh
2023 A* conf
Pre-training via Denoising for Molecular Property Prediction.
ICLR
Sheheryar Zaidi, Michael Schaarschmidt, James Martens, Hyunjik Kim, Yee Whye Teh, Alvaro Sanchez-Gonzalez, Peter W. Battaglia, Razvan Pascanu, Jonathan Godwin
2022 A* conf
Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers.
ICLR
Guodong Zhang, Aleksandar Botev, James Martens
2022 J jnl
Deep Learning without Shortcuts: Shaping the Kernel with Tailored Rectifiers.
CoRR
Guodong Zhang, Aleksandar Botev, James Martens
2022 J jnl
Pre-training via Denoising for Molecular Property Prediction.
CoRR
Sheheryar Zaidi, Michael Schaarschmidt, James Martens, Hyunjik Kim, Yee Whye Teh, Alvaro Sanchez-Gonzalez, Peter W. Battaglia, Razvan Pascanu, Jonathan Godwin
2021 J jnl
On the validity of kernel approximations for orthogonally-initialized neural networks.
CoRR
James Martens
2021 J jnl
Rapid training of deep neural networks without skip connections or normalization layers using Deep Kernel Shaping.
CoRR
James Martens, Andy Ballard, Guillaume Desjardins, Grzegorz Swirszcz, Valentin Dalibard, Jascha Sohl-Dickstein, Samuel S. Schoenholz
2020 B conf
Blockchain-based Verifiable Credential Sharing with Selective Disclosure.
TrustCom
Rahma Mukta, James Martens, Hye-Young Paik, Qinghua Lu, Salil S. Kanhere
2020 J jnl
New Insights and Perspectives on the Natural Gradient Method.
J. Mach. Learn. Res.
James Martens
2019 A* conf
Adversarial Robustness through Local Linearization.
NeurIPS
Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, Pushmeet Kohli
2019 J jnl
Adversarial Robustness through Local Linearization.
CoRR
Chongli Qin, James Martens, Sven Gowal, Dilip Krishnan, Krishnamurthy Dvijotham, Alhussein Fawzi, Soham De, Robert Stanforth, Pushmeet Kohli
2019 J jnl
Differentiable Game Mechanics.
CoRR
Alistair Letcher, David Balduzzi, Sébastien Racanière, James Martens, Jakob N. Foerster, Karl Tuyls, Thore Graepel
2019 J jnl
Differentiable Game Mechanics.
J. Mach. Learn. Res.
Alistair Letcher, David Balduzzi, Sébastien Racanière, James Martens, Jakob N. Foerster, Karl Tuyls, Thore Graepel
2019 A* conf
Fast Convergence of Natural Gradient Descent for Over-Parameterized Neural Networks.
NeurIPS
Guodong Zhang, James Martens, Roger B. Grosse
2019 J jnl
Fast Convergence of Natural Gradient Descent for Overparameterized Neural Networks.
CoRR
Guodong Zhang, James Martens, Roger B. Grosse
2019 J jnl
On the Variance of Unbiased Online Recurrent Optimization.
CoRR
Tim Cooijmans, James Martens
2019 A* conf
Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model.
NeurIPS
Guodong Zhang, Lala Li, Zachary Nado, James Martens, Sushant Sachdeva, George E. Dahl, Christopher J. Shallue, Roger B. Grosse
2019 J jnl
Which Algorithmic Choices Matter at Which Batch Sizes? Insights From a Noisy Quadratic Model.
CoRR
Guodong Zhang, Lala Li, Zachary Nado, James Martens, Sushant Sachdeva, George E. Dahl, Christopher J. Shallue, Roger B. Grosse
2018 — conf
Kronecker-factored Curvature Approximations for Recurrent Neural Networks.
ICLR (Poster)
James Martens, Jimmy Ba, Matt Johnson
2018 — conf
Stochastic Gradient Langevin dynamics that Exploit Neural Network Structure.
ICLR (Workshop)
Zachary Nado, Jasper Snoek, Roger B. Grosse, David Duvenaud, Bowen Xu, James Martens
2018 A* conf
The Mechanics of n-Player Differentiable Games.
ICML
David Balduzzi, Sébastien Racanière, James Martens, Jakob N. Foerster, Karl Tuyls, Thore Graepel
2018 J jnl
The Mechanics of n-Player Differentiable Games.
CoRR
David Balduzzi, Sébastien Racanière, James Martens, Jakob N. Foerster, Karl Tuyls, Thore Graepel
2017 — conf
Distributed Second-Order Optimization using Kronecker-Factored Approximations.
ICLR (Poster)
Jimmy Ba, Roger B. Grosse, James Martens
2016 A* conf
A Kronecker-factored approximate Fisher matrix for convolution layers.
ICML
Roger B. Grosse, James Martens
2016 J jnl
A Kronecker-factored approximate Fisher matrix for convolution layers.
CoRR
Roger B. Grosse, James Martens
2016 — —
Second-order Optimization for Neural Networks.
James Martens
2015 J jnl
Adding Gradient Noise Improves Learning for Very Deep Networks.
CoRR
Arvind Neelakantan, Luke Vilnis, Quoc V. Le, Ilya Sutskever, Lukasz Kaiser, Karol Kurach, James Martens
2015 A* conf
Optimizing Neural Networks with Kronecker-factored Approximate Curvature.
ICML
James Martens, Roger B. Grosse
2015 J jnl
Optimizing Neural Networks with Kronecker-factored Approximate Curvature.
CoRR
James Martens, Roger B. Grosse
2014 J jnl
New perspectives on the natural gradient method.
CoRR
James Martens
2014 J jnl
On the Expressive Efficiency of Sum Product Networks.
CoRR
James Martens, Venkatesh Medabalimi
2013 — conf
On the Expressive Power of Restricted Boltzmann Machines.
NIPS
James Martens, Arkadev Chattopadhyay, Toniann Pitassi, Richard S. Zemel
2013 — conf
On the importance of initialization and momentum in deep learning.
ICML (3)
Ilya Sutskever, James Martens, George E. Dahl, Geoffrey E. Hinton
2012 A* conf
Estimating the Hessian by Back-propagating Curvature.
ICML
James Martens, Ilya Sutskever, Kevin Swersky
2012 — ch.
Training Deep and Recurrent Networks with Hessian-Free Optimization.
Neural Networks: Tricks of the Trade (2nd ed.)
James Martens, Ilya Sutskever
2011 A* conf
Generating Text with Recurrent Neural Networks.
ICML
Ilya Sutskever, James Martens, Geoffrey E. Hinton
2011 A* conf
Learning Recurrent Neural Networks with Hessian-Free Optimization.
ICML
James Martens, Ilya Sutskever
2011 J jnl
Normalization for probabilistic inference with neurons.
Biol. Cybern.
Chris Eliasmith, James Martens
2010 A* conf
Deep learning via Hessian-free optimization.
ICML
James Martens
2010 A* conf
Learning the Linear Dynamical System with ASOS.
ICML
James Martens
2010 A conf
Parallelizable Sampling of Markov Random Fields.
AISTATS
James Martens, Ilya Sutskever