Samuel L. Smith

38 papers A* 9Journal 25Unranked 4

Year	Rank	Type	Title / Venue / Authors
2024	J	`jnl`	Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models. CoRR Soham De, Samuel L. Smith, Anushan Fernando, Aleksandar Botev, George-Cristian Muraru, Albert Gu, Ruba Haroun, Leonard Berrada, Yutian Chen, Srivatsan Srinivasan, Guillaume Desjardins, Arnaud Doucet, David Budden, Yee Whye Teh, Razvan Pascanu, Nando de Freitas, Caglar Gulcehre
2024	J	`jnl`	RecurrentGemma: Moving Past Transformers for Efficient Open Language Models. CoRR Aleksandar Botev, Soham De, Samuel L. Smith, Anushan Fernando, George-Cristian Muraru, Ruba Haroun, Leonard Berrada, Razvan Pascanu, Pier Giuseppe Sessa, Robert Dadashi, Léonard Hussenot, Johan Ferret, Sertan Girgin, Olivier Bachem, Alek Andreev, Kathleen Kenealy, Thomas Mesnard, Cassidy Hardin, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Armand Joulin, Noah Fiedel, Evan Senter, Yutian Chen, Srivatsan Srinivasan, Guillaume Desjardins, David Budden, Arnaud Doucet, Sharad Vikram, Adam Paszke, Trevor Gale, Sebastian Borgeaud, Charlie Chen, Andy Brock, Antonia Paterson, Jenny Brennan, Meg Risdal, Raj Gundluru, Nesh Devanathan, Paul Mooney, Nilay Chauhan, Phil Culliton, Luiz Gustavo Martins, Elisa Bandy, David Huntsperger, Glenn Cameron, Arthur Zucker, Tris Warkentin, Ludovic Peran, Minh Giang, Zoubin Ghahramani, Clément Farabet, Koray Kavukcuoglu, Demis Hassabis, Raia Hadsell, Yee Whye Teh, Nando de Frietas
2024	A*	`conf`	Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues. ICML Antonio Orvieto, Soham De, Caglar Gulcehre, Razvan Pascanu, Samuel L. Smith
2023	J	`jnl`	ConvNets Match Vision Transformers at Scale. CoRR Samuel L. Smith, Andrew Brock, Leonard Berrada, Soham De
2023	A*	`conf`	Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation. ICLR Bobby He, James Martens, Guodong Zhang, Aleksandar Botev, Andrew Brock, Samuel L. Smith, Yee Whye Teh
2023	J	`jnl`	Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation. CoRR Bobby He, James Martens, Guodong Zhang, Aleksandar Botev, Andrew Brock, Samuel L. Smith, Yee Whye Teh
2023	J	`jnl`	Differentially Private Diffusion Models Generate Useful Synthetic Images. CoRR Sahra Ghalebikesabi, Leonard Berrada, Sven Gowal, Ira Ktena, Robert Stanforth, Jamie Hayes, Soham De, Samuel L. Smith, Olivia Wiles, Borja Balle
2023	J	`jnl`	On the Universality of Linear Recurrences Followed by Nonlinear Projections. CoRR Antonio Orvieto, Soham De, Çaglar Gülçehre, Razvan Pascanu, Samuel L. Smith
2023	A*	`conf`	Resurrecting Recurrent Neural Networks for Long Sequences. ICML Antonio Orvieto, Samuel L. Smith, Albert Gu, Anushan Fernando, Çaglar Gülçehre, Razvan Pascanu, Soham De
2023	J	`jnl`	Resurrecting Recurrent Neural Networks for Long Sequences. CoRR Antonio Orvieto, Samuel L. Smith, Albert Gu, Anushan Fernando, Çaglar Gülçehre, Razvan Pascanu, Soham De
2023	J	`jnl`	Unlocking Accuracy and Fairness in Differentially Private Image Classification. CoRR Leonard Berrada, Soham De, Judy Hanwen Shen, Jamie Hayes, Robert Stanforth, David Stutz, Pushmeet Kohli, Samuel L. Smith, Borja Balle
2022	J	`jnl`	Unlocking High-Accuracy Differentially Private Image Classification through Scale. CoRR Soham De, Leonard Berrada, Jamie Hayes, Samuel L. Smith, Borja Balle
2021	J	`jnl`	A study on the plasticity of neural networks. CoRR Tudor Berariu, Wojciech Czarnecki, Soham De, Jörg Bornschein, Samuel L. Smith, Razvan Pascanu, Claudia Clopath
2021	A*	`conf`	Characterizing signal propagation to close the performance gap in unnormalized ResNets. ICLR Andrew Brock, Soham De, Samuel L. Smith
2021	J	`jnl`	Characterizing signal propagation to close the performance gap in unnormalized ResNets. CoRR Andrew Brock, Soham De, Samuel L. Smith
2021	J	`jnl`	Drawing Multiple Augmentation Samples Per Image During Training Efficiently Decreases Test Error. CoRR Stanislav Fort, Andrew Brock, Razvan Pascanu, Soham De, Samuel L. Smith
2021	A*	`conf`	High-Performance Large-Scale Image Recognition Without Normalization. ICML Andy Brock, Soham De, Samuel L. Smith, Karen Simonyan
2021	J	`jnl`	High-Performance Large-Scale Image Recognition Without Normalization. CoRR Andrew Brock, Soham De, Samuel L. Smith, Karen Simonyan
2021	A*	`conf`	On the Origin of Implicit Regularization in Stochastic Gradient Descent. ICLR Samuel L. Smith, Benoit Dherin, David G. T. Barrett, Soham De
2021	J	`jnl`	On the Origin of Implicit Regularization in Stochastic Gradient Descent. CoRR Samuel L. Smith, Benoit Dherin, David G. T. Barrett, Soham De
2020	J	`jnl`	BYOL works even without batch statistics. CoRR Pierre H. Richemond, Jean-Bastien Grill, Florent Altché, Corentin Tallec, Florian Strub, Andrew Brock, Samuel L. Smith, Soham De, Razvan Pascanu, Bilal Piot, Michal Valko
2020	J	`jnl`	Batch Normalization Biases Deep Residual Networks Towards Shallow Paths. CoRR Soham De, Samuel L. Smith
2020	A*	`conf`	Batch Normalization Biases Residual Blocks Towards the Identity Function in Deep Networks. NeurIPS Soham De, Samuel L. Smith
2020	J	`jnl`	Cold Posteriors and Aleatoric Uncertainty. CoRR Ben Adlam, Jasper Snoek, Samuel L. Smith
2020	A*	`conf`	On the Generalization Benefit of Noise in Stochastic Gradient Descent. ICML Samuel L. Smith, Erich Elsen, Soham De
2020	J	`jnl`	On the Generalization Benefit of Noise in Stochastic Gradient Descent. CoRR Samuel L. Smith, Erich Elsen, Soham De
2019	A*	`conf`	The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study. ICML Daniel S. Park, Jascha Sohl-Dickstein, Quoc V. Le, Samuel L. Smith
2019	J	`jnl`	The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study. CoRR Daniel S. Park, Jascha Sohl-Dickstein, Quoc V. Le, Samuel L. Smith
2018	—	`conf`	A Bayesian Perspective on Generalization and Stochastic Gradient Descent. ICLR (Poster) Samuel L. Smith, Quoc V. Le
2018	—	`conf`	Decoding Decoders: Finding Optimal Representation Spaces for Unsupervised Similarity Tasks. ICLR (Workshop) Vitalii Zhelezniak, Dan Busbridge, April Shen, Samuel L. Smith, Nils Y. Hammerla
2018	J	`jnl`	Decoding Decoders: Finding Optimal Representation Spaces for Unsupervised Similarity Tasks. CoRR Vitalii Zhelezniak, Dan Busbridge, April Shen, Samuel L. Smith, Nils Y. Hammerla
2018	—	`conf`	Don't Decay the Learning Rate, Increase the Batch Size. ICLR (Poster) Samuel L. Smith, Pieter-Jan Kindermans, Chris Ying, Quoc V. Le
2018	J	`jnl`	Stochastic natural gradient descent draws posterior samples in function space. CoRR Samuel L. Smith, Daniel Duckworth, Quoc V. Le, Jascha Sohl-Dickstein
2017	J	`jnl`	A Bayesian Perspective on Generalization and Stochastic Gradient Descent. CoRR Samuel L. Smith, Quoc V. Le
2017	J	`jnl`	Don't Decay the Learning Rate, Increase the Batch Size. CoRR Samuel L. Smith, Pieter-Jan Kindermans, Quoc V. Le
2017	—	`conf`	Offline bilingual word vectors, orthogonal transformations and the inverted softmax. ICLR (Poster) Samuel L. Smith, David H. P. Turban, Steven Hamblin, Nils Y. Hammerla
2017	J	`jnl`	Offline bilingual word vectors, orthogonal transformations and the inverted softmax. CoRR Samuel L. Smith, David H. P. Turban, Steven Hamblin, Nils Y. Hammerla
2016	J	`jnl`	Monte Carlo Sort for unreliable human comparisons. CoRR Samuel L. Smith