Johannes von Oswald

41 papers A* 14Journal 26

Year	Rank	Type	Title / Venue / Authors
2026	J	`jnl`	From Growing to Looping: A Unified View of Iterative Computation in LLMs. CoRR Ferdinand Kapl, Emmanouil Angelis, Kaitlin Maile, Johannes von Oswald, Stefan Bauer
2026	J	`jnl`	From Words to Amino Acids: Does the Curse of Depth Persist? CoRR Aleena Siji, Amir Mohammad Karimi-Mamaghan, Ferdinand Kapl, Tobias Höppe, Emmanouil Angelis, Andrea Dittadi, Maurice Brenner, Michael Heinzinger, Karl Henrik Johansson, Kaitlin Maile, Johannes von Oswald, Stefan Bauer
2025	J	`jnl`	Do Depth-Grown Models Overcome the Curse of Depth? An In-Depth Analysis. CoRR Ferdinand Kapl, Emmanouil Angelis, Tobias Höppe, Kaitlin Maile, Johannes von Oswald, Nino Scherrer, Stefan Bauer
2025	J	`jnl`	Emergent temporal abstractions in autoregressive models enable hierarchical reinforcement learning. CoRR Seijin Kobayashi, Yanick Schimpf, Maximilian Schlegel, Angelika Steger, Maciej Wolczyk, Johannes von Oswald, Nino Scherrer, Kaitlin Maile, Guillaume Lajoie, Blake A. Richards, Rif A. Saurous, James Manyika, Blaise Agüera y Arcas, Alexander Meulemans, João Sacramento
2025	A*	`conf`	Learning Randomized Algorithms with Transformers. ICLR Johannes von Oswald, Seijin Kobayashi, Yassir Akram, Angelika Steger
2025	J	`jnl`	MesaNet: Sequence Modeling by Locally Optimal Test-Time Training. CoRR Johannes von Oswald, Nino Scherrer, Seijin Kobayashi, Luca Versari, Songlin Yang, Maximilian Schlegel, Kaitlin Maile, Yanick Schimpf, Oliver Sieberling, Alexander Meulemans, Rif A. Saurous, Guillaume Lajoie, Charlotte Frenkel, Razvan Pascanu, Blaise Agüera y Arcas, João Sacramento
2025	A*	`conf`	Multi-agent cooperation through learning-aware policy gradients. ICLR Alexander Meulemans, Seijin Kobayashi, Johannes von Oswald, Nino Scherrer, Eric Elmoznino, Blake Aaron Richards, Guillaume Lajoie, Blaise Agüera y Arcas, João Sacramento
2025	J	`jnl`	Understanding In-Context Learning of Linear Models in Transformers Through an Adversarial Lens. Trans. Mach. Learn. Res. Usman Anwar, Johannes von Oswald, Louis Kirsch, David Krueger, Spencer Frei
2024	J	`jnl`	Adversarial Robustness of In-Context Learning in Transformers for Linear Regression. CoRR Usman Anwar, Johannes von Oswald, Louis Kirsch, David Krueger, Spencer Frei
2024	A*	`conf`	Discovering modular solutions that generalize compositionally. ICLR Simon Schug, Seijin Kobayashi, Yassir Akram, Maciej Wolczyk, Alexandra Maria Proca, Johannes von Oswald, Razvan Pascanu, João Sacramento, Angelika Steger
2024	—	`—`	Interpretability of Learning Algorithms Encoded in Deep Neural Networks. Johannes von Oswald
2024	J	`jnl`	Learning Randomized Algorithms with Transformers. CoRR Johannes von Oswald, Seijin Kobayashi, Yassir Akram, Angelika Steger
2024	A*	`conf`	Linear Transformers are Versatile In-Context Learners. NeurIPS Max Vladymyrov, Johannes von Oswald, Mark Sandler, Rong Ge
2024	J	`jnl`	Linear Transformers are Versatile In-Context Learners. CoRR Max Vladymyrov, Johannes von Oswald, Mark Sandler, Rong Ge
2024	J	`jnl`	Multi-agent cooperation through learning-aware policy gradients. CoRR Alexander Meulemans, Seijin Kobayashi, Johannes von Oswald, Nino Scherrer, Eric Elmoznino, Blake A. Richards, Guillaume Lajoie, Blaise Agüera y Arcas, João Sacramento
2024	J	`jnl`	State Soup: In-Context Skill Learning, Retrieval and Mixing. CoRR Maciej Pióro, Maciej Wolczyk, Razvan Pascanu, Johannes von Oswald, João Sacramento
2024	A*	`conf`	Weight decay induces low-rank attention layers. NeurIPS Seijin Kobayashi, Yassir Akram, Johannes von Oswald
2024	J	`jnl`	Weight decay induces low-rank attention layers. CoRR Seijin Kobayashi, Yassir Akram, Johannes von Oswald
2024	J	`jnl`	When can transformers compositionally generalize in-context? CoRR Seijin Kobayashi, Simon Schug, Yassir Akram, Florian Redhardt, Johannes von Oswald, Razvan Pascanu, Guillaume Lajoie, João Sacramento
2023	J	`jnl`	Discovering modular solutions that generalize compositionally. CoRR Simon Schug, Seijin Kobayashi, Yassir Akram, Maciej Wolczyk, Alexandra Maria Proca, Johannes von Oswald, Razvan Pascanu, João Sacramento, Angelika Steger
2023	J	`jnl`	Gated recurrent neural networks discover attention. CoRR Nicolas Zucchet, Seijin Kobayashi, Yassir Akram, Johannes von Oswald, Maxime Larcher, Angelika Steger, João Sacramento
2023	A*	`conf`	Transformers Learn In-Context by Gradient Descent. ICML Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, Max Vladymyrov
2023	J	`jnl`	Uncovering mesa-optimization algorithms in Transformers. CoRR Johannes von Oswald, Eyvind Niklasson, Maximilian Schlegel, Seijin Kobayashi, Nicolas Zucchet, Nino Scherrer, Nolan Miller, Mark Sandler, Blaise Agüera y Arcas, Max Vladymyrov, Razvan Pascanu, João Sacramento
2022	A*	`conf`	A contrastive rule for meta-learning. NeurIPS Nicolas Zucchet, Simon Schug, Johannes von Oswald, Dominic Zhao, João Sacramento
2022	A*	`conf`	Disentangling the Predictive Variance of Deep Ensembles through the Neural Tangent Kernel. NeurIPS Seijin Kobayashi, Pau Vilimelis Aceituno, Johannes von Oswald
2022	J	`jnl`	Disentangling the Predictive Variance of Deep Ensembles through the Neural Tangent Kernel. CoRR Seijin Kobayashi, Pau Vilimelis Aceituno, Johannes von Oswald
2022	J	`jnl`	Random initialisations performing above chance and how to find them. CoRR Frederik Benzing, Simon Schug, Robert Meier, Johannes von Oswald, Yassir Akram, Nicolas Zucchet, Laurence Aitchison, Angelika Steger
2022	J	`jnl`	The least-control principle for learning at equilibrium. CoRR Alexander Meulemans, Nicolas Zucchet, Seijin Kobayashi, Johannes von Oswald, João Sacramento
2022	A*	`conf`	The least-control principle for local learning at equilibrium. NeurIPS Alexander Meulemans, Nicolas Zucchet, Seijin Kobayashi, Johannes von Oswald, João Sacramento
2022	J	`jnl`	Transformers learn in-context by gradient descent. CoRR Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordvintsev, Andrey Zhmoginov, Max Vladymyrov
2021	J	`jnl`	A contrastive rule for meta-learning. CoRR Nicolas Zucchet, Simon Schug, Johannes von Oswald, Dominic Zhao, João Sacramento
2021	A*	`conf`	Continual learning in recurrent neural networks. ICLR Benjamin Ehret, Christian Henning, Maria R. Cervera, Alexander Meulemans, Johannes von Oswald, Benjamin F. Grewe
2021	A*	`conf`	Learning where to learn: Gradient sparsity in meta and continual learning. NeurIPS Johannes von Oswald, Dominic Zhao, Seijin Kobayashi, Simon Schug, Massimo Caccia, Nicolas Zucchet, João Sacramento
2021	J	`jnl`	Learning where to learn: Gradient sparsity in meta and continual learning. CoRR Johannes von Oswald, Dominic Zhao, Seijin Kobayashi, Simon Schug, Massimo Caccia, Nicolas Zucchet, João Sacramento
2021	A*	`conf`	Neural networks with late-phase weights. ICLR Johannes von Oswald, Seijin Kobayashi, João Sacramento, Alexander Meulemans, Christian Henning, Benjamin F. Grewe
2021	A*	`conf`	Posterior Meta-Replay for Continual Learning. NeurIPS Christian Henning, Maria R. Cervera, Francesco D'Angelo, Johannes von Oswald, Regina Traber, Benjamin Ehret, Seijin Kobayashi, Benjamin F. Grewe, João Sacramento
2021	J	`jnl`	Posterior Meta-Replay for Continual Learning. CoRR Christian Henning, Maria R. Cervera, Francesco D'Angelo, Johannes von Oswald, Regina Traber, Benjamin Ehret, Seijin Kobayashi, João Sacramento, Benjamin F. Grewe
2020	J	`jnl`	Continual Learning in Recurrent Neural Networks with Hypernetworks. CoRR Benjamin Ehret, Christian Henning, Maria R. Cervera, Alexander Meulemans, Johannes von Oswald, Benjamin F. Grewe
2020	A*	`conf`	Continual learning with hypernetworks. ICLR Johannes von Oswald, Christian Henning, João Sacramento, Benjamin F. Grewe
2020	J	`jnl`	Economical ensembles with hypernetworks. CoRR João Sacramento, Johannes von Oswald, Seijin Kobayashi, Christian Henning, Benjamin F. Grewe
2019	J	`jnl`	Continual learning with hypernetworks. CoRR Johannes von Oswald, Christian Henning, João Sacramento, Benjamin F. Grewe