Kazuki Irie - RankMe

81 papers A* 14A 8C 2Misc 8Journal 42Unranked 6

Year	Rank	Type	Title / Venue / Authors
2026	J	`jnl`	Fast weight programming and linear transformers: from machine learning to neurobiology. Trans. Mach. Learn. Res. Kazuki Irie, Samuel J. Gershman
2026	J	`jnl`	Learning Abstractions for Hierarchical Planning in Program-Synthesis Agents. CoRR Zergham Ahmed, Kazuki Irie, Joshua B. Tenenbaum, Christopher Bates, Samuel J. Gershman
2026	J	`jnl`	Why Depth Matters in Parallelizable Sequence Models: A Lie Algebraic View. CoRR Gyuryang Heo, Timothy Ngotiaoco, Kazuki Irie, Samuel J. Gershman, Bernardo Sabatini
2025	J	`jnl`	Blending Complementary Memory Systems in Hybrid Quadratic-Linear Transformers. CoRR Kazuki Irie, Morris Yau, Samuel J. Gershman
2025	J	`jnl`	Fast weight programming and linear transformers: from machine learning to neurobiology. CoRR Kazuki Irie, Samuel J. Gershman
2025	J	`jnl`	Key-value memory in the brain. CoRR Samuel J. Gershman, Ila Fiete, Kazuki Irie
2025	J	`jnl`	Metalearning Continual Learning Algorithms. Trans. Mach. Learn. Res. Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber
2025	J	`jnl`	Mindstorms in Natural Language-Based Societies of Mind. Comput. Vis. Media Mingchen Zhuge, Haozhe Liu, Francesco Faccio, Dylan R. Ashley, Róbert Csordás, Anand Gopalakrishnan, Abdullah Hamdi, Hasan Abed Al Kader Hammoud, Vincent Herrmann, Kazuki Irie, Louis Kirsch, Bing Li, Guohao Li, Shuming Liu, Jinjie Mai, Piotr Piekos, Aditya A. Ramesh, Imanol Schlag, Weimin Shi, Aleksandar Stanic, Wenyi Wang, Yuhui Wang, Mengmeng Xu, Deng-Ping Fan, Bernard Ghanem, Jürgen Schmidhuber
2025	J	`jnl`	Sequential-Parallel Duality in Prefix Scannable Models. CoRR Morris Yau, Sharut Gupta, Valerie Engelmayer, Kazuki Irie, Stefanie Jegelka, Jacob Andreas
2025	—	`conf`	Why Are Positional Encodings Nonessential for Deep Autoregressive Transformers? A Petroglyph Revisited. ACL (Findings) Kazuki Irie
2025	J	`jnl`	Why Are Positional Encodings Nonessential for Deep Autoregressive Transformers? Revisiting a Petroglyph. CoRR Kazuki Irie
2024	A*	`conf`	Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers. NeurIPS Lorenzo Tiberi, Francesca Mignacco, Kazuki Irie, Haim Sompolinsky
2024	J	`jnl`	Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers. CoRR Lorenzo Tiberi, Francesca Mignacco, Kazuki Irie, Haim Sompolinsky
2024	A*	`conf`	Exploring the Promise and Limits of Real-Time Recurrent Learning. ICLR Kazuki Irie, Anand Gopalakrishnan, Jürgen Schmidhuber
2024	A*	`conf`	MoEUT: Mixture-of-Experts Universal Transformers. NeurIPS Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber, Christopher Potts, Christopher D. Manning
2024	J	`jnl`	MoEUT: Mixture-of-Experts Universal Transformers. CoRR Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber, Christopher Potts, Christopher D. Manning
2024	J	`jnl`	Neural networks that overcome classic challenges through practice. CoRR Kazuki Irie, Brenden M. Lake
2024	—	`conf`	Self-organising Neural Discrete Representation Learning à la Kohonen. ICANN (1) Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber
2024	A*	`conf`	SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention. NeurIPS Róbert Csordás, Piotr Piekos, Kazuki Irie, Jürgen Schmidhuber
2023	J	`jnl`	Accelerating Neural Self-Improvement via Bootstrapping. CoRR Kazuki Irie, Jürgen Schmidhuber
2023	—	`conf`	Approximating Two-Layer Feedforward Networks for Efficient Transformers. EMNLP (Findings) Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber
2023	J	`jnl`	Approximating Two-Layer Feedforward Networks for Efficient Transformers. CoRR Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber
2023	J	`jnl`	Automating Continual Learning. CoRR Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber
2023	A*	`conf`	Contrastive Training of Complex-Valued Autoencoders for Object Discovery. NeurIPS Aleksandar Stanic, Anand Gopalakrishnan, Kazuki Irie, Jürgen Schmidhuber
2023	J	`jnl`	Contrastive Training of Complex-Valued Autoencoders for Object Discovery. CoRR Aleksandar Stanic, Anand Gopalakrishnan, Kazuki Irie, Jürgen Schmidhuber
2023	J	`jnl`	Exploring the Promise and Limits of Real-Time Recurrent Learning. CoRR Kazuki Irie, Anand Gopalakrishnan, Jürgen Schmidhuber
2023	A*	`conf`	Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rules. ICLR Kazuki Irie, Jürgen Schmidhuber
2023	J	`jnl`	Mindstorms in Natural Language-Based Societies of Mind. CoRR Mingchen Zhuge, Haozhe Liu, Francesco Faccio, Dylan R. Ashley, Róbert Csordás, Anand Gopalakrishnan, Abdullah Hamdi, Hasan Abed Al Kader Hammoud, Vincent Herrmann, Kazuki Irie, Louis Kirsch, Bing Li, Guohao Li, Shuming Liu, Jinjie Mai, Piotr Piekos, Aditya A. Ramesh, Imanol Schlag, Weimin Shi, Aleksandar Stanic, Wenyi Wang, Yuhui Wang, Mengmeng Xu, Deng-Ping Fan, Bernard Ghanem, Jürgen Schmidhuber
2023	A*	`conf`	Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensions. EMNLP Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber
2023	J	`jnl`	Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensions. CoRR Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber
2023	J	`jnl`	SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention. CoRR Róbert Csordás, Piotr Piekos, Kazuki Irie, Jürgen Schmidhuber
2023	J	`jnl`	Topological Neural Discrete Representation Learning à la Kohonen. CoRR Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber
2023	J	`jnl`	Unsupervised Learning of Temporal Abstractions With Slot-Based Transformers. Neural Comput. Anand Gopalakrishnan, Kazuki Irie, Jürgen Schmidhuber, Sjoerd van Steenkiste
2022	A*	`conf`	A Modern Self-Referential Weight Matrix That Learns to Modify Itself. ICML Kazuki Irie, Imanol Schlag, Róbert Csordás, Jürgen Schmidhuber
2022	J	`jnl`	A Modern Self-Referential Weight Matrix That Learns to Modify Itself. CoRR Kazuki Irie, Imanol Schlag, Róbert Csordás, Jürgen Schmidhuber
2022	A*	`conf`	CTL++: Evaluating Generalization on Never-Seen Compositional Patterns of Known Functions, and Compatibility of Neural Representations. EMNLP Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber
2022	J	`jnl`	CTL++: Evaluating Generalization on Never-Seen Compositional Patterns of Known Functions, and Compatibility of Neural Representations. CoRR Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber
2022	J	`jnl`	Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rules. CoRR Kazuki Irie, Jürgen Schmidhuber
2022	J	`jnl`	Learning to Control Rapidly Changing Synaptic Connections: An Alternative Type of Memory in Sequence Processing Artificial Neural Networks. CoRR Kazuki Irie, Jürgen Schmidhuber
2022	A*	`conf`	Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules. NeurIPS Kazuki Irie, Francesco Faccio, Jürgen Schmidhuber
2022	J	`jnl`	Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules. CoRR Kazuki Irie, Francesco Faccio, Jürgen Schmidhuber
2022	A*	`conf`	The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention. ICML Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber
2022	J	`jnl`	The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention. CoRR Kazuki Irie, Róbert Csordás, Jürgen Schmidhuber
2022	A*	`conf`	The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization. ICLR Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber
2022	J	`jnl`	Unsupervised Learning of Temporal Abstractions with Slot-based Transformers. CoRR Anand Gopalakrishnan, Kazuki Irie, Jürgen Schmidhuber, Sjoerd van Steenkiste
2021	A*	`conf`	Going Beyond Linear Transformers with Recurrent Fast Weight Programmers. NeurIPS Kazuki Irie, Imanol Schlag, Róbert Csordás, Jürgen Schmidhuber
2021	J	`jnl`	Going Beyond Linear Transformers with Recurrent Fast Weight Programmers. CoRR Kazuki Irie, Imanol Schlag, Róbert Csordás, Jürgen Schmidhuber
2021	J	`jnl`	Improving Baselines in the Wild. CoRR Kazuki Irie, Imanol Schlag, Róbert Csordás, Jürgen Schmidhuber
2021	J	`jnl`	Linear Transformers Are Secretly Fast Weight Memory Systems. CoRR Imanol Schlag, Kazuki Irie, Jürgen Schmidhuber
2021	A*	`conf`	Linear Transformers Are Secretly Fast Weight Programmers. ICML Imanol Schlag, Kazuki Irie, Jürgen Schmidhuber
2021	—	`conf`	The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers. EMNLP (1) Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber
2021	J	`jnl`	The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers. CoRR Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber
2021	J	`jnl`	The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization. CoRR Róbert Csordás, Kazuki Irie, Jürgen Schmidhuber
2021	J	`jnl`	Training and Generating Neural Networks in Compressed Weight Space. CoRR Kazuki Irie, Jürgen Schmidhuber
2020	—	`—`	Advancing neural language modeling in automatic speech recognition. Kazuki Irie
2020	Misc	`conf`	Domain Robust, Fast, and Compact Neural Language Models. ICASSP Alexander Gerstenberger, Kazuki Irie, Pavel Golik, Eugen Beck, Hermann Ney
2020	Misc	`conf`	How Much Self-Attention Do We Need? Trading Attention for Feed-Forward Layers. ICASSP Kazuki Irie, Alexander Gerstenberger, Ralf Schlüter, Hermann Ney
2020	J	`jnl`	The RWTH ASR System for TED-LIUM Release 2: Improving Hybrid HMM with SpecAugment. CoRR Wei Zhou, Wilfried Michel, Kazuki Irie, Markus Kitza, Ralf Schlüter, Hermann Ney
2020	Misc	`conf`	The Rwth Asr System for Ted-Lium Release 2: Improving Hybrid Hmm With Specaugment. ICASSP Wei Zhou, Wilfried Michel, Kazuki Irie, Markus Kitza, Ralf Schlüter, Hermann Ney
2019	C	`conf`	A Comparison of Transformer and LSTM Encoder Decoder Models for ASR. ASRU Albert Zeyer, Parnia Bahar, Kazuki Irie, Ralf Schlüter, Hermann Ney
2019	A	`conf`	Language Modeling with Deep Transformers. INTERSPEECH Kazuki Irie, Albert Zeyer, Ralf Schlüter, Hermann Ney
2019	J	`jnl`	Language Modeling with Deep Transformers. CoRR Kazuki Irie, Albert Zeyer, Ralf Schlüter, Hermann Ney
2019	J	`jnl`	Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling. CoRR Jonathan Shen, Patrick Nguyen, Yonghui Wu, Zhifeng Chen, Mia Xu Chen, Ye Jia, Anjuli Kannan, Tara N. Sainath, Yuan Cao, Chung-Cheng Chiu, Yanzhang He, Jan Chorowski, Smit Hinsu, Stella Laurenzo, James Qin, Orhan Firat, Wolfgang Macherey, Suyog Gupta, Ankur Bapna, Shuyuan Zhang, Ruoming Pang, Ron J. Weiss, Rohit Prabhavalkar, Qiao Liang, Benoit Jacob, Bowen Liang, HyoukJoong Lee, Ciprian Chelba, Sébastien Jean, Bo Li, Melvin Johnson, Rohan Anil, Rajat Tibrewal, Xiaobing Liu, Akiko Eriguchi, Navdeep Jaitly, Naveen Ari, Colin Cherry, Parisa Haghani, Otavio Good, Youlong Cheng, Raziel Alvarez, Isaac Caswell, Wei-Ning Hsu, Zongheng Yang, Kuan-Chieh Wang, Ekaterina Gonina, Katrin Tomanek, Ben Vanik, Zelin Wu, Llion Jones, Mike Schuster, Yanping Huang, Dehao Chen, Kazuki Irie, George F. Foster, John Richardson, Klaus Macherey, Antoine Bruguier, Heiga Zen, Colin Raffel, Shankar Kumar, Kanishka Rao, David Rybach, Matthew Murray, Vijayaditya Peddinti, Maxim Krikun, Michiel Bacchiani, Thomas B. Jablin, Robert Suderman, Ian Williams, Benjamin Lee, Deepti Bhatia, Justin Carlson, Semih Yavuz, Yu Zhang, Ian McGraw, Max Galkin, Qi Ge, Golan Pundak, Chad Whipkey, Todd Wang, Uri Alon, Dmitry Lepikhin, Ye Tian, Sara Sabour, William Chan, Shubham Toshniwal, Baohua Liao, Michael Nirschl, Pat Rondon
2019	J	`jnl`	Model Unit Exploration for Sequence-to-Sequence Speech Recognition. CoRR Kazuki Irie, Rohit Prabhavalkar, Anjuli Kannan, Antoine Bruguier, David Rybach, Patrick Nguyen
2019	A	`conf`	On the Choice of Modeling Unit for Sequence-to-Sequence Speech Recognition. INTERSPEECH Kazuki Irie, Rohit Prabhavalkar, Anjuli Kannan, Antoine Bruguier, David Rybach, Patrick Nguyen
2019	J	`jnl`	RWTH ASR Systems for LibriSpeech: Hybrid vs Attention - w/o Data Augmentation. CoRR Christoph Lüscher, Eugen Beck, Kazuki Irie, Markus Kitza, Wilfried Michel, Albert Zeyer, Ralf Schlüter, Hermann Ney
2019	A	`conf`	RWTH ASR Systems for LibriSpeech: Hybrid vs Attention. INTERSPEECH Christoph Lüscher, Eugen Beck, Kazuki Irie, Markus Kitza, Wilfried Michel, Albert Zeyer, Ralf Schlüter, Hermann Ney
2019	C	`conf`	Training Language Models for Long-Span Cross-Sentence Evaluation. ASRU Kazuki Irie, Albert Zeyer, Ralf Schlüter, Hermann Ney
2018	A	`conf`	Improved Training of End-to-end Attention Models for Speech Recognition. INTERSPEECH Albert Zeyer, Kazuki Irie, Ralf Schlüter, Hermann Ney
2018	J	`jnl`	Improved training of end-to-end attention models for speech recognition. CoRR Albert Zeyer, Kazuki Irie, Ralf Schlüter, Hermann Ney
2018	A	`conf`	Investigation on Estimation of Sentence Probability by Combining Forward, Backward and Bi-directional LSTM-RNNs. INTERSPEECH Kazuki Irie, Zhihong Lei, Liuhui Deng, Ralf Schlüter, Hermann Ney
2018	Misc	`conf`	Prediction of LSTM-RNN Full Context States as a Subtask for N-Gram Feedforward Language Models. ICASSP Kazuki Irie, Zhihong Lei, Ralf Schlüter, Hermann Ney
2018	Misc	`conf`	RADMM: Recurrent Adaptive Mixture Model with Applications to Domain Robust Language Modeling. ICASSP Kazuki Irie, Shankar Kumar, Michael Nirschl, Hank Liao
2017	Misc	`conf`	Investigations on byte-level convolutional neural networks for language modeling in low resource speech recognition. ICASSP Kazuki Irie, Pavel Golik, Ralf Schlüter, Hermann Ney
2017	—	`conf`	The 2016 RWTH Keyword Search System for Low-Resource Languages. SPECOM Pavel Golik, Zoltán Tüske, Kazuki Irie, Eugen Beck, Ralf Schlüter, Hermann Ney
2016	—	`conf`	Automatic Speech Recognition Based on Neural Networks. SPECOM Ralf Schlüter, Patrick Doetsch, Pavel Golik, Markus Kitza, Tobias Menne, Kazuki Irie, Zoltán Tüske, Albert Zeyer
2016	Misc	`conf`	Investigation on log-linear interpolation of multi-domain neural network language model. ICASSP Zoltán Tüske, Kazuki Irie, Ralf Schlüter, Hermann Ney
2016	A	`conf`	LSTM, GRU, Highway and a Bit of Attention: An Empirical Overview for Language Modeling in Speech Recognition. INTERSPEECH Kazuki Irie, Zoltán Tüske, Tamer Alkhouli, Ralf Schlüter, Hermann Ney
2015	A	`conf`	Bag-of-words input for long history representation in neural network-based language models for speech recognition. INTERSPEECH Kazuki Irie, Ralf Schlüter, Hermann Ney
2015	A	`conf`	On efficient training of word classes and their application to recurrent neural network language models. INTERSPEECH Rami Botros, Kazuki Irie, Martin Sundermeyer, Hermann Ney
2014	Misc	`conf`	The RWTH English lecture recognition system. ICASSP Simon Wiesler, Kazuki Irie, Zoltán Tüske, Ralf Schlüter, Hermann Ney