James Lee-Thorp

15 papers A* 3Journal 9Unranked 3
YearRankTypeTitle / Venue / Authors
2024 — conf
Memory Augmented Language Models through Mixture of Word Experts.
NAACL-HLT
Cícero Nogueira dos Santos, James Lee-Thorp, Isaac Noble, Chung-Ching Chang, David C. Uthus
2023 A* conf
CoLT5: Faster Long-Range Transformers with Conditional Computation.
EMNLP
Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontañón, Siddhartha Brahma, Yury Zemlyanskiy, David C. Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, Sumit Sanghai
2023 J jnl
CoLT5: Faster Long-Range Transformers with Conditional Computation.
CoRR
Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontañón, Siddhartha Brahma, Yury Zemlyanskiy, David C. Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, Sumit Sanghai
2023 A* conf
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints.
EMNLP
Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, Sumit Sanghai
2023 J jnl
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints.
CoRR
Joshua Ainslie, James Lee-Thorp, Michiel de Jong, Yury Zemlyanskiy, Federico Lebrón, Sumit Sanghai
2023 J jnl
Memory Augmented Language Models through Mixture of Word Experts.
CoRR
Cícero Nogueira dos Santos, James Lee-Thorp, Isaac Noble, Chung-Ching Chang, David C. Uthus
2023 J jnl
Scaling Up Models and Data with t5x and seqio.
J. Mach. Learn. Res.
Adam Roberts, Hyung Won Chung, Gaurav Mishra, Anselm Levskaya, James Bradbury, Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, Curtis Hawthorne, Aitor Lewkowycz, Alex Salcianu, Marc van Zee, Jacob Austin, Sebastian Goodman, Livio Baldini Soares, Haitang Hu, Sasha Tsvyashchenko, Aakanksha Chowdhery, Jasmijn Bastings, Jannis Bulian, Xavier Garcia, Jianmo Ni, Andrew Chen, Kathleen Kenealy, Kehang Han, Michelle Casbon, Jonathan H. Clark, Stephan Lee, Dan Garrette, James Lee-Thorp, Colin Raffel, Noam Shazeer, Marvin Ritter, Maarten Bosma, Alexandre Passos, Jeremy Maitin-Shepard, Noah Fiedel, Mark Omernick, Brennan Saeta, Ryan Sepassi, Alexander Spiridonov, Joshua Newlan, Andrea Gesmundo
2023 A* conf
Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints.
ICLR
Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, Neil Houlsby
2022 — conf
FNet: Mixing Tokens with Fourier Transforms.
NAACL-HLT
James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, Santiago Ontañón
2022 J jnl
Scaling Up Models and Data with t5x and seqio.
CoRR
Adam Roberts, Hyung Won Chung, Anselm Levskaya, Gaurav Mishra, James Bradbury, Daniel Andor, Sharan Narang, Brian Lester, Colin Gaffney, Afroz Mohiuddin, Curtis Hawthorne, Aitor Lewkowycz, Alex Salcianu, Marc van Zee, Jacob Austin, Sebastian Goodman, Livio Baldini Soares, Haitang Hu, Sasha Tsvyashchenko, Aakanksha Chowdhery, Jasmijn Bastings, Jannis Bulian, Xavier Garcia, Jianmo Ni, Andrew Chen, Kathleen Kenealy, Jonathan H. Clark, Stephan Lee, Dan Garrette, James Lee-Thorp, Colin Raffel, Noam Shazeer, Marvin Ritter, Maarten Bosma, Alexandre Passos, Jeremy Maitin-Shepard, Noah Fiedel, Mark Omernick, Brennan Saeta, Ryan Sepassi, Alexander Spiridonov, Joshua Newlan, Andrea Gesmundo
2022 — conf
Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT.
EMNLP (Findings)
James Lee-Thorp, Joshua Ainslie
2022 J jnl
Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT.
CoRR
James Lee-Thorp, Joshua Ainslie
2022 J jnl
Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints.
CoRR
Aran Komatsuzaki, Joan Puigcerver, James Lee-Thorp, Carlos Riquelme Ruiz, Basil Mustafa, Joshua Ainslie, Yi Tay, Mostafa Dehghani, Neil Houlsby
2021 J jnl
FNet: Mixing Tokens with Fourier Transforms.
CoRR
James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, Santiago Ontañón
2021 J jnl
ShopTalk: A System for Conversational Faceted Search.
CoRR
Gurmeet Manku, James Lee-Thorp, Bhargav Kanagal, Joshua Ainslie, Jingchen Feng, Zach Pearson, Ebenezer Anjorin, Sudeep Gandhe, Ilya Eckstein, Jim Rosswog, Sumit Sanghai, Michael Pohl, Larry Adams, D. Sivakumar