Quentin Anthony

57 papers A* 3A 4B 3C 4Misc 4Journal 31Unranked 8

Year	Rank	Type	Title / Venue / Authors
2026	J	`jnl`	Covenant-72B: Pre-Training a 72B LLM with Trustless Peers Over-the-Internet. CoRR Joel Lidin, Amir Sarfi, Erfan Miahi, Quentin Anthony, Shivam Chauhan, Evangelos Pappas, Benjamin Thérien, Eugene Belilovsky, Samuel Dare
2025	Misc	`conf`	Characterizing Communication Patterns in Distributed Large Language Model Inference. HOTI Lang Xu, Kaushik Kandadi Suresh, Quentin Anthony, Nawras Alnaasan, Dhabaleswar K. Panda
2025	J	`jnl`	Characterizing Communication Patterns in Distributed Large Language Model Inference. CoRR Lang Xu, Kaushik Kandadi Suresh, Quentin Anthony, Nawras Alnaasan, Dhabaleswar K. Panda
2025	J	`jnl`	Compressed Convolutional Attention: Efficient Attention in a Compressed Latent Space. CoRR Tomas Figliolia, Nicholas Alonso, Rishi Iyer, Quentin Anthony, Beren Millidge
2025	J	`jnl`	Deep Ignorance: Filtering Pretraining Data Builds Tamper-Resistant Safeguards into Open-Weight LLMs. CoRR Kyle O'Brien, Stephen Casper, Quentin Anthony, Tomek Korbak, Robert Kirk, Xander Davies, Ishan Mishra, Geoffrey Irving, Yarin Gal, Stella Biderman
2025	—	`conf`	HARVEST Inference: Characterizing Digital Agriculture Workloads across Compute Continuum. ICPP Workshops Tian Chen, Quentin Anthony, Dhabaleswar K. Panda
2025	J	`jnl`	PyLO: Towards Accessible Learned Optimizers in PyTorch. CoRR Paul Janson, Benjamin Thérien, Quentin Anthony, Xiaolong Huang, Abhinav Moudgil, Eugene Belilovsky
2025	J	`jnl`	Robin: a Suite of Multi-Scale Vision-Language Models and the CHIRP Evaluation Benchmark. CoRR Alexis Roger, Prateek Humane, Daniel Z. Kaplan, Kshitij Gupta, Qi Sun, George Adamopoulos, Jonathan Siu Chi Lim, Quentin Anthony, Edwin Fennell, Irina Rish
2025	J	`jnl`	Scaling Large Language Model Training on Frontier with Low-Bandwidth Partitioning. CoRR Lang Xu, Quentin Anthony, Jacob Hatef, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2025	J	`jnl`	Training Foundation Models on a Full-Stack AMD Platform: Compute, Networking, and System Design. CoRR Quentin Anthony, Yury Tokpanov, Skyler Szot, Srivatsan Rajagopal, Praneeth Medepalli, Anna Golubeva, Vasudev Shyam, Robert Washbourne, Rishi Iyer, Ansh Chaurasia, Tomas Figliolia, Xiao Yang, Abhinav Sarje, Drew Thorstensen, Amartey Pearson, Zack Grossbart, Jason van Patten, Emad Barsoum, Zhenyu Gu, Yao Fu, Beren Millidge
2025	J	`jnl`	Understanding and Characterizing Communication Characteristics for Distributed Transformer Models. IEEE Micro Quentin Anthony, Benjamin Michalowicz, Jacob Hatef, Lang Xu, Mustafa Abduljabbar, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2024	B	`conf`	Accelerating Large Language Model Training with Hybrid GPU-based Compression. CCGrid Lang Xu, Quentin Anthony, Qinghua Zhou, Nawras Alnaasan, Radha Gulhane, Aamir Shafi, Hari Subramoni, Dhabaleswar K. D. K. Panda
2024	J	`jnl`	Accelerating Large Language Model Training with Hybrid GPU-based Compression. CoRR Lang Xu, Quentin Anthony, Qinghua Zhou, Nawras Alnaasan, Radha Gulhane, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2024	J	`jnl`	BlackMamba: Mixture of Experts for State-Space Models. CoRR Quentin Anthony, Yury Tokpanov, Paolo Glorioso, Beren Millidge
2024	A	`conf`	Comparative Study of Large Language Model Architectures on Frontier. IPDPS Junqi Yin, Avishek Bose, Guojing Cong, Isaac Lyngaas, Quentin Anthony
2024	J	`jnl`	Comparative Study of Large Language Model Architectures on Frontier. CoRR Junqi Yin, Avishek Bose, Guojing Cong, Isaac Lyngaas, Quentin Anthony
2024	Misc	`conf`	Demystifying the Communication Characteristics for Distributed Transformer Models. HOTI Quentin Anthony, Benjamin Michalowicz, Jacob Hatef, Lang Xu, Mustafa Abdul Jabbar, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2024	J	`jnl`	Demystifying the Communication Characteristics for Distributed Transformer Models. CoRR Quentin Anthony, Benjamin Michalowicz, Jacob Hatef, Lang Xu, Mustafa Abduljabbar, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2024	J	`jnl`	Eagle and Finch: RWKV with Matrix-Valued States and Dynamic Recurrence. CoRR Bo Peng, Daniel Goldstein, Quentin Anthony, Alon Albalak, Eric Alcaide, Stella Biderman, Eugene Cheah, Xingjian Du, Teddy Ferdinan, Haowen Hou, Przemyslaw Kazienko, Kranthi Kiran GV, Jan Kocon, Bartlomiej Koptyra, Satyapriya Krishna, Ronald McClelland Jr., Niklas Muennighoff, Fares Obeid, Atsushi Saito, Guangyu Song, Haoqin Tu, Stanislaw Wozniak, Ruichong Zhang, Bingchen Zhao, Qihang Zhao, Peng Zhou, Jian Zhu, Rui-Jie Zhu
2024	A	`conf`	Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference. IPDPS Jinghan Yao, Quentin Anthony, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2024	J	`jnl`	Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference. CoRR Jinghan Yao, Quentin Anthony, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2024	—	`conf`	Infer-HiRes: Accelerating Inference for High-Resolution Images with Quantization and Distributed Deep Learning. PEARC Radha Gulhane, Quentin Anthony, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2024	A*	`conf`	RedPajama: an Open Dataset for Training Large Language Models. NeurIPS Maurice Weber, Daniel Y. Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, Ben Athiwaratkun, Rahul Chalamala, Kezhen Chen, Max Ryabinin, Tri Dao, Percy Liang, Christopher Ré, Irina Rish, Ce Zhang
2024	J	`jnl`	RedPajama: an Open Dataset for Training Large Language Models. CoRR Maurice Weber, Daniel Y. Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, Ben Athiwaratkun, Rahul Chalamala, Kezhen Chen, Max Ryabinin, Tri Dao, Percy Liang, Christopher Ré, Irina Rish, Ce Zhang
2024	Misc	`conf`	Scaling Large Language Model Training on Frontier with Low-Bandwidth Partitioning. HiPC Lang Xu, Quentin Anthony, Jacob Hatef, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2024	J	`jnl`	Simple and Scalable Strategies to Continually Pre-train Large Language Models. CoRR Adam Ibrahim, Benjamin Thérien, Kshitij Gupta, Mats L. Richter, Quentin Anthony, Timothée Lesort, Eugene Belilovsky, Irina Rish
2024	B	`conf`	The Case for Co-Designing Model Architectures with Hardware. ICPP Quentin Anthony, Jacob Hatef, Deepak Narayanan, Stella Biderman, Stas Bekman, Junqi Yin, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2024	J	`jnl`	The Case for Co-Designing Model Architectures with Hardware. CoRR Quentin Anthony, Jacob Hatef, Deepak Narayanan, Stella Biderman, Stas Bekman, Junqi Yin, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2024	J	`jnl`	The Zamba2 Suite: Technical Report. CoRR Paolo Glorioso, Quentin Anthony, Yury Tokpanov, Anna Golubeva, Vasudev Shyam, James Whittington, Jonathan Pilault, Beren Millidge
2024	J	`jnl`	Tree Attention: Topology-aware Decoding for Long-Context Attention on GPU clusters. CoRR Vasudev Shyam, Jonathan Pilault, Emily Shepperd, Quentin Anthony, Beren Millidge
2024	J	`jnl`	Zamba: A Compact 7B SSM Hybrid Model. CoRR Paolo Glorioso, Quentin Anthony, Yury Tokpanov, James Whittington, Jonathan Pilault, Adam Ibrahim, Beren Millidge
2024	J	`jnl`	Zyda-2: a 5 Trillion Token High-Quality Dataset. CoRR Yury Tokpanov, Paolo Glorioso, Quentin Anthony, Beren Millidge
2024	J	`jnl`	Zyda: A 1.3T Dataset for Open Language Modeling. CoRR Yury Tokpanov, Beren Millidge, Paolo Glorioso, Jonathan Pilault, Adam Ibrahim, James Whittington, Quentin Anthony
2023	A	`conf`	Accelerating Distributed Deep Learning Training with Compression Assisted Allgather and Reduce-Scatter Communication. IPDPS Qinghua Zhou, Quentin Anthony, Lang Xu, Aamir Shafi, Mustafa Abduljabbar, Hari Subramoni, Dhabaleswar K. Panda
2023	J	`jnl`	Continual Pre-Training of Large Language Models: How to (re)warm your model? CoRR Kshitij Gupta, Benjamin Thérien, Adam Ibrahim, Mats L. Richter, Quentin Anthony, Eugene Belilovsky, Irina Rish, Timothée Lesort
2023	A*	`conf`	Emergent and Predictable Memorization in Large Language Models. NeurIPS Stella Biderman, USVSN Sai Prashanth, Lintang Sutawika, Hailey Schoelkopf, Quentin Anthony, Shivanshu Purohit, Edward Raff
2023	J	`jnl`	Emergent and Predictable Memorization in Large Language Models. CoRR Stella Biderman, USVSN Sai Prashanth, Lintang Sutawika, Hailey Schoelkopf, Quentin Anthony, Shivanshu Purohit, Edward Raff
2023	A	`conf`	MCR-DL: Mix-and-Match Communication Runtime for Deep Learning. IPDPS Quentin Anthony, Ammar Ahmad Awan, Jeff Rasley, Yuxiong He, Aamir Shafi, Mustafa Abduljabbar, Hari Subramoni, Dhabaleswar K. Panda
2023	J	`jnl`	MCR-DL: Mix-and-Match Communication Runtime for Deep Learning. CoRR Quentin Anthony, Ammar Ahmad Awan, Jeff Rasley, Yuxiong He, Aamir Shafi, Mustafa Abduljabbar, Hari Subramoni, Dhabaleswar K. Panda
2023	J	`jnl`	Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling. CoRR Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya Skowron, Lintang Sutawika, Oskar van der Wal
2023	—	`conf`	RWKV: Reinventing RNNs for the Transformer Era. EMNLP (Findings) Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Stella Biderman, Huanqi Cao, Xin Cheng, Michael Chung, Leon Derczynski, Xingjian Du, Matteo Grella, Kranthi Kiran GV, Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartlomiej Koptyra, Hayden Lau, Jiaju Lin, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Guangyu Song, Xiangru Tang, Johan S. Wind, Stanislaw Wozniak, Zhenyuan Zhang, Qinghua Zhou, Jian Zhu, Rui-Jie Zhu
2023	J	`jnl`	RWKV: Reinventing RNNs for the Transformer Era. CoRR Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, Kranthi Kiran G. V., Xuzheng He, Haowen Hou, Przemyslaw Kazienko, Jan Kocon, Jiaming Kong, Bartlomiej Koptyra, Hayden Lau, Krishna Sri Ipsit Mantri, Ferdinand Mom, Atsushi Saito, Xiangru Tang, Bolun Wang, Johan S. Wind, Stanislaw Wozniak, Ruichong Zhang, Zhenyuan Zhang, Qihang Zhao, Peng Zhou, Jian Zhu, Rui-Jie Zhu
2023	B	`conf`	ScaMP: Scalable Meta-Parallelism for Deep Learning Search. CCGrid Quentin Anthony, Lang Xu, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2023	—	`conf`	ScaMP: Scalable Meta-Parallelism for Deep Learning Search. CCGridW Quentin Anthony, Lang Xu, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2023	A*	`conf`	trlX: A Framework for Large Scale Reinforcement Learning from Human Feedback. EMNLP Alexander Havrilla, Maksym Zhuravinskyi, Duy Phung, Aman Tiwari, Jonathan Tow, Stella Biderman, Quentin Anthony, Louis Castricato
2022	Misc	`conf`	Accelerating Broadcast Communication with GPU Compression for Deep Learning Workloads. HIPC Qinghua Zhou, Quentin Anthony, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2022	C	`conf`	Accelerating MPI All-to-All Communication with Online Compression on Modern GPU Clusters. ISC Qinghua Zhou, Pouya Kousha, Quentin Anthony, Kawthar Shafie Khorassani, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2022	J	`jnl`	GPT-NeoX-20B: An Open-Source Autoregressive Language Model. CoRR Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, Samuel Weinbach
2022	C	`conf`	Hy-Fi: Hybrid Five-Dimensional Parallel DNN Training on High-Performance GPU Clusters. ISC Arpan Jain, Aamir Shafi, Quentin Anthony, Pouya Kousha, Hari Subramoni, Dhabaleswar K. Panda
2021	J	`jnl`	Cross-layer Visualization and Profiling of Network and I/O Communication for HPC Clusters. CoRR Pouya Kousha, Quentin Anthony, Hari Subramoni, Dhabaleswar K. Panda
2021	—	`conf`	Evaluating Multi-Level Checkpointing for Distributed Deep Neural Network Training. SC (Workshops) Quentin Anthony, Donglai Dai
2021	—	`conf`	Scaling Single-Image Super-Resolution Training on Modern HPC Clusters: Early Experiences. IPDPS Workshops Quentin Anthony, Lang Xu, Hari Subramoni, Dhabaleswar K. D. K. Panda
2020	—	`conf`	Accelerating GPU-based Machine Learning in Python using MPI Library: A Case Study with MVAPICH2-GDR. MLHPC/AI4S@SC Seyedeh Mahdieh Ghazimirsaeed, Quentin Anthony, Aamir Shafi, Hari Subramoni, Dhabaleswar K. D. K. Panda
2020	—	`conf`	Efficient Training of Semantic Image Segmentation on Summit using Horovod and MVAPICH2-GDR. IPDPS Workshops Quentin Anthony, Ammar Ahmad Awan, Arpan Jain, Hari Subramoni, Dhabaleswar K. D. K. Panda
2020	C	`conf`	HyPar-Flow: Exploiting MPI and Keras for Scalable Hybrid-Parallel DNN Training with TensorFlow. ISC Ammar Ahmad Awan, Arpan Jain, Quentin Anthony, Hari Subramoni, Dhabaleswar K. Panda
2019	J	`jnl`	HyPar-Flow: Exploiting MPI and Keras for Scalable Hybrid-Parallel DNN Training using TensorFlow. CoRR Ammar Ahmad Awan, Arpan Jain, Quentin Anthony, Hari Subramoni, Dhabaleswar K. Panda
2019	C	`conf`	Performance Characterization of DNN Training using TensorFlow and PyTorch on Modern Clusters. CLUSTER Arpan Jain, Ammar Ahmad Awan, Quentin Anthony, Hari Subramoni, Dhabaleswar K. Panda