Mohammad Shoeybi

108 papers A* 21A 2Journal 73Unranked 12

Year	Rank	Type	Title / Venue / Authors
2026	J	`jnl`	LatentMoE: Toward Optimal Accuracy per FLOP and Parameter in Mixture of Experts. CoRR Venmugil Elango, Nidhi Bhatia, Roger Waleffe, Rasoul Shafipour, Tomer Asida, Abhinav Khattar, Nave Assaf, Maximilian Golub, Joey Guman, Tiyasa Mitra, Ritchie Zhao, Ritika Borkar, Ran Zilberstein, Mostofa Patwary, Mohammad Shoeybi, Bita Rouhani
2026	J	`jnl`	MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos. CoRR Arushi Goel, Sreyan Ghosh, Vatsal Agarwal, Nishit Anand, Kaousheik Jayakumar, Lasha Koroshinadze, Yao Xu, Katie Lyons, James Case, Karan Sapra, Kevin J. Shih, Siddharth Gururani, Abhinav Shrivastava, Ramani Duraiswami, Dinesh Manocha, Andrew Tao, Bryan Catanzaro, Mohammad Shoeybi, Wei Ping
2026	J	`jnl`	Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation. CoRR Zhuolin Yang, Zihan Liu, Yang Chen, Wenliang Dai, Boxin Wang, Sheng-Chieh Lin, Chankyu Lee, Yangyi Chen, Dongfu Jiang, Jiafan He, Renjie Pi, Grace Lam, Nayeon Lee, Alexander Bukharin, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping
2026	—	`conf`	Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning. EACL (Volume 1: Long Papers) Syeda Nahida Akter, Shrimai Prabhumoye, Matvei Novikov, Seungju Han, Ying Lin, Evelina Bakhturina, Eric Nyberg, Yejin Choi, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2026	J	`jnl`	On Data Engineering for Scaling LLM Terminal Capabilities. CoRR Renjie Pi, Grace Lam, Mohammad Shoeybi, Pooya Jannaty, Bryan Catanzaro, Wei Ping
2026	J	`jnl`	Scalable Training of Mixture-of-Experts Models with Megatron Core. CoRR Zijie Yan, Hongxiao Bai, Xin Yao, Dennis Liu, Tong Liu, Hongbin Liu, Pingtian Li, Evan Wu, Shiqing Fan, Li Tao, Robin Zhang, Yuzhong Wang, Shifang Xu, Jack Chang, Xuwen Chen, Kunlun Li, Yan Bai, Gao Deng, Nan Zheng, Vijay Anand Korthikanti, Abhinav Khattar, Ethan He, Soham Govande, Sangkug Lym, Zhongbo Zhu, Qi Zhang, Haochen Yuan, Xiaowei Ren, Deyu Fu, Tailai Ma, Shunkang Zhang, Jiang Shao, Ray Wang, Vasudevan Rengasamy, Rachit Garg, Santosh Bhavani, Xipeng Li, Chandler Zhou, David Wu, Yingcan Wei, Ashwath Aithal, Michael Andersch, Mohammad Shoeybi, Jiajie Yao, June Yang
2025	—	`conf`	AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling. ACL (Findings) Zihan Liu, Yang Chen, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping
2025	J	`jnl`	AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergy. CoRR Zihan Liu, Zhuolin Yang, Yang Chen, Chankyu Lee, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping
2025	J	`jnl`	AceReason-Nemotron: Advancing Math and Code Reasoning through Reinforcement Learning. CoRR Yang Chen, Zhuolin Yang, Zihan Liu, Chankyu Lee, Peng Xu, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping
2025	A*	`conf`	ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities. ICLR Peng Xu, Wei Ping, Xianchao Wu, Chejian Xu, Zihan Liu, Mohammad Shoeybi, Bryan Catanzaro
2025	J	`jnl`	Efficient Hybrid Language Model Compression through Group-Aware SSM Pruning. CoRR Ali Taghibakhshi, Sharath Turuvekere Sreenivas, Saurav Muralidharan, Marcin Chochowski, Yashaswi Karnati, Raviraj Joshi, Ameya Sunil Mahabaleshwarkar, Zijia Chen, Yoshi Suhara, Oluwatobi Olabiyi, Daniel Korzekwa, Mostofa Patwary, Mohammad Shoeybi, Jan Kautz, Bryan Catanzaro, Ashwath Aithal, Nima Tajbakhsh, Pavlo Molchanov
2025	J	`jnl`	From 128K to 4M: Efficient Training of Ultra-Long Context Large Language Models. CoRR Chejian Xu, Wei Ping, Peng Xu, Zihan Liu, Boxin Wang, Mohammad Shoeybi, Bo Li, Bryan Catanzaro
2025	J	`jnl`	Front-Loading Reasoning: The Synergy between Pretraining and Post-Training Data. CoRR Syeda Nahida Akter, Shrimai Prabhumoye, Eric Nyberg, Mostofa Patwary, Mohammad Shoeybi, Yejin Choi, Bryan Catanzaro
2025	J	`jnl`	Fusing LLM Capabilities with Routing Data. CoRR Tao Feng, Haozhen Zhang, Zijie Lei, Pengrui Han, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jiaxuan You
2025	A*	`conf`	MIND: Math Informed syNthetic Dialogues for Pretraining LLMs. ICLR Syeda Nahida Akter, Shrimai Prabhumoye, John Kamalu, Sanjeev Satheesh, Eric Nyberg, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2025	A*	`conf`	Mm-Embed: Universal Multimodal Retrieval with Multimodal LLMS. ICLR Sheng-Chieh Lin, Chankyu Lee, Mohammad Shoeybi, Jimmy Lin, Bryan Catanzaro, Wei Ping
2025	J	`jnl`	MoE Parallel Folding: Heterogeneous Parallelism Mappings for Efficient Large-Scale MoE Model Training with Megatron Core. CoRR Dennis Liu, Zijie Yan, Xin Yao, Tong Liu, Vijay Korthikanti, Evan Wu, Shiqing Fan, Gao Deng, Hongxiao Bai, Jianbin Chang, Ashwath Aithal, Michael Andersch, Mohammad Shoeybi, Jiajie Yao, Chandler Zhou, David Wu, Xipeng Li, June Yang
2025	J	`jnl`	Music Flamingo: Scaling Music Understanding in Audio Language Models. CoRR Sreyan Ghosh, Arushi Goel, Lasha Koroshinadze, Sang-gil Lee, Zhifeng Kong, João Felipe Santos, Ramani Duraiswami, Dinesh Manocha, Weiping Ding, Mohammad Shoeybi, Bryan Catanzaro
2025	A*	`conf`	NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models. ICLR Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping
2025	J	`jnl`	Nemotron Elastic: Towards Efficient Many-in-One Reasoning LLMs. CoRR Ali Taghibakhshi, Sharath Turuvekere Sreenivas, Saurav Muralidharan, Ruisi Cai, Marcin Chochowski, Ameya Sunil Mahabaleshwarkar, Yoshi Suhara, Oluwatobi Olabiyi, Daniel Korzekwa, Mostofa Patwary, Mohammad Shoeybi, Jan Kautz, Bryan Catanzaro, Ashwath Aithal, Nima Tajbakhsh, Pavlo Molchanov
2025	J	`jnl`	Nemotron-CC-Math: A 133 Billion-Token-Scale High Quality Math Pretraining Dataset. CoRR Rabeeh Karimi Mahabadi, Sanjeev Satheesh, Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2025	—	`conf`	Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Dataset. ACL (1) Dan Su, Kezhi Kong, Ying Lin, Joseph Jennings, Brandon Norick, Markus Kliegl, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2025	J	`jnl`	Nemotron-Cascade: Scaling Cascaded Reinforcement Learning for General-Purpose Reasoning Models. CoRR Boxin Wang, Chankyu Lee, Nayeon Lee, Sheng-Chieh Lin, Wenliang Dai, Yang Chen, Yangyi Chen, Zhuolin Yang, Zihan Liu, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping
2025	J	`jnl`	Nemotron-CrossThink: Scaling Self-Learning beyond Math Reasoning. CoRR Syeda Nahida Akter, Shrimai Prabhumoye, Matvei Novikov, Seungju Han, Ying Lin, Evelina Bakhturina, Eric Nyberg, Yejin Choi, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2025	J	`jnl`	Prismatic Synthesis: Gradient-based Data Diversification Boosts Generalization in LLM Reasoning. CoRR Jaehun Jung, Seungju Han, Ximing Lu, Skyler Hallinan, David Acuna, Shrimai Prabhumoye, Mostafa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi
2025	J	`jnl`	RLP: Reinforcement as a Pretraining Objective. CoRR Ali Hatamizadeh, Syeda Nahida Akter, Shrimai Prabhumoye, Jan Kautz, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi
2025	J	`jnl`	Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoning. CoRR Ximing Lu, Seungju Han, David Acuna, Hyunwoo Kim, Jaehun Jung, Shrimai Prabhumoye, Niklas Muennighoff, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Yejin Choi
2025	J	`jnl`	UALM: Unified Audio Language Model for Understanding, Generation and Reasoning. CoRR Jinchuan Tian, Sang-gil Lee, Zhifeng Kong, Sreyan Ghosh, Arushi Goel, Chao-Han Huck Yang, Wenliang Dai, Zihan Liu, Hanrong Ye, Shinji Watanabe, Mohammad Shoeybi, Bryan Catanzaro, Rafael Valle, Wei Ping
2024	J	`jnl`	AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling. CoRR Zihan Liu, Yang Chen, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping
2024	J	`jnl`	An Empirical Study of Mamba-based Language Models. CoRR Roger Waleffe, Wonmin Byeon, Duncan Riach, Brandon Norick, Vijay Korthikanti, Tri Dao, Albert Gu, Ali Hatamizadeh, Sudhakar Singh, Deepak Narayanan, Garvit Kulshreshtha, Vartika Singh, Jared Casper, Jan Kautz, Mohammad Shoeybi, Bryan Catanzaro
2024	J	`jnl`	ChatQA 2: Bridging the Gap to Proprietary LLMs in Long Context and RAG Capabilities. CoRR Peng Xu, Wei Ping, Xianchao Wu, Zihan Liu, Mohammad Shoeybi, Bryan Catanzaro
2024	J	`jnl`	ChatQA: Building GPT-4 Level Conversational QA Models. CoRR Zihan Liu, Wei Ping, Rajarshi Roy, Peng Xu, Chankyu Lee, Mohammad Shoeybi, Bryan Catanzaro
2024	A*	`conf`	ChatQA: Surpassing GPT-4 on Conversational QA and RAG. NeurIPS Zihan Liu, Wei Ping, Rajarshi Roy, Peng Xu, Chankyu Lee, Mohammad Shoeybi, Bryan Catanzaro
2024	A*	`conf`	Compact Language Models via Pruning and Knowledge Distillation. NeurIPS Saurav Muralidharan, Sharath Turuvekere Sreenivas, Raviraj Joshi, Marcin Chochowski, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jan Kautz, Pavlo Molchanov
2024	J	`jnl`	Compact Language Models via Pruning and Knowledge Distillation. CoRR Saurav Muralidharan, Sharath Turuvekere Sreenivas, Raviraj Joshi, Marcin Chochowski, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jan Kautz, Pavlo Molchanov
2024	A*	`conf`	Data, Data Everywhere: A Guide for Pretraining Dataset Construction. EMNLP Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Bo Liu, Aastha Jhunjhunwala, Zhilin Wang, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2024	J	`jnl`	Data, Data Everywhere: A Guide for Pretraining Dataset Construction. CoRR Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Bo Li, Aastha Jhunjhunwala, Zhilin Wang, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2024	A*	`conf`	InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining. ICML Boxin Wang, Wei Ping, Lawrence McAfee, Peng Xu, Bo Li, Mohammad Shoeybi, Bryan Catanzaro
2024	J	`jnl`	LLM Pruning and Distillation in Practice: The Minitron Approach. CoRR Sharath Turuvekere Sreenivas, Saurav Muralidharan, Raviraj Joshi, Marcin Chochowski, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jan Kautz, Pavlo Molchanov
2024	A*	`conf`	LLM-Evolve: Evaluation for LLM's Evolving Capability on Benchmarks. EMNLP Jiaxuan You, Mingjie Liu, Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2024	J	`jnl`	MIND: Math Informed syNthetic Dialogues for Pretraining LLMs. CoRR Syeda Nahida Akter, Shrimai Prabhumoye, John Kamalu, Sanjeev Satheesh, Eric Nyberg, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2024	J	`jnl`	MM-Embed: Universal Multimodal Retrieval with Multimodal LLMs. CoRR Sheng-Chieh Lin, Chankyu Lee, Mohammad Shoeybi, Jimmy Lin, Bryan Catanzaro, Wei Ping
2024	J	`jnl`	Maximize Your Data's Potential: Enhancing LLM Accuracy with Two-Phase Pretraining. CoRR Steven Feng, Shrimai Prabhumoye, Kezhi Kong, Dan Su, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2024	J	`jnl`	NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models. CoRR Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping
2024	J	`jnl`	NVLM: Open Frontier-Class Multimodal LLMs. CoRR Wenliang Dai, Nayeon Lee, Boxin Wang, Zhuoling Yang, Zihan Liu, Jon Barker, Tuomas Rintamaki, Mohammad Shoeybi, Bryan Catanzaro, Wei Ping
2024	J	`jnl`	Nemotron-4 15B Technical Report. CoRR Jupinder Parmar, Shrimai Prabhumoye, Joseph Jennings, Mostofa Patwary, Sandeep Subramanian, Dan Su, Chen Zhu, Deepak Narayanan, Aastha Jhunjhunwala, Ayush Dattagupta, Vibhu Jawa, Jiwei Liu, Ameya Mahabaleshwarkar, Osvald Nitski, Annika Brundyn, James Maki, Miguel Martinez, Jiaxuan You, John Kamalu, Patrick LeGresley, Denys Fridman, Jared Casper, Ashwath Aithal, Oleksii Kuchaiev, Mohammad Shoeybi, Jonathan M. Cohen, Bryan Catanzaro
2024	J	`jnl`	Nemotron-CC: Transforming Common Crawl into a Refined Long-Horizon Pretraining Dataset. CoRR Dan Su, Kezhi Kong, Ying Lin, Joseph Jennings, Brandon Norick, Markus Kliegl, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2024	A*	`conf`	ODIN: Disentangled Reward Mitigates Hacking in RLHF. ICML Lichang Chen, Chen Zhu, Jiuhai Chen, Davit Soselia, Tianyi Zhou, Tom Goldstein, Heng Huang, Mohammad Shoeybi, Bryan Catanzaro
2024	J	`jnl`	ODIN: Disentangled Reward Mitigates Hacking in RLHF. CoRR Lichang Chen, Chen Zhu, Davit Soselia, Jiuhai Chen, Tianyi Zhou, Tom Goldstein, Heng Huang, Mohammad Shoeybi, Bryan Catanzaro
2024	A*	`conf`	RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs. NeurIPS Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, Bryan Catanzaro
2024	J	`jnl`	RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs. CoRR Yue Yu, Wei Ping, Zihan Liu, Boxin Wang, Jiaxuan You, Chao Zhang, Mohammad Shoeybi, Bryan Catanzaro
2024	A*	`conf`	Retrieval meets Long Context Large Language Models. ICLR Peng Xu, Wei Ping, Xianchao Wu, Lawrence McAfee, Chen Zhu, Zihan Liu, Sandeep Subramanian, Evelina Bakhturina, Mohammad Shoeybi, Bryan Catanzaro
2024	J	`jnl`	Reuse, Don't Retrain: A Recipe for Continued Pretraining of Language Models. CoRR Jupinder Parmar, Sanjeev Satheesh, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2024	J	`jnl`	Upcycling Large Language Models into Mixture of Experts. CoRR Ethan He, Abhinav Khattar, Ryan Prenger, Vijay Korthikanti, Zijie Yan, Tong Liu, Shiqing Fan, Ashwath Aithal, Mohammad Shoeybi, Bryan Catanzaro
2024	A*	`conf`	VILA: On Pre-training for Visual Language Models. CVPR Ji Lin, Hongxu Yin, Wei Ping, Pavlo Molchanov, Mohammad Shoeybi, Song Han
2023	J	`jnl`	Adding Instructions during Pretraining: Effective Way of Controlling Toxicity in Language Models. CoRR Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2023	A	`conf`	Adding Instructions during Pretraining: Effective way of Controlling Toxicity in Language Models. EACL Shrimai Prabhumoye, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2023	—	`conf`	Context Generation Improves Open Domain Question Answering. EACL (Findings) Dan Su, Mostofa Patwary, Shrimai Prabhumoye, Peng Xu, Ryan Prenger, Mohammad Shoeybi, Pascale Fung, Anima Anandkumar, Bryan Catanzaro
2023	J	`jnl`	InstructRetro: Instruction Tuning post Retrieval-Augmented Pretraining. CoRR Boxin Wang, Wei Ping, Lawrence McAfee, Peng Xu, Bo Li, Mohammad Shoeybi, Bryan Catanzaro
2023	J	`jnl`	RAVEN: In-Context Learning with Retrieval Augmented Encoder-Decoder Language Models. CoRR Jie Huang, Wei Ping, Peng Xu, Mohammad Shoeybi, Kevin Chen-Chuan Chang, Bryan Catanzaro
2023	—	`conf`	Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning. EMNLP (Findings) Zhuolin Yang, Wei Ping, Zihan Liu, Vijay Korthikanti, Weili Nie, De-An Huang, Linxi Fan, Zhiding Yu, Shiyi Lan, Bo Li, Mohammad Shoeybi, Ming-Yu Liu, Yuke Zhu, Bryan Catanzaro, Chaowei Xiao, Anima Anandkumar
2023	J	`jnl`	Re-ViLM: Retrieval-Augmented Visual Language Model for Zero and Few-Shot Image Captioning. CoRR Zhuolin Yang, Wei Ping, Zihan Liu, Vijay Korthikanti, Weili Nie, De-An Huang, Linxi Fan, Zhiding Yu, Shiyi Lan, Bo Li, Ming-Yu Liu, Yuke Zhu, Mohammad Shoeybi, Bryan Catanzaro, Chaowei Xiao, Anima Anandkumar
2023	—	`conf`	Reducing Activation Recomputation in Large Transformer Models. MLSys Vijay Anand Korthikanti, Jared Casper, Sangkug Lym, Lawrence McAfee, Michael Andersch, Mohammad Shoeybi, Bryan Catanzaro
2023	J	`jnl`	Retrieval meets Long Context Large Language Models. CoRR Peng Xu, Wei Ping, Xianchao Wu, Lawrence McAfee, Chen Zhu, Zihan Liu, Sandeep Subramanian, Evelina Bakhturina, Mohammad Shoeybi, Bryan Catanzaro
2023	A*	`conf`	Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study. EMNLP Boxin Wang, Wei Ping, Peng Xu, Lawrence McAfee, Zihan Liu, Mohammad Shoeybi, Yi Dong, Oleksii Kuchaiev, Bo Li, Chaowei Xiao, Anima Anandkumar, Bryan Catanzaro
2023	J	`jnl`	Shall We Pretrain Autoregressive Language Models with Retrieval? A Comprehensive Study. CoRR Boxin Wang, Wei Ping, Peng Xu, Lawrence McAfee, Zihan Liu, Mohammad Shoeybi, Yi Dong, Oleksii Kuchaiev, Bo Li, Chaowei Xiao, Anima Anandkumar, Bryan Catanzaro
2023	J	`jnl`	VILA: On Pre-training for Visual Language Models. CoRR Ji Lin, Hongxu Yin, Wei Ping, Yao Lu, Pavlo Molchanov, Andrew Tao, Huizi Mao, Jan Kautz, Mohammad Shoeybi, Song Han
2022	J	`jnl`	Context Generation Improves Open Domain Question Answering. CoRR Dan Su, Mostofa Patwary, Shrimai Prabhumoye, Peng Xu, Ryan Prenger, Mohammad Shoeybi, Pascale Fung, Anima Anandkumar, Bryan Catanzaro
2022	A*	`conf`	Evaluating Parameter Efficient Learning for Generation. EMNLP Peng Xu, Mostofa Patwary, Shrimai Prabhumoye, Virginia Adams, Ryan Prenger, Wei Ping, Nayeon Lee, Mohammad Shoeybi, Bryan Catanzaro
2022	J	`jnl`	Evaluating Parameter Efficient Learning for Generation. CoRR Peng Xu, Mostofa Patwary, Shrimai Prabhumoye, Virginia Adams, Ryan J. Prenger, Wei Ping, Nayeon Lee, Mohammad Shoeybi, Bryan Catanzaro
2022	A*	`conf`	Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models. NeurIPS Boxin Wang, Wei Ping, Chaowei Xiao, Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Bo Li, Anima Anandkumar, Bryan Catanzaro
2022	J	`jnl`	Exploring the Limits of Domain-Adaptive Training for Detoxifying Large-Scale Language Models. CoRR Boxin Wang, Wei Ping, Chaowei Xiao, Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Bo Li, Anima Anandkumar, Bryan Catanzaro
2022	J	`jnl`	FP8 Formats for Deep Learning. CoRR Paulius Micikevicius, Dusan Stosic, Neil Burgess, Marius Cornea, Pradeep Dubey, Richard Grisenthwaite, Sangwon Ha, Alexander Heinecke, Patrick Judd, John Kamalu, Naveen Mellempudi, Stuart F. Oberman, Mohammad Shoeybi, Michael Y. Siu, Hao Wu
2022	A*	`conf`	Factuality Enhanced Language Models for Open-Ended Text Generation. NeurIPS Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale Fung, Mohammad Shoeybi, Bryan Catanzaro
2022	J	`jnl`	Factuality Enhanced Language Models for Open-Ended Text Generation. CoRR Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2022	—	`conf`	Multi-Stage Prompting for Knowledgeable Dialogue Generation. ACL (Findings) Zihan Liu, Mostofa Patwary, Ryan Prenger, Shrimai Prabhumoye, Wei Ping, Mohammad Shoeybi, Bryan Catanzaro
2022	J	`jnl`	Multi-Stage Prompting for Knowledgeable Dialogue Generation. CoRR Zihan Liu, Mostofa Patwary, Ryan Prenger, Shrimai Prabhumoye, Wei Ping, Mohammad Shoeybi, Bryan Catanzaro
2022	—	`conf`	Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models. EMNLP (Findings) David Wingate, Mohammad Shoeybi, Taylor Sorensen
2022	J	`jnl`	Prompt Compression and Contrastive Conditioning for Controllability and Toxicity Reduction in Language Models. CoRR David Wingate, Mohammad Shoeybi, Taylor Sorensen
2022	J	`jnl`	Reducing Activation Recomputation in Large Transformer Models. CoRR Vijay Korthikanti, Jared Casper, Sangkug Lym, Lawrence McAfee, Michael Andersch, Mohammad Shoeybi, Bryan Catanzaro
2022	J	`jnl`	Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model. CoRR Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, Elton Zheng, Rewon Child, Reza Yazdani Aminabadi, Julie Bernauer, Xia Song, Mohammad Shoeybi, Yuxiong He, Michael Houston, Saurabh Tiwary, Bryan Catanzaro
2021	J	`jnl`	Efficient Large-Scale Language Model Training on GPU Clusters. CoRR Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vijay Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, Matei Zaharia
2021	A	`conf`	Efficient large-scale language model training on GPU clusters using megatron-LM. SC Deepak Narayanan, Mohammad Shoeybi, Jared Casper, Patrick LeGresley, Mostofa Patwary, Vijay Korthikanti, Dmitri Vainbrand, Prethvi Kashinkunti, Julie Bernauer, Bryan Catanzaro, Amar Phanishayee, Matei Zaharia
2021	—	`conf`	End-to-End Training of Neural Retrievers for Open-Domain Question Answering. ACL/IJCNLP (1) Devendra Singh Sachan, Mostofa Patwary, Mohammad Shoeybi, Neel Kant, Wei Ping, William L. Hamilton, Bryan Catanzaro
2021	J	`jnl`	End-to-End Training of Neural Retrievers for Open-Domain Question Answering. CoRR Devendra Singh Sachan, Mostofa Patwary, Mohammad Shoeybi, Neel Kant, Wei Ping, William L. Hamilton, Bryan Catanzaro
2021	J	`jnl`	Few-shot Instruction Prompts for Pretrained Language Models to Detect Social Biases. CoRR Shrimai Prabhumoye, Rafal Kocielnik, Mohammad Shoeybi, Anima Anandkumar, Bryan Catanzaro
2021	A*	`conf`	Long-Short Transformer: Efficient Transformers for Language and Vision. NeurIPS Chen Zhu, Wei Ping, Chaowei Xiao, Mohammad Shoeybi, Tom Goldstein, Anima Anandkumar, Bryan Catanzaro
2021	J	`jnl`	Long-Short Transformer: Efficient Transformers for Language and Vision. CoRR Chen Zhu, Wei Ping, Chaowei Xiao, Mohammad Shoeybi, Tom Goldstein, Anima Anandkumar, Bryan Catanzaro
2020	—	`conf`	BioMegatron: Larger Biomedical Domain Language Model. EMNLP (1) Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi, Raghav Mani
2020	J	`jnl`	BioMegatron: Larger Biomedical Domain Language Model. CoRR Hoo-Chang Shin, Yang Zhang, Evelina Bakhturina, Raul Puri, Mostofa Patwary, Mohammad Shoeybi, Raghav Mani
2020	A*	`conf`	Large Scale Multi-Actor Generative Dialog Modeling. ACL Alex Boyd, Raul Puri, Mohammad Shoeybi, Mostofa Patwary, Bryan Catanzaro
2020	J	`jnl`	Large Scale Multi-Actor Generative Dialog Modeling. CoRR Alex Boyd, Raul Puri, Mohammad Shoeybi, Mostofa Patwary, Bryan Catanzaro
2020	J	`jnl`	Local Knowledge Powered Conversational Agents. CoRR Sashank Santhanam, Wei Ping, Raul Puri, Mohammad Shoeybi, Mostofa Patwary, Bryan Catanzaro
2020	—	`conf`	MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models. EMNLP (1) Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar, Bryan Catanzaro
2020	J	`jnl`	MEGATRON-CNTRL: Controllable Story Generation with External Knowledge Using Large-Scale Language Models. CoRR Peng Xu, Mostofa Patwary, Mohammad Shoeybi, Raul Puri, Pascale Fung, Anima Anandkumar, Bryan Catanzaro
2020	J	`jnl`	Style Example-Guided Text Generation using Generative Adversarial Transformers. CoRR Kuo-Hao Zeng, Mohammad Shoeybi, Ming-Yu Liu
2020	—	`conf`	Training Question Answering Models From Synthetic Data. EMNLP (1) Raul Puri, Ryan Spring, Mohammad Shoeybi, Mostofa Patwary, Bryan Catanzaro
2020	J	`jnl`	Training Question Answering Models From Synthetic Data. CoRR Raul Puri, Ryan Spring, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro
2019	J	`jnl`	Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism. CoRR Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, Bryan Catanzaro
2019	J	`jnl`	Neural ODEs for Image Segmentation with Level Sets. CoRR Rafael Valle, Fitsum A. Reda, Mohammad Shoeybi, Patrick LeGresley, Andrew Tao, Bryan Catanzaro
2019	A*	`conf`	Unsupervised Video Interpolation Using Cycle Consistency. ICCV Fitsum A. Reda, Deqing Sun, Aysegul Dundar, Mohammad Shoeybi, Guilin Liu, Kevin J. Shih, Andrew Tao, Jan Kautz, Bryan Catanzaro
2019	J	`jnl`	Unsupervised Video Interpolation Using Cycle Consistency. CoRR Fitsum A. Reda, Deqing Sun, Aysegul Dundar, Mohammad Shoeybi, Guilin Liu, Kevin J. Shih, Andrew Tao, Jan Kautz, Bryan Catanzaro
2017	A*	`conf`	Deep Voice: Real-time Neural Text-to-Speech. ICML Sercan Ömer Arik, Mike Chrzanowski, Adam Coates, Gregory Frederick Diamos, Andrew Gibiansky, Yongguo Kang, Xian Li, John Miller, Andrew Y. Ng, Jonathan Raiman, Shubho Sengupta, Mohammad Shoeybi
2017	J	`jnl`	Deep Voice: Real-time Neural Text-to-Speech. CoRR Sercan Ömer Arik, Mike Chrzanowski, Adam Coates, Greg Diamos, Andrew Gibiansky, Yongguo Kang, Xian Li, John Miller, Jonathan Raiman, Shubho Sengupta, Mohammad Shoeybi
2017	J	`jnl`	Trace norm regularization and faster inference for embedded speech recognition RNNs. CoRR Markus Kliegl, Siddharth Goyal, Kexin Zhao, Kavya Srinet, Mohammad Shoeybi
2010	J	`jnl`	An adaptive implicit-explicit scheme for the DNS and LES of compressible flows on unstructured grids. J. Comput. Phys. Mohammad Shoeybi, Magnus Svärd, Frank E. Ham, Parviz Moin
2008	J	`jnl`	Stable and accurate schemes for the compressible Navier-Stokes equations. J. Comput. Phys. Ken Mattsson, Magnus Svärd, Mohammad Shoeybi
2006	J	`jnl`	Towards Wall-Normal Filtering for Large-Eddy Simulation. Multiscale Model. Simul. Jeremy A. Templeton, Mohammad Shoeybi