Ben Athiwaratkun

59 papers A* 10Misc 1Journal 40Unranked 8

Year	Rank	Type	Title / Venue / Authors
2026	J	`jnl`	CARE: Covariance-Aware and Rank-Enhanced Decomposition for Enabling Multi-Head Latent Attention. CoRR Zhongzhu Zhou, Fengxiang Bie, Ziyan Chen, Zhenyu Zhang, Yibo Yang, Junxiong Wang, Ben Athiwaratkun, Xiaoxia Wu, Shuaiwen Leon Song
2026	J	`jnl`	V CoRR Harman Singh, Xiuyu Li, Kusha Sareen, Monishwaran Maheswaran, Sijun Tan, Xiaoxia Wu, Junxiong Wang, Alpay Ariyak, Qingyang Wu, Samir Khaki, Rishabh Tiwari, Long Lian, Yucheng Lu, Boyi Li, Alane Suhr, Ben Athiwaratkun, Kurt Keutzer
2026	J	`jnl`	When RL Meets Adaptive Speculative Training: A Unified Training-Serving System. CoRR Junxiong Wang, Fengxiang Bie, Jisen Li, Zhongzhu Zhou, Zelei Shao, Yubo Wang, Yinghui Liu, Qingyang Wu, Avner May, Sri Yanamandra, Yineng Zhang, Ce Zhang, Tri Dao, Percy Liang, Ben Athiwaratkun, Shuaiwen Leon Song, Chenfeng Xu, Xiaoxia Wu
2025	J	`jnl`	Beat the long tail: Distribution-Aware Speculative Decoding for RL Training. CoRR Zelei Shao, Vikranth Srivatsa, Sanjana Srivastava, Qingyang Wu, Alpay Ariyak, Xiaoxia Wu, Ameen Patel, Jue Wang, Percy Liang, Tri Dao, Ce Zhang, Yiying Zhang, Ben Athiwaratkun, Chenfeng Xu, Junxiong Wang
2025	J	`jnl`	CDLM: Consistency Diffusion Language Models For Faster Sampling. CoRR Minseo Kim, Chenfeng Xu, Coleman Hooper, Harman Singh, Ben Athiwaratkun, Ce Zhang, Kurt Keutzer, Amir Gholami
2025	J	`jnl`	Data Diversification Methods In Alignment Enhance Math Performance In LLMs. CoRR Berkan Dokmeci, Qingyang Wu, Ben Athiwaratkun, Ce Zhang, Shuaiwen Leon Song, James Zou
2025	J	`jnl`	Disentangling Reasoning and Knowledge in Medical Large Language Models. CoRR Rahul Thapa, Qingyang Wu, Kevin Wu, Harrison Zhang, Angela Zhang, Eric Wu, Haotian Ye, Suhana Bedi, Nevin Aresh, Joseph Boen, Shriya Reddy, Ben Athiwaratkun, Shuaiwen Leon Song, James Zou
2025	J	`jnl`	How Well Can General Vision-Language Models Learn Medicine By Watching Public Educational Videos? CoRR Rahul Thapa, Andrew Li, Qingyang Wu, Bryan He, Yuki Sahashi, Christina Binder, Angela Zhang, Ben Athiwaratkun, Shuaiwen Leon Song, David Ouyang, James Zou
2025	J	`jnl`	Imitate Optimal Policy: Prevail and Induce Action Collapse in Policy Gradient. CoRR Zhongzhu Zhou, Yibo Yang, Ziyan Chen, Fengxiang Bie, Haojun Xia, Xiaoxia Wu, Robert Wu, Ben Athiwaratkun, Bernard Ghanem, Shuaiwen Leon Song
2025	J	`jnl`	Improving Model Alignment Through Collective Intelligence of Open-Source LLMS. CoRR Junlin Wang, Roy Xie, Shang Zhu, Jue Wang, Ben Athiwaratkun, Bhuwan Dhingra, Shuaiwen Leon Song, Ce Zhang, James Zou
2025	A*	`conf`	Improving Model Alignment Through Collective Intelligence of Open-Source Models. ICML Junlin Wang, Roy Xie, Shang Zhu, Jue Wang, Ben Athiwaratkun, Bhuwan Dhingra, Shuaiwen Leon Song, Ce Zhang, James Zou
2025	J	`jnl`	Intelligence per Watt: Measuring Intelligence Efficiency of Local AI. CoRR Jon Saad-Falcon, Avanika Narayan, Hakki Orhun Akengin, J. Wes Griffin, Herumb Shandilya, Adrian Gamarra Lafuente, Medhya Goel, Rebecca Joseph, Shlok Natarajan, Etash Kumar Guha, Shang Zhu, Ben Athiwaratkun, John Hennessy, Azalia Mirhoseini, Christopher Ré
2025	J	`jnl`	Kitty: Accurate and Efficient 2-bit KV Cache Quantization with Dynamic Channel-wise Precision Boost. CoRR Haojun Xia, Xiaoxia Wu, Jisen Li, Robert Wu, Junxiong Wang, Jue Wang, Chenxi Li, Aman Singhal, Alay Dilipbhai Shah, Alpay Ariyak, Donglin Zhuang, Zhongzhu Zhou, Ben Athiwaratkun, Zhen Zheng, Shuaiwen Leon Song
2025	A*	`conf`	Ladder-Residual: Parallelism-Aware Architecture for Accelerating Large Model Inference with Communication Overlapping. ICML Muru Zhang, Mayank Mishra, Zhongzhu Zhou, William Brandon, Jue Wang, Yoon Kim, Jonathan Ragan-Kelley, Shuaiwen Leon Song, Ben Athiwaratkun, Tri Dao
2025	J	`jnl`	Ladder-residual: parallelism-aware architecture for accelerating large model inference with communication overlapping. CoRR Muru Zhang, Mayank Mishra, Zhongzhu Zhou, William Brandon, Jue Wang, Yoon Kim, Jonathan Ragan-Kelley, Shuaiwen Leon Song, Ben Athiwaratkun, Tri Dao
2025	A*	`conf`	Mixture-of-Agents Enhances Large Language Model Capabilities. ICLR Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, James Zou
2025	J	`jnl`	Opportunistic Expert Activation: Batch-Aware Expert Routing for Faster Decode Without Retraining. CoRR Costin-Andrei Oncescu, Qingyang Wu, Wai Tong Chung, Robert Wu, Bryan Gopal, Junxiong Wang, Tri Dao, Ben Athiwaratkun
2025	J	`jnl`	Scaling Instruction-Tuned LLMs to Million-Token Contexts via Hierarchical Synthetic Data Generation. CoRR Linda He, Jue Wang, Maurice Weber, Shang Zhu, Ben Athiwaratkun, Ce Zhang
2025	A*	`conf`	Scaling Instruction-tuned LLMs to Million-token Contexts via Hierarchical Synthetic Data Generation. ICLR Linda He, Jue Wang, Maurice Weber, Shang Zhu, Ben Athiwaratkun, Ce Zhang
2025	J	`jnl`	Shrinking the Generation-Verification Gap with Weak Verifiers. CoRR Jon Saad-Falcon, Estefany Kelly Buchanan, Mayee F. Chen, Tzu-Heng Huang, Brendan McLaughlin, Tanvir Bhathal, Shang Zhu, Ben Athiwaratkun, Frederic Sala, Scott W. Linderman, Azalia Mirhoseini, Christopher Ré
2025	J	`jnl`	Staircase Streaming for Low-Latency Multi-Agent Inference. CoRR Junlin Wang, Jue Wang, Zhen Xu, Ben Athiwaratkun, Bhuwan Dhingra, Ce Zhang, James Zou
2025	J	`jnl`	Think Deep, Think Fast: Investigating Efficiency of Verifier-free Inference-time-scaling Methods. CoRR Junlin Wang, Shang Zhu, Jon Saad-Falcon, Ben Athiwaratkun, Qingyang Wu, Jue Wang, Shuaiwen Leon Song, Ce Zhang, Bhuwan Dhingra, James Zou
2025	A*	`conf`	Training-Free Activation Sparsity in Large Language Models. ICLR James Liu, Pragaash Ponnusamy, Tianle Cai, Han Guo, Yoon Kim, Ben Athiwaratkun
2025	J	`jnl`	Understanding and Steering the Cognitive Behaviors of Reasoning Models at Test-Time. CoRR Zhenyu Zhang, Xiaoxia Wu, Zhongzhu Zhou, Qingyang Wu, Yineng Zhang, Pragaash Ponnusamy, Harikaran Subbaraj, Jue Wang, Shuaiwen Leon Song, Ben Athiwaratkun
2025	J	`jnl`	When Does Divide and Conquer Work for Long Context LLM? A Noise Decomposition Framework. CoRR Zhen Xu, Shang Zhu, Jue Wang, Junlin Wang, Ben Athiwaratkun, Chi Wang, James Zou, Ce Zhang
2024	A*	`conf`	Bifurcated Attention for Single-Context Large-Batch Sampling. ICML Ben Athiwaratkun, Sujan Kumar Gonugondla, Sanjay Krishna Gouda, Haifeng Qian, Hantian Ding, Qing Sun, Jun Wang, Jiacheng Guo, Liangfu Chen, Parminder Bhatia, Ramesh Nallapati, Sudipta Sengupta, Bing Xiang
2024	J	`jnl`	Bifurcated Attention for Single-Context Large-Batch Sampling. CoRR Ben Athiwaratkun, Sujan Kumar Gonugondla, Sanjay Krishna Gouda, Haifeng Qian, Hantian Ding, Qing Sun, Jun Wang, Jiacheng Guo, Liangfu Chen, Parminder Bhatia, Ramesh Nallapati, Sudipta Sengupta, Bing Xiang
2024	J	`jnl`	Dragonfly: Multi-Resolution Zoom Supercharges Large Visual-Language Model. CoRR Kezhen Chen, Rahul Thapa, Rahul Chalamala, Ben Athiwaratkun, Shuaiwen Leon Song, James Zou
2024	J	`jnl`	Mixture-of-Agents Enhances Large Language Model Capabilities. CoRR Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, James Zou
2024	A*	`conf`	Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies. EMNLP Junlin Wang, Siddhartha Jain, Dejiao Zhang, Baishakhi Ray, Varun Kumar, Ben Athiwaratkun
2024	J	`jnl`	Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies. CoRR Junlin Wang, Siddhartha Jain, Dejiao Zhang, Baishakhi Ray, Varun Kumar, Ben Athiwaratkun
2024	A*	`conf`	RedPajama: an Open Dataset for Training Large Language Models. NeurIPS Maurice Weber, Daniel Y. Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, Ben Athiwaratkun, Rahul Chalamala, Kezhen Chen, Max Ryabinin, Tri Dao, Percy Liang, Christopher Ré, Irina Rish, Ce Zhang
2024	J	`jnl`	RedPajama: an Open Dataset for Training Large Language Models. CoRR Maurice Weber, Daniel Y. Fu, Quentin Anthony, Yonatan Oren, Shane Adams, Anton Alexandrov, Xiaozhong Lyu, Huu Nguyen, Xiaozhe Yao, Virginia Adams, Ben Athiwaratkun, Rahul Chalamala, Kezhen Chen, Max Ryabinin, Tri Dao, Percy Liang, Christopher Ré, Irina Rish, Ce Zhang
2024	—	`conf`	Token Alignment via Character Matching for Subword Completion. ACL (Findings) Ben Athiwaratkun, Shiqi Wang, Mingyue Shang, Yuchen Tian, Zijian Wang, Sujan Kumar Gonugondla, Sanjay Krishna Gouda, Robert Kwiatkowski, Ramesh Nallapati, Parminder Bhatia, Bing Xiang
2024	J	`jnl`	Token Alignment via Character Matching for Subword Completion. CoRR Ben Athiwaratkun, Shiqi Wang, Mingyue Shang, Yuchen Tian, Zijian Wang, Sujan Kumar Gonugondla, Sanjay Krishna Gouda, Rob Kwiatkowski, Ramesh Nallapati, Bing Xiang
2024	J	`jnl`	Training-Free Activation Sparsity in Large Language Models. CoRR James Liu, Pragaash Ponnusamy, Tianle Cai, Han Guo, Yoon Kim, Ben Athiwaratkun
2023	J	`jnl`	Greener yet Powerful: Taming Large Code Generation Models with Quantization. CoRR Xiaokai Wei, Sujan K. Gonugondla, Wasi Uddin Ahmad, Shiqi Wang, Baishakhi Ray, Haifeng Qian, Xiaopeng Li, Varun Kumar, Zijian Wang, Yuchen Tian, Qing Sun, Ben Athiwaratkun, Mingyue Shang, Murali Krishna Ramanathan, Parminder Bhatia, Bing Xiang
2023	A*	`conf`	Multi-lingual Evaluation of Code Generation Models. ICLR Ben Athiwaratkun, Sanjay Krishna Gouda, Zijian Wang, Xiaopeng Li, Yuchen Tian, Ming Tan, Wasi Uddin Ahmad, Shiqi Wang, Qing Sun, Mingyue Shang, Sujan Kumar Gonugondla, Hantian Ding, Varun Kumar, Nathan Fulton, Arash Farahani, Siddhartha Jain, Robert Giaquinto, Haifeng Qian, Murali Krishna Ramanathan, Ramesh Nallapati
2023	—	`conf`	Towards Greener Yet Powerful Code Generation via Quantization: An Empirical Study. ESEC/SIGSOFT FSE Xiaokai Wei, Sujan Kumar Gonugondla, Shiqi Wang, Wasi Uddin Ahmad, Baishakhi Ray, Haifeng Qian, Xiaopeng Li, Varun Kumar, Zijian Wang, Yuchen Tian, Qing Sun, Ben Athiwaratkun, Mingyue Shang, Murali Krishna Ramanathan, Parminder Bhatia, Bing Xiang
2022	J	`jnl`	Multi-lingual Evaluation of Code Generation Models. CoRR Ben Athiwaratkun, Sanjay Krishna Gouda, Zijian Wang, Xiaopeng Li, Yuchen Tian, Ming Tan, Wasi Uddin Ahmad, Shiqi Wang, Qing Sun, Mingyue Shang, Sujan Kumar Gonugondla, Hantian Ding, Varun Kumar, Nathan Fulton, Arash Farahani, Siddhartha Jain, Robert Giaquinto, Haifeng Qian, Murali Krishna Ramanathan, Ramesh Nallapati, Baishakhi Ray, Parminder Bhatia, Sudipta Sengupta, Dan Roth, Bing Xiang
2021	—	`conf`	Generative Context Pair Selection for Multi-hop Question Answering. EMNLP (1) Dheeru Dua, Cícero Nogueira dos Santos, Patrick Ng, Ben Athiwaratkun, Bing Xiang, Matt Gardner, Sameer Singh
2021	J	`jnl`	Generative Context Pair Selection for Multi-hop Question Answering. CoRR Dheeru Dua, Cícero Nogueira dos Santos, Patrick Ng, Ben Athiwaratkun, Bing Xiang, Matt Gardner, Sameer Singh
2021	J	`jnl`	Joint Text and Label Generation for Spoken Language Understanding. CoRR Yang Li, Ben Athiwaratkun, Cícero Nogueira dos Santos, Bing Xiang
2021	A*	`conf`	Structured Prediction as Translation between Augmented Natural Languages. ICLR Giovanni Paolini, Ben Athiwaratkun, Jason Krone, Jie Ma, Alessandro Achille, Rishita Anubhai, Cícero Nogueira dos Santos, Bing Xiang, Stefano Soatto
2021	J	`jnl`	Structured Prediction as Translation between Augmented Natural Languages. CoRR Giovanni Paolini, Ben Athiwaratkun, Jason Krone, Jie Ma, Alessandro Achille, Rishita Anubhai, Cícero Nogueira dos Santos, Bing Xiang, Stefano Soatto
2020	—	`conf`	Augmented Natural Language for Generative Sequence Labeling. EMNLP (1) Ben Athiwaratkun, Cícero Nogueira dos Santos, Jason Krone, Bing Xiang
2020	J	`jnl`	Augmented Natural Language for Generative Sequence Labeling. CoRR Ben Athiwaratkun, Cícero Nogueira dos Santos, Jason Krone, Bing Xiang
2019	—	`conf`	There Are Many Consistent Explanations of Unlabeled Data: Why You Should Average. ICLR (Poster) Ben Athiwaratkun, Marc Finzi, Pavel Izmailov, Andrew Gordon Wilson
2018	J	`jnl`	Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification. Trans. Assoc. Comput. Linguistics Xilun Chen, Yu Sun, Ben Athiwaratkun, Claire Cardie, Kilian Q. Weinberger
2018	—	`conf`	Hierarchical Density Order Embeddings. ICLR (Poster) Ben Athiwaratkun, Andrew Gordon Wilson
2018	J	`jnl`	Hierarchical Density Order Embeddings. CoRR Ben Athiwaratkun, Andrew Gordon Wilson
2018	J	`jnl`	Improving Consistency-Based Semi-Supervised Learning with Weight Averaging. CoRR Ben Athiwaratkun, Marc Finzi, Pavel Izmailov, Andrew Gordon Wilson
2018	—	`conf`	Probabilistic FastText for Multi-Sense Word Embeddings. ACL (1) Ben Athiwaratkun, Andrew Gordon Wilson, Anima Anandkumar
2018	J	`jnl`	Probabilistic FastText for Multi-Sense Word Embeddings. CoRR Ben Athiwaratkun, Andrew Gordon Wilson, Anima Anandkumar
2017	Misc	`conf`	Malware classification with LSTM and GRU language models and a character-level CNN. ICASSP Ben Athiwaratkun, Jack W. Stokes
2017	—	`conf`	Multimodal Word Distributions. ACL (1) Ben Athiwaratkun, Andrew Gordon Wilson
2017	J	`jnl`	Multimodal Word Distributions. CoRR Ben Athiwaratkun, Andrew Gordon Wilson
2016	J	`jnl`	Adversarial Deep Averaging Networks for Cross-Lingual Sentiment Classification. CoRR Xilun Chen, Ben Athiwaratkun, Yu Sun, Kilian Q. Weinberger, Claire Cardie
2015	J	`jnl`	Feature Representation in Convolutional Neural Networks. CoRR Ben Athiwaratkun, Keegan Kang