Joel Hestness - RankMe

44 papers A* 5A 1B 1Journal 28Unranked 9

Year	Rank	Type	Title / Venue / Authors
2025	J	`jnl`	Don't be lazy: CompleteP enables compute-efficient deep transformers. CoRR Nolan Dey, Bin Claire Zhang, Lorenzo Noci, Mufan Bill Li, Blake Bordelon, Shane Bergsma, Cengiz Pehlevan, Boris Hanin, Joel Hestness
2025	J	`jnl`	Power Lines: Scaling Laws for Weight Decay and Batch Size in LLM Pre-training. CoRR Shane Bergsma, Nolan Dey, Gurpreet Gosal, Gavia Gray, Daria Soboleva, Joel Hestness
2025	J	`jnl`	Predicting Training Re-evaluation Curves Enables Effective Data Curriculums for LLMs. CoRR Shane Bergsma, Nolan Dey, Joel Hestness
2025	J	`jnl`	Scaling with Collapse: Efficient and Predictable Training of LLM Families. CoRR Shane Bergsma, Bin Claire Zhang, Nolan Dey, Shaheer Muhammad, Gurpreet Gosal, Joel Hestness
2025	A*	`conf`	Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs. ICLR Shane Bergsma, Nolan Simran Dey, Gurpreet Gosal, Gavia Gray, Daria Soboleva, Joel Hestness
2025	J	`jnl`	Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs. CoRR Shane Bergsma, Nolan Dey, Gurpreet Gosal, Gavia Gray, Daria Soboleva, Joel Hestness
2024	J	`jnl`	Bilingual Adaptation of Monolingual Foundation Models. CoRR Gurpreet Gosal, Yishi Xu, Gokul Ramakrishnan, Rituraj Joshi, Avraham Sheinin, Zhiming Chen, Biswajit Mishra, Natalia Vassilieva, Joel Hestness, Neha Sengupta, Sunil Kumar Sahu, Bokang Jia, Onkar Pandit, Satheesh Katipomu, Samta Kamboj, Samujjwal Ghosh, Rahul Pal, Parvez Mullah, Soundar Doraiswamy, Mohamed El Karim Chami, Preslav Nakov
2024	J	`jnl`	Crystal: Illuminating LLM Abilities on Language and Code. CoRR Tianhua Tao, Junbo Li, Bowen Tan, Hongyi Wang, William Marshall, Bhargav Kanakiya, Joel Hestness, Natalia Vassilieva, Zhiqiang Shen, Eric P. Xing, Zhengzhong Liu
2024	—	`conf`	Efficiently Disentangle Causal Representations. CPAL Yuanpeng Li, Joel Hestness, Mohamed Elhoseiny, Liang Zhao, Kenneth Church
2024	—	`conf`	MediSwift: Efficient Sparse Pre-trained Biomedical Language Models. ACL (Findings) Vithursan Thangarasa, Mahmoud Salem, Shreyas Saxena, Chen-Yu Leong, Joel Hestness, Sean Lie
2024	J	`jnl`	MediSwift: Efficient Sparse Pre-trained Biomedical Language Models. CoRR Vithursan Thangarasa, Mahmoud Salem, Shreyas Saxena, Kevin Leong, Joel Hestness, Sean Lie
2024	A*	`conf`	Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers. NeurIPS Gavia Gray, Aman Tiwari, Shane Bergsma, Joel Hestness
2024	J	`jnl`	Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers. CoRR Gavia Gray, Aman Tiwari, Shane Bergsma, Joel Hestness
2024	A*	`conf`	Sparse maximal update parameterization: A holistic approach to sparse training dynamics. NeurIPS Nolan Dey, Shane Bergsma, Joel Hestness
2024	J	`jnl`	Sparse maximal update parameterization: A holistic approach to sparse training dynamics. CoRR Nolan Dey, Shane Bergsma, Joel Hestness
2023	J	`jnl`	BTLM-3B-8K: 7B Parameter Performance in a 3B Parameter Model. CoRR Nolan Dey, Daria Soboleva, Faisal Al-Khateeb, Bowen Yang, Ribhu Pathria, Hemant Khachane, Shaheer Muhammad, Zhiming Chen, Robert Myers, Jacob Robert Steeves, Natalia Vassilieva, Marvin Tom, Joel Hestness
2023	J	`jnl`	Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster. CoRR Nolan Dey, Gurpreet Gosal, Zhiming Chen, Hemant Khachane, William Marshall, Ribhu Pathria, Marvin Tom, Joel Hestness
2023	J	`jnl`	Position Interpolation Improves ALiBi Extrapolation. CoRR Faisal Al-Khateeb, Nolan Dey, Daria Soboleva, Joel Hestness
2023	—	`conf`	RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network. MLSys Vitaliy Chiley, Vithursan Thangarasa, Abhay Gupta, Anshul Samar, Joel Hestness, Dennis DeCoste
2023	J	`jnl`	SlimPajama-DC: Understanding Data Combinations for LLM Training. CoRR Zhiqiang Shen, Tianhua Tao, Liqun Ma, Willie Neiswanger, Zhengzhong Liu, Hongyi Wang, Bowen Tan, Joel Hestness, Natalia Vassilieva, Daria Soboleva, Eric P. Xing
2022	J	`jnl`	Efficiently Disentangle Causal Representations. CoRR Yuanpeng Li, Joel Hestness, Mohamed Elhoseiny, Liang Zhao, Kenneth Church
2022	J	`jnl`	RevBiFPN: The Fully Reversible Bidirectional Feature Pyramid Network. CoRR Vitaliy Chiley, Vithursan Thangarasa, Abhay Gupta, Anshul Samar, Joel Hestness, Dennis DeCoste
2022	J	`jnl`	Time Dependency, Data Flow, and Competitive Advantage. CoRR Ehsan Valavi, Joel Hestness, Marco Iansiti, Newsha Ardalani, Feng Zhu, Karim R. Lakhani
2022	J	`jnl`	Time and the Value of Data. CoRR Ehsan Valavi, Joel Hestness, Newsha Ardalani, Marco Iansiti
2021	J	`jnl`	Memory Efficient 3D U-Net with Reversible Mobile Inverted Bottlenecks for Brain Tumor Segmentation. CoRR Mihir Pendse, Vithursan Thangarasa, Vitaliy Chiley, Ryan Holmdahl, Joel Hestness, Dennis DeCoste
2021	—	`conf`	Pipelined Backpropagation at Scale: Training Large Models without Batches. MLSys Atli Kosson, Vitaliy Chiley, Abhinav Venigalla, Joel Hestness, Urs Köster
2020	—	`conf`	Memory Efficient 3D U-Net with Reversible Mobile Inverted Bottlenecks for Brain Tumor Segmentation. BrainLes@MICCAI (2) Mihir Pendse, Vithursan Thangarasa, Vitaliy Chiley, Ryan Holmdahl, Joel Hestness, Dennis DeCoste
2020	J	`jnl`	Pipelined Backpropagation at Scale: Training Large Models without Batches. CoRR Atli Kosson, Vitaliy Chiley, Abhinav Venigalla, Joel Hestness, Urs Köster
2019	J	`jnl`	A survey of 25 years of evaluation. Nat. Lang. Eng. Kenneth Ward Church, Joel Hestness
2019	J	`jnl`	Beyond Human-Level Accuracy: Computational Challenges in Deep Learning. CoRR Joel Hestness, Newsha Ardalani, Greg Diamos
2019	B	`conf`	Beyond human-level accuracy: computational challenges in deep learning. PPoPP Joel Hestness, Newsha Ardalani, Gregory F. Diamos
2019	—	`conf`	Compositional Generalization for Primitive Substitutions. EMNLP/IJCNLP (1) Yuanpeng Li, Liang Zhao, Jianyu Wang, Joel Hestness
2019	J	`jnl`	Compositional Generalization for Primitive Substitutions. CoRR Yuanpeng Li, Liang Zhao, Jianyu Wang, Joel Hestness
2017	A	`conf`	Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting. INTERSPEECH Sercan Ömer Arik, Markus Kliegl, Rewon Child, Joel Hestness, Andrew Gibiansky, Christopher Fougner, Ryan Prenger, Adam Coates
2017	J	`jnl`	Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting. CoRR Sercan Ömer Arik, Markus Kliegl, Rewon Child, Joel Hestness, Andrew Gibiansky, Christopher Fougner, Ryan Prenger, Adam Coates
2017	J	`jnl`	Deep Learning Scaling is Predictable, Empirically. CoRR Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory F. Diamos, Heewoo Jun, Hassan Kianinejad, Md. Mostofa Ali Patwary, Yang Yang, Yanqi Zhou
2015	—	`conf`	GPU Computing Pipeline Inefficiencies and Optimization Opportunities in Heterogeneous CPU-GPU Processors. IISWC Joel Hestness, Stephen W. Keckler, David A. Wood
2015	J	`jnl`	gem5-gpu: A Heterogeneous CPU-GPU Simulator. IEEE Comput. Archit. Lett. Jason Power, Joel Hestness, Marc S. Orr, Mark D. Hill, David A. Wood
2014	—	`conf`	A comparative analysis of microarchitecture effects on CPU and GPU memory system behavior. IISWC Joel Hestness, Stephen W. Keckler, David A. Wood
2012	J	`jnl`	A QoS-Enabled On-Die Interconnect Fabric for Kilo-Node Chips. IEEE Micro Boris Grot, Joel Hestness, Stephen W. Keckler, Onur Mutlu
2011	A*	`conf`	Kilo-NOC: a heterogeneous network-on-chip architecture for scalability and service guarantees. ISCA Boris Grot, Joel Hestness, Stephen W. Keckler, Onur Mutlu
2011	J	`jnl`	The gem5 simulator. SIGARCH Comput. Archit. News Nathan L. Binkert, Bradford M. Beckmann, Gabriel Black, Steven K. Reinhardt, Ali G. Saidi, Arkaprava Basu, Joel Hestness, Derek Hower, Tushar Krishna, Somayeh Sardashti, Rathijit Sen, Korey Sewell, Muhammad Shoaib Bin Altaf, Nilay Vaish, Mark D. Hill, David A. Wood
2010	—	`conf`	Netrace: dependency-driven trace-based network-on-chip simulation. NoCArc@MICRO Joel Hestness, Boris Grot, Stephen W. Keckler
2009	A*	`conf`	Express Cube Topologies for on-Chip Interconnects. HPCA Boris Grot, Joel Hestness, Stephen W. Keckler, Onur Mutlu