Olatunji Ruwase

50 papers A* 12A 7B 1Journal 22Unranked 7

Year	Rank	Type	Title / Venue / Authors
2026	—	`conf`	SuperOffload: Unleashing the Power of Large-Scale LLM Training on Superchips. ASPLOS (1) Xinyu Lian, Masahiro Tanaka, Olatunji Ruwase, Minjia Zhang
2025	J	`jnl`	DeepCompile: A Compiler-Driven Approach to Optimizing Distributed Deep Learning Training. CoRR Masahiro Tanaka, Du Li, Umesh Chand, Ali Zafar, Haiying Shen, Olatunji Ruwase
2025	J	`jnl`	SuperOffload: Unleashing the Power of Large-Scale LLM Training on Superchips. CoRR Xinyu Lian, Masahiro Tanaka, Olatunji Ruwase, Minjia Zhang
2025	—	`conf`	Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer. MLSys Jinghan Yao, Sam Ade Jacobs, Masahiro Tanaka, Olatunji Ruwase, Hari Subramoni, Dhabaleswar K. Panda
2025	—	`conf`	Universal Checkpointing: A Flexible and Efficient Distributed Checkpointing System for Large-Scale DNN Training with Reconfigurable Parallelism. USENIX ATC Xinyu Lian, Sam Ade Jacobs, Lev Kurilenko, Masahiro Tanaka, Stas Bekman, Olatunji Ruwase, Minjia Zhang
2024	J	`jnl`	Domino: Eliminating Communication in LLM Training via Generic Tensor Slicing and Overlapping. CoRR Guanhua Wang, Chengming Zhang, Zheyu Shen, Ang Li, Olatunji Ruwase
2024	J	`jnl`	FP6-LLM: Efficiently Serving Large Language Models Through FP6-Centric Algorithm-System Co-Design. CoRR Haojun Xia, Zhen Zheng, Xiaoxia Wu, Shiyang Chen, Zhewei Yao, Stephen Youn, Arash Bakhtiari, Michael Wyatt, Donglin Zhuang, Zhongzhu Zhou, Olatunji Ruwase, Yuxiong He, Shuaiwen Leon Song
2024	J	`jnl`	FastPersist: Accelerating Model Checkpointing in Deep Learning. CoRR Guanhua Wang, Olatunji Ruwase, Bing Xie, Yuxiong He
2024	A*	`conf`	Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding. NeurIPS Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang
2024	J	`jnl`	Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding. CoRR Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang
2024	J	`jnl`	Mojito: Motion Trajectory and Intensity Control for Video Generation. CoRR Xuehai He, Shuohang Wang, Jianwei Yang, Xiaoxia Wu, Yiping Wang, Kuan Wang, Zheng Zhan, Olatunji Ruwase, Yelong Shen, Xin Eric Wang
2024	—	`conf`	Quant-LLM: Accelerating the Serving of Large Language Models via FP6-Centric Algorithm-System Co-Design on Modern GPUs. USENIX ATC Haojun Xia, Zhen Zheng, Xiaoxia Wu, Shiyang Chen, Zhewei Yao, Stephen Youn, Arash Bakhtiari, Michael Wyatt, Donglin Zhuang, Zhongzhu Zhou, Olatunji Ruwase, Yuxiong He, Shuaiwen Leon Song
2024	A	`conf`	RecFlex: Enabling Feature Heterogeneity-Aware Optimization for Deep Recommendation Models with Flexible Schedules. SC Zaifeng Pan, Zhen Zheng, Feng Zhang, Bing Xie, Ruofan Wu, Shaden Smith, Chuanjie Liu, Olatunji Ruwase, Xiaoyong Du, Yufei Ding
2024	J	`jnl`	Training Ultra Long Context Language Model with Fully Pipelined Distributed Transformer. CoRR Jinghan Yao, Sam Ade Jacobs, Masahiro Tanaka, Olatunji Ruwase, Aamir Shafi, Hari Subramoni, Dhabaleswar K. Panda
2024	J	`jnl`	Universal Checkpointing: Efficient and Flexible Checkpointing for Large Scale Distributed Training. CoRR Xinyu Lian, Sam Ade Jacobs, Lev Kurilenko, Masahiro Tanaka, Stas Bekman, Olatunji Ruwase, Minjia Zhang
2024	A*	`conf`	ZeRO++: Extremely Efficient Collective Communication for Large Model Training. ICLR Guanhua Wang, Heyang Qin, Sam Ade Jacobs, Xiaoxia Wu, Connor Holmes, Zhewei Yao, Samyam Rajbhandari, Olatunji Ruwase, Feng Yan, Lei Yang, Yuxiong He
2023	A	`conf`	A Hybrid Tensor-Expert-Data Parallelism Approach to Optimize Mixture-of-Experts Training. ICS Siddharth Singh, Olatunji Ruwase, Ammar Ahmad Awan, Samyam Rajbhandari, Yuxiong He, Abhinav Bhatele
2023	J	`jnl`	A Novel Tensor-Expert Hybrid Parallelism Approach to Scale Mixture-of-Experts Training. CoRR Siddharth Singh, Olatunji Ruwase, Ammar Ahmad Awan, Samyam Rajbhandari, Yuxiong He, Abhinav Bhatele
2023	J	`jnl`	DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales. CoRR Zhewei Yao, Reza Yazdani Aminabadi, Olatunji Ruwase, Samyam Rajbhandari, Xiaoxia Wu, Ammar Ahmad Awan, Jeff Rasley, Minjia Zhang, Conglong Li, Connor Holmes, Zhongzhu Zhou, Michael Wyatt, Molly Smith, Lev Kurilenko, Heyang Qin, Masahiro Tanaka, Shuai Che, Shuaiwen Leon Song, Yuxiong He
2023	J	`jnl`	DeepSpeed-VisualChat: Multi-Round Multi-Image Interleave Chat via Multi-Modal Causal Attention. CoRR Zhewei Yao, Xiaoxia Wu, Conglong Li, Minjia Zhang, Heyang Qin, Olatunji Ruwase, Ammar Ahmad Awan, Samyam Rajbhandari, Yuxiong He
2023	J	`jnl`	SHARP: An Adaptable, Energy-Efficient Accelerator for Recurrent Neural Networks. ACM Trans. Embed. Comput. Syst. Reza Yazdani Aminabadi, Olatunji Ruwase, Minjia Zhang, Yuxiong He, José-María Arnau, Antonio González
2023	J	`jnl`	ZeRO++: Extremely Efficient Collective Communication for Giant Model Training. CoRR Guanhua Wang, Heyang Qin, Sam Ade Jacobs, Connor Holmes, Samyam Rajbhandari, Olatunji Ruwase, Feng Yan, Lei Yang, Yuxiong He
2023	J	`jnl`	ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Tasks. CoRR Xiaoxia Wu, Haojun Xia, Stephen Youn, Zhen Zheng, Shiyang Chen, Arash Bakhtiari, Michael Wyatt, Reza Yazdani Aminabadi, Yuxiong He, Olatunji Ruwase, Leon Song, Zhewei Yao
2022	J	`jnl`	DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale. CoRR Reza Yazdani Aminabadi, Samyam Rajbhandari, Minjia Zhang, Ammar Ahmad Awan, Cheng Li, Du Li, Elton Zheng, Jeff Rasley, Shaden Smith, Olatunji Ruwase, Yuxiong He
2022	A	`conf`	DeepSpeed- Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale. SC Reza Yazdani Aminabadi, Samyam Rajbhandari, Ammar Ahmad Awan, Cheng Li, Du Li, Elton Zheng, Olatunji Ruwase, Shaden Smith, Minjia Zhang, Jeff Rasley, Yuxiong He
2021	A*	`conf`	SimiGrad: Fine-Grained Adaptive Batching for Large Scale Training using Gradient Similarity Measurement. NeurIPS Heyang Qin, Samyam Rajbhandari, Olatunji Ruwase, Feng Yan, Lei Yang, Yuxiong He
2021	J	`jnl`	ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning. CoRR Samyam Rajbhandari, Olatunji Ruwase, Jeff Rasley, Shaden Smith, Yuxiong He
2021	—	`conf`	ZeRO-Offload: Democratizing Billion-Scale Model Training. USENIX ATC Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, Olatunji Ruwase, Shuangyan Yang, Minjia Zhang, Dong Li, Yuxiong He
2021	J	`jnl`	ZeRO-Offload: Democratizing Billion-Scale Model Training. CoRR Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, Olatunji Ruwase, Shuangyan Yang, Minjia Zhang, Dong Li, Yuxiong He
2021	A	`conf`	ZeRO-infinity: breaking the GPU memory wall for extreme scale deep learning. SC Samyam Rajbhandari, Olatunji Ruwase, Jeff Rasley, Shaden Smith, Yuxiong He
2020	A*	`conf`	DeepSpeed: System Optimizations Enable Training Deep Learning Models with Over 100 Billion Parameters. KDD Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, Yuxiong He
2020	A	`conf`	ZeRO: memory optimizations toward training trillion parameter models. SC Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, Yuxiong He
2019	—	`conf`	Accelerating Large Scale Deep Learning Inference through DeepCPU at Microsoft. OpML Minjia Zhang, Samyam Rajbhandari, Wenhan Wang, Elton Zheng, Olatunji Ruwase, Jeff Rasley, Jason Li, Junhua Wang, Yuxiong He
2019	J	`jnl`	LSTM-Sharp: An Adaptable, Energy-Efficient Hardware Accelerator for Long Short-Term Memory. CoRR Reza Yazdani, Olatunji Ruwase, Minjia Zhang, Yuxiong He, José-María Arnau, Antonio González
2019	J	`jnl`	ZeRO: Memory Optimization Towards Training A Trillion Parameter Models. CoRR Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, Yuxiong He
2018	J	`jnl`	Efficient Deep Neural Network Serving: Fast and Furious. IEEE Trans. Netw. Serv. Manag. Feng Yan, Yuxiong He, Olatunji Ruwase, Evgenia Smirni
2017	A	`conf`	HyperDrive: exploring hyperparameters with POP scheduling. Middleware Jeff Rasley, Yuxiong He, Feng Yan, Olatunji Ruwase, Rodrigo Fonseca
2017	A*	`conf`	Optimizing CNNs on Multicores for Scalability, Performance and Goodput. ASPLOS Samyam Rajbhandari, Yuxiong He, Olatunji Ruwase, Michael Carbin, Trishul M. Chilimbi
2016	A	`conf`	SERF: efficient scheduling for fast deep neural network serving via judicious parallelism. SC Feng Yan, Yuxiong He, Olatunji Ruwase, Evgenia Smirni
2015	A*	`conf`	Page overlays: an enhanced virtual memory framework to enable fine-grained memory management. ISCA Vivek Seshadri, Gennady Pekhimenko, Olatunji Ruwase, Onur Mutlu, Phillip B. Gibbons, Michael A. Kozuch, Todd C. Mowry, Trishul M. Chilimbi
2015	A*	`conf`	Performance Modeling and Scalability Optimization of Distributed Deep Learning Systems. KDD Feng Yan, Olatunji Ruwase, Yuxiong He, Trishul M. Chilimbi
2015	—	`conf`	Toward accelerating deep learning at scale using specialized hardware in the datacenter. Hot Chips Symposium Kalin Ovtcharov, Olatunji Ruwase, Joo-Young Kim, Jeremy Fowers, Karin Strauss, Eric S. Chung
2014	A*	`conf`	Guardrail: a high fidelity approach to protecting hardware devices from buggy drivers. ASPLOS Olatunji Ruwase, Michael A. Kozuch, Phillip B. Gibbons, Todd C. Mowry
2013	—	`—`	Improving Device Driver Reliability through Decoupled Dynamic Binary Analyses. Olatunji Ruwase
2010	A*	`conf`	Decoupled lifeguards: enabling path optimizations for dynamic correctness checking tools. PLDI Olatunji Ruwase, Shimin Chen, Phillip B. Gibbons, Todd C. Mowry
2009	J	`jnl`	Flexible Hardware Acceleration for Instruction-Grain Lifeguards. IEEE Micro Shimin Chen, Michael Kozuch, Phillip B. Gibbons, Michael P. Ryan, Theodoros Strigkos, Todd C. Mowry, Olatunji Ruwase, Evangelos Vlachos, Babak Falsafi, Vijaya Ramachandran
2008	A*	`conf`	Ditto: a system for opportunistic caching in multi-hop wireless networks. MobiCom Fahad R. Dogar, Amar Phanishayee, Himabindu Pucha, Olatunji Ruwase, David G. Andersen
2008	A*	`conf`	Flexible Hardware Acceleration for Instruction-Grain Program Monitoring. ISCA Shimin Chen, Michael Kozuch, Theodoros Strigkos, Babak Falsafi, Phillip B. Gibbons, Todd C. Mowry, Vijaya Ramachandran, Olatunji Ruwase, Michael P. Ryan, Evangelos Vlachos
2008	B	`conf`	Parallelizing dynamic information flow tracking. SPAA Olatunji Ruwase, Phillip B. Gibbons, Todd C. Mowry, Vijaya Ramachandran, Shimin Chen, Michael Kozuch, Michael P. Ryan
2004	A*	`conf`	A Practical Dynamic Buffer Overflow Detector. NDSS Olatunji Ruwase, Monica S. Lam