Shiwei Liu - RankMe

139 papers A* 40A 2B 1C 1Misc 1Journal 84Unranked 10

Year	Rank	Type	Title / Venue / Authors
2026	J	`jnl`	ActTail: Global Activation Sparsity in Large Language Models. CoRR Wenwen Hou, Xinyuan Song, Shiwei Liu
2026	J	`jnl`	Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning. Trans. Mach. Learn. Res. Mingyu Cao, Gen Li, Jie Ji, Jiaqi Zhang, Ajay Jaiswal, Li Shen, Xiaolong Ma, Shiwei Liu, Lu Yin
2026	J	`jnl`	When Does Sparsity Mitigate the Curse of Depth in LLMs. CoRR Dilxat Muhtar, Xinyuan Song, Sebastian Pokutta, Max Zimmer, Nico Pelleriti, Thomas Hofmann, Shiwei Liu
2026	J	`jnl`	Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding? CoRR Pengxiang Li, Dilxat Muhtar, Tianlong Chen, Lu Yin, Shiwei Liu
2025	J	`jnl`	A Technical Study into 0.5B Reasoning Language Models. CoRR Xialie Zhuang, Peixian Ma, Zhikai Jia, Shiwei Liu, Zheng Cao
2025	J	`jnl`	AlphaDecay: Module-wise Weight Decay for Heavy-Tailed Balancing in LLMs. CoRR Di He, Ajay Jaiswal, Songjun Tu, Li Shen, Ganzhao Yuan, Shiwei Liu, Lu Yin
2025	J	`jnl`	Chain-of-Experts: Unlocking the Communication Power of Mixture-of-Experts Models. CoRR Zihan Wang, Rui Pan, Jiarui Yao, Robert Csordas, Linjie Li, Lu Yin, Jiajun Wu, Tong Zhang, Manling Li, Shiwei Liu
2025	A*	`conf`	Composable Interventions for Language Models. ICLR Arinbjörn Kolbeinsson, Kyle O'Brien, Tianjin Huang, Shanghua Gao, Shiwei Liu, Jonathan Richard Schwarz, Anurag Jayant Vaidya, Faisal Mahmood, Marinka Zitnik, Tianlong Chen, Thomas Hartvigsen
2025	J	`jnl`	Data-Adaptive Weight-Ensembling for Multi-task Model Fusion. Int. J. Comput. Vis. Anke Tang, Li Shen, Yong Luo, Shiwei Liu, Han Hu, Bo Du, Dacheng Tao
2025	J	`jnl`	Demystifying the Roles of LLM Layers in Retrieval, Knowledge, and Reasoning. CoRR Xinyuan Song, Keyu Wang, Pengxiang Li, Lu Yin, Shiwei Liu
2025	J	`jnl`	Diffusion Language Models Know the Answer Before Decoding. CoRR Pengxiang Li, Yefan Zhou, Dilxat Muhtar, Lu Yin, Shilin Yan, Li Shen, Yi Liang, Soroush Vosoughi, Shiwei Liu
2025	J	`jnl`	Double-Checker: Enhancing Reasoning of Slow-Thinking LLMs via Self-Critical Fine-Tuning. CoRR Xin Xu, Tianlong Chen, Fan Zhang, Wanlong Liu, Pengxiang Li, Ajay Kumar Jaiswal, Yuchen Yan, Jishan Hu, Yang Wang, Hao Chen, Shiwei Liu, Shizhe Diao, Can Yang, Lu Yin
2025	J	`jnl`	FS-GNN: Improving Fairness in Graph Neural Networks via Joint Sparsification. Neurocomputing Jiaxu Zhao, Tianjin Huang, Shiwei Liu, Jie Yin, Yulong Pei, Meng Fang, Mykola Pechenizkiy
2025	A*	`conf`	From Low Rank Gradient Subspace Stabilization to Low-Rank Weights: Observations, Theories, and Applications. ICML Ajay Kumar Jaiswal, Yifan Wang, Lu Yin, Shiwei Liu, Runjin Chen, Jiawei Zhao, Ananth Grama, Yuandong Tian, Zhangyang Wang
2025	Misc	`conf`	Full-Rank No More: Low-Rank Weight Training for Modern Speech Recognition Models. ICASSP Adriana Fernandez-Lopez, Shiwei Liu, Lu Yin, Stavros Petridis, Maja Pantic
2025	J	`jnl`	GPAS: Accelerating Convergence of LLM Pretraining via Gradient-Preserving Activation Scaling. CoRR Tianlong Chen, Xin Xu, Zijing Liu, Pengxiang Li, Xinyuan Song, Ajay Kumar Jaiswal, Fan Zhang, Jishan Hu, Yang Wang, Hao Chen, Shizhe Diao, Shiwei Liu, Yu Li, Yin Lu, Can Yang
2025	J	`jnl`	GPTailor: Large Language Model Pruning Through Layer Cutting and Stitching. CoRR Guinan Su, Li Shen, Lu Yin, Shiwei Liu, Yanwu Yang, Jonas Geiping
2025	A*	`conf`	LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning. ICML Zihang Liu, Tianyu Pang, Oleg Balabanov, Chaoqun Yang, Tianjin Huang, Lu Yin, Yaoqing Yang, Shiwei Liu
2025	J	`jnl`	LIFT the Veil for the Truth: Principal Weights Emerge after Rank Reduction for Reasoning-Focused Supervised Fine-Tuning. CoRR Zihang Liu, Tianyu Pang, Oleg Balabanov, Chaoqun Yang, Tianjin Huang, Lu Yin, Yaoqing Yang, Shiwei Liu
2025	J	`jnl`	LOST: Low-rank and Sparse Pre-training for Large Language Models. CoRR Jiaxi Li, Lu Yin, Li Shen, Jinjin Xu, Liwu Xu, Tianjin Huang, Wenwu Wang, Shiwei Liu, Xilu Wang
2025	J	`jnl`	Leave it to the Specialist: Repair Sparse LLMs with Sparse Fine-Tuning via Sparsity Evolution. CoRR Qiao Xiao, Alan Ansell, Boqian Wu, Lu Yin, Mykola Pechenizkiy, Shiwei Liu, Decebal Constantin Mocanu
2025	A*	`conf`	Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More. ICML Xialie Zhuang, Zhikai Jia, Jianjin Li, Zhenyu Zhang, Li Shen, Zheng Cao, Shiwei Liu
2025	J	`jnl`	Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More. CoRR Xialie Zhuang, Zhikai Jia, Jianjin Li, Zhenyu Zhang, Li Shen, Zheng Cao, Shiwei Liu
2025	A*	`conf`	Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN. ICLR Pengxiang Li, Lu Yin, Shiwei Liu
2025	J	`jnl`	Model Unmerging: Making Your Models Unmergeable for Secure Model Sharing. CoRR Zihao Wang, Enneng Yang, Lu Yin, Shiwei Liu, Li Shen
2025	J	`jnl`	Neural Sum-of-Squares: Certifying the Nonnegativity of Polynomials with Transformers. CoRR Nico Pelleriti, Christoph Spiegel, Shiwei Liu, David Martínez-Rubio, Max Zimmer, Sebastian Pokutta
2025	J	`jnl`	NeuroTrails: Training with Dynamic Sparse Heads as the Key to Effective Ensembling. CoRR Bram Grooten, Farid Hasanov, Chenxiang Zhang, Qiao Xiao, Boqian Wu, Zahra Atashgahi, Ghada Sokar, Shiwei Liu, Lu Yin, Elena Mocanu, Mykola Pechenizkiy, Decebal Constantin Mocanu
2025	J	`jnl`	O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning. CoRR Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, Dacheng Tao
2025	—	`conf`	Outlier-weighed Layerwise Sampling for LLM Fine-tuning. ACL (Findings) Pengxiang Li, Lu Yin, Xiaowei Gao, Shiwei Liu
2025	—	`conf`	Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients. CPAL Zhenyu Zhang, Ajay Kumar Jaiswal, Lu Yin, Shiwei Liu, Jiawei Zhao, Yuandong Tian, Zhangyang Wang
2025	J	`jnl`	Revisiting Flatness-Aware Optimization in Continual Learning With Orthogonal Gradient Projection. IEEE Trans. Pattern Anal. Mach. Intell. Enneng Yang, Li Shen, Zhenyi Wang, Shiwei Liu, Guibing Guo, Xing-wei Wang, Dacheng Tao
2025	J	`jnl`	Rewiring Experts on the Fly:Continuous Rerouting for Better Online Adaptation in Mixture-of-Expert models. CoRR Guinan Su, Yanwu Yang, Li Shen, Lu Yin, Shiwei Liu, Jonas Geiping
2025	A*	`conf`	SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training. ICLR Tianjin Huang, Ziquan Zhu, Gaojie Jin, Lu Liu, Zhangyang Wang, Shiwei Liu
2025	J	`jnl`	SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training. CoRR Tianjin Huang, Ziquan Zhu, Gaojie Jin, Lu Liu, Zhangyang Wang, Shiwei Liu
2025	J	`jnl`	SoS1: O1 and R1-Like Reasoning LLMs are Sum-of-Square Solvers. CoRR Kechen Li, Wenqi Zhu, Coralia Cartis, Tianbo Ji, Shiwei Liu
2025	J	`jnl`	Stable-SPAM: How to Train in 4-Bit More Stably than 16-Bit Adam. CoRR Tianjin Huang, Haotian Hu, Zhenyu Zhang, Gaojie Jin, Xiang Li, Li Shen, Tianlong Chen, Lu Liu, Qingsong Wen, Zhangyang Wang, Shiwei Liu
2025	J	`jnl`	The Curse of Depth in Large Language Models. CoRR Wenfang Sun, Xinyuan Song, Pengxiang Li, Lu Yin, Yefeng Zheng, Shiwei Liu
2025	A*	`conf`	Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective. AAAI Can Jin, Tianjin Huang, Yihua Zhang, Mykola Pechenizkiy, Sijia Liu, Shiwei Liu, Tianlong Chen
2025	J	`jnl`	When Fewer Layers Break More Chains: Layer Pruning Harms Test-Time Scaling in LLMs. CoRR Keyu Wang, Tian Lyu, Guinan Su, Jonas Geiping, Lu Yin, Marco Canini, Shiwei Liu
2024	J	`jnl`	(PASS) Visual Prompt Locates Good Structure Sparsity through a Recurrent HyperNetwork. CoRR Tianjin Huang, Meng Fang, Li Shen, Fan Liu, Yulong Pei, Mykola Pechenizkiy, Shiwei Liu, Tianlong Chen
2024	A*	`conf`	AdaMerging: Adaptive Model Merging for Multi-Task Learning. ICLR Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, Dacheng Tao
2024	A*	`conf`	Advancing Dynamic Sparse Training by Exploring Optimization Opportunities. ICML Jie Ji, Gen Li, Lu Yin, Minghai Qin, Geng Yuan, Linke Guo, Shiwei Liu, Xiaolong Ma
2024	A*	`conf`	AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models. NeurIPS Haiquan Lu, Yefan Zhou, Shiwei Liu, Zhangyang Wang, Michael W. Mahoney, Yaoqing Yang
2024	J	`jnl`	AlphaPruning: Using Heavy-Tailed Self Regularization Theory for Improved Layer-wise Pruning of Large Language Models. CoRR Haiquan Lu, Yefan Zhou, Shiwei Liu, Zhangyang Wang, Michael W. Mahoney, Yaoqing Yang
2024	A*	`conf`	CaM: Cache Merging for Memory-efficient LLMs Inference. ICML Yuxin Zhang, Yuxuan Du, Gen Luo, Yunshan Zhong, Zhenyu Zhang, Shiwei Liu, Rongrong Ji
2024	J	`jnl`	Composable Interventions for Language Models. CoRR Arinbjörn Kolbeinsson, Kyle O'Brien, Tianjin Huang, Shanghua Gao, Shiwei Liu, Jonathan Richard Schwarz, Anurag Vaidya, Faisal Mahmood, Marinka Zitnik, Tianlong Chen, Thomas Hartvigsen
2024	J	`jnl`	Condense, Don't Just Prune: Enhancing Efficiency and Performance in MoE Layer Pruning. CoRR Mingyu Cao, Gen Li, Jie Ji, Jiaqi Zhang, Xiaolong Ma, Shiwei Liu, Lu Yin
2024	A	`conf`	Dynamic Data Pruning for Automatic Speech Recognition. INTERSPEECH Qiao Xiao, Pingchuan Ma, Adriana Fernandez-Lopez, Boqian Wu, Lu Yin, Stavros Petridis, Mykola Pechenizkiy, Maja Pantic, Decebal Constantin Mocanu, Shiwei Liu
2024	J	`jnl`	Dynamic Data Pruning for Automatic Speech Recognition. CoRR Qiao Xiao, Pingchuan Ma, Adriana Fernandez-Lopez, Boqian Wu, Lu Yin, Stavros Petridis, Mykola Pechenizkiy, Maja Pantic, Decebal Constantin Mocanu, Shiwei Liu
2024	A*	`conf`	Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs. ICLR Yuxin Zhang, Lirui Zhao, Mingbao Lin, Yunyun Sun, Yiwu Yao, Xingjia Han, Jared Tanner, Shiwei Liu, Rongrong Ji
2024	A*	`conf`	E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation. NeurIPS Boqian Wu, Qiao Xiao, Shiwei Liu, Lu Yin, Mykola Pechenizkiy, Decebal Constantin Mocanu, Maurice van Keulen, Elena Mocanu
2024	A*	`conf`	FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping. EMNLP Ajay Jaiswal, Bodun Hu, Lu Yin, Yeonju Ro, Tianlong Chen, Shiwei Liu, Aditya Akella
2024	J	`jnl`	FFN-SkipLLM: A Hidden Gem for Autoregressive Decoding with Adaptive Feed Forward Skipping. CoRR Ajay Jaiswal, Bodun Hu, Lu Yin, Yeonju Ro, Shiwei Liu, Tianlong Chen, Aditya Akella
2024	A*	`conf`	Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding. NeurIPS Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang
2024	J	`jnl`	Found in the Middle: How Language Models Use Long Contexts Better via Plug-and-Play Positional Encoding. CoRR Zhenyu Zhang, Runjin Chen, Shiwei Liu, Zhewei Yao, Olatunji Ruwase, Beidi Chen, Xiaoxia Wu, Zhangyang Wang
2024	J	`jnl`	From GaLore to WeLore: How Low-Rank Weights Non-uniformly Emerge from Low-Rank Gradients. CoRR Ajay Jaiswal, Lu Yin, Zhenyu Zhang, Shiwei Liu, Jiawei Zhao, Yuandong Tian, Zhangyang Wang
2024	J	`jnl`	Full-Rank No More: Low-Rank Weight Training for Modern Speech Recognition Models. CoRR Adriana Fernandez-Lopez, Shiwei Liu, Lu Yin, Stavros Petridis, Maja Pantic
2024	—	`conf`	HRBP: Hardware-friendly Regrouping towards Block-based Pruning for Sparse CNN Training. CPAL Haoyu Ma, Chengming Zhang, Lizhi Xiang, Xiaolong Ma, Geng Yuan, Wenkai Zhang, Shiwei Liu, Tianlong Chen, Dingwen Tao, Yanzhi Wang, Zhangyang Wang, Xiaohui Xie
2024	A*	`conf`	Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning. EMNLP Abhinav Bandari, Lu Yin, Cheng-Yu Hsieh, Ajay Jaiswal, Tianlong Chen, Li Shen, Ranjay Krishna, Shiwei Liu
2024	J	`jnl`	Is C4 Dataset Optimal for Pruning? An Investigation of Calibration Data for LLM Pruning. CoRR Abhinav Bandari, Lu Yin, Cheng-Yu Hsieh, Ajay Kumar Jaiswal, Tianlong Chen, Li Shen, Ranjay Krishna, Shiwei Liu
2024	A*	`conf`	Junk DNA Hypothesis: Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMs. ICML Lu Yin, Ajay Kumar Jaiswal, Shiwei Liu, Souvik Kundu, Zhangyang Wang
2024	A	`conf`	MSRS: Training Multimodal Speech Recognition Models from Scratch with Sparse Mask Optimization. INTERSPEECH Adriana Fernandez-Lopez, Honglie Chen, Pingchuan Ma, Lu Yin, Qiao Xiao, Stavros Petridis, Shiwei Liu, Maja Pantic
2024	J	`jnl`	MSRS: Training Multimodal Speech Recognition Models from Scratch with Sparse Mask Optimization. CoRR Adriana Fernandez-Lopez, Honglie Chen, Pingchuan Ma, Lu Yin, Qiao Xiao, Stavros Petridis, Shiwei Liu, Maja Pantic
2024	J	`jnl`	Mix-LN: Unleashing the Power of Deeper Layers by Combining Pre-LN and Post-LN. CoRR Pengxiang Li, Lu Yin, Shiwei Liu
2024	A*	`conf`	NeurRev: Train Better Sparse Neural Network Practically via Neuron Revitalization. ICLR Gen Li, Lu Yin, Jie Ji, Wei Niu, Minghai Qin, Bin Ren, Linke Guo, Shiwei Liu, Xiaolong Ma
2024	A*	`conf`	Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity. ICML Lu Yin, You Wu, Zhenyu Zhang, Cheng-Yu Hsieh, Yaqing Wang, Yiling Jia, Gen Li, Ajay Kumar Jaiswal, Mykola Pechenizkiy, Yi Liang, Michael Bendersky, Zhangyang Wang, Shiwei Liu
2024	J	`jnl`	OwLore: Outlier-weighed Layerwise Sampled Low-Rank Projection for Memory-Efficient LLM Fine-tuning. CoRR Pengxiang Li, Lu Yin, Xiaowei Gao, Shiwei Liu
2024	J	`jnl`	Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradients. CoRR Zhenyu Zhang, Ajay Jaiswal, Lu Yin, Shiwei Liu, Jiawei Zhao, Yuandong Tian, Zhangyang Wang
2024	—	`conf`	Q-Hitter: A Better Token Oracle for Efficient LLM Inference via Sparse-Quantized KV Cache. MLSys Zhenyu Zhang, Shiwei Liu, Runjin Chen, Bhavya Kailkhura, Beidi Chen, Atlas Wang
2024	A*	`conf`	Sparse Cocktail: Every Sparse Pattern Every Sparse Ratio All At Once. ICML Zhangheng Li, Shiwei Liu, Tianlong Chen, Ajay Kumar Jaiswal, Zhenyu Zhang, Dilin Wang, Raghuraman Krishnamoorthi, Shiyu Chang, Zhangyang Wang
2024	B	`conf`	Sparse Sounds: Exploring Low-Dimensionality in Music Generation Model. IEEE Big Data Shu Wang, Shiwei Liu
2024	J	`jnl`	Towards Efficient Pareto Set Approximation via Mixture of Experts Based Model Fusion. CoRR Anke Tang, Li Shen, Yong Luo, Shiwei Liu, Han Hu, Bo Du
2023	J	`jnl`	AdaMerging: Adaptive Model Merging for Multi-Task Learning. CoRR Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, Dacheng Tao
2023	A*	`conf`	Are Large Kernels Better Teachers than Transformers for ConvNets? ICML Tianjin Huang, Lu Yin, Zhenyu Zhang, Li Shen, Meng Fang, Mykola Pechenizkiy, Zhangyang Wang, Shiwei Liu
2023	J	`jnl`	Are Large Kernels Better Teachers than Transformers for ConvNets? CoRR Tianjin Huang, Lu Yin, Zhenyu Zhang, Li Shen, Meng Fang, Mykola Pechenizkiy, Zhangyang Wang, Shiwei Liu
2023	A*	`conf`	Data Augmented Flatness-aware Gradient Projection for Continual Learning. ICCV Enneng Yang, Li Shen, Zhenyi Wang, Shiwei Liu, Guibing Guo, Xingwei Wang
2023	J	`jnl`	Don't Be So Dense: Sparse-to-Sparse GAN Training Without Sacrificing Performance. Int. J. Comput. Vis. Shiwei Liu, Yuesong Tian, Tianlong Chen, Li Shen
2023	A*	`conf`	Don't just prune by magnitude! Your mask topology is a secret weapon. NeurIPS Duc Hoang, Souvik Kundu, Shiwei Liu, Zhangyang Wang
2023	J	`jnl`	Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs. CoRR Yuxin Zhang, Lirui Zhao, Mingbao Lin, Yunyun Sun, Yiwu Yao, Xingjia Han, Jared Tanner, Shiwei Liu, Rongrong Ji
2023	A*	`conf`	Dynamic Sparsity Is Channel-Level Sparsity Learner. NeurIPS Lu Yin, Gen Li, Meng Fang, Li Shen, Tianjin Huang, Zhangyang Wang, Vlado Menkovski, Xiaolong Ma, Mykola Pechenizkiy, Shiwei Liu
2023	J	`jnl`	Dynamic Sparsity Is Channel-Level Sparsity Learner. CoRR Lu Yin, Gen Li, Meng Fang, Li Shen, Tianjin Huang, Zhangyang Wang, Vlado Menkovski, Xiaolong Ma, Mykola Pechenizkiy, Shiwei Liu
2023	J	`jnl`	E2ENet: Dynamic Sparse Feature Fusion for Accurate and Efficient 3D Medical Image Segmentation. CoRR Boqian Wu, Qiao Xiao, Shiwei Liu, Lu Yin, Mykola Pechenizkiy, Decebal Constantin Mocanu, Maurice van Keulen, Elena Mocanu
2023	—	`conf`	Enhancing Adversarial Training via Reweighting Optimization Trajectory. ECML/PKDD (1) Tianjin Huang, Shiwei Liu, Tianlong Chen, Meng Fang, Li Shen, Vlado Menkovski, Lu Yin, Yulong Pei, Mykola Pechenizkiy
2023	J	`jnl`	Enhancing Adversarial Training via Reweighting Optimization Trajectory. CoRR Tianjin Huang, Shiwei Liu, Tianlong Chen, Meng Fang, Li Shen, Vlado Menkovski, Lu Yin, Yulong Pei, Mykola Pechenizkiy
2023	A*	`conf`	Graph Ladling: Shockingly Simple Parallel GNN Training without Intermediate Communication. ICML Ajay Kumar Jaiswal, Shiwei Liu, Tianlong Chen, Ying Ding, Zhangyang Wang
2023	J	`jnl`	Graph Ladling: Shockingly Simple Parallel GNN Training without Intermediate Communication. CoRR Ajay Jaiswal, Shiwei Liu, Tianlong Chen, Ying Ding, Zhangyang Wang
2023	A*	`conf`	Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models. ICML Ajay Kumar Jaiswal, Shiwei Liu, Tianlong Chen, Ying Ding, Zhangyang Wang
2023	J	`jnl`	Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models. CoRR Ajay Jaiswal, Shiwei Liu, Tianlong Chen, Ying Ding, Zhangyang Wang
2023	J	`jnl`	Junk DNA Hypothesis: A Task-Centric Angle of LLM Pre-trained Weights through Sparsity. CoRR Lu Yin, Shiwei Liu, Ajay Jaiswal, Souvik Kundu, Zhangyang Wang
2023	A*	`conf`	Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost. AAAI Lu Yin, Shiwei Liu, Meng Fang, Tianjin Huang, Vlado Menkovski, Mykola Pechenizkiy
2023	—	`conf`	Many-Task Federated Learning: A New Problem Setting and A Simple Baseline. CVPR Workshops Ruisi Cai, Xiaohan Chen, Shiwei Liu, Jayanth Srinivasa, Myungjin Lee, Ramana Kompella, Zhangyang Wang
2023	A*	`conf`	More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity. ICLR Shiwei Liu, Tianlong Chen, Xiaohan Chen, Xuxi Chen, Qiao Xiao, Boqian Wu, Tommi Kärkkäinen, Mykola Pechenizkiy, Decebal Constantin Mocanu, Zhangyang Wang
2023	J	`jnl`	Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity. CoRR Lu Yin, You Wu, Zhenyu Zhang, Cheng-Yu Hsieh, Yaqing Wang, Yiling Jia, Mykola Pechenizkiy, Yi Liang, Zhangyang Wang, Shiwei Liu
2023	—	`conf`	REST: Enhancing Group Robustness in DNNs Through Reweighted Sparse Training. ECML/PKDD (2) Jiaxu Zhao, Lu Yin, Shiwei Liu, Meng Fang, Mykola Pechenizkiy
2023	J	`jnl`	REST: Enhancing Group Robustness in DNNs through Reweighted Sparse Training. CoRR Jiaxu Zhao, Lu Yin, Shiwei Liu, Meng Fang, Mykola Pechenizkiy
2023	A*	`conf`	Revisiting Pruning at Initialization Through the Lens of Ramanujan Graph. ICLR Duc N. M. Hoang, Shiwei Liu, Radu Marculescu, Zhangyang Wang
2023	A*	`conf`	Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers. ICLR Tianlong Chen, Zhenyu Zhang, Ajay Kumar Jaiswal, Shiwei Liu, Zhangyang Wang
2023	J	`jnl`	Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformers. CoRR Tianlong Chen, Zhenyu Zhang, Ajay Jaiswal, Shiwei Liu, Zhangyang Wang
2023	A*	`conf`	Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! ICLR Shiwei Liu, Tianlong Chen, Zhenyu Zhang, Xuxi Chen, Tianjin Huang, Ajay Kumar Jaiswal, Zhangyang Wang
2023	J	`jnl`	Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! CoRR Shiwei Liu, Tianlong Chen, Zhenyu Zhang, Xuxi Chen, Tianjin Huang, Ajay Jaiswal, Zhangyang Wang
2023	J	`jnl`	Supervised Feature Selection with Neuron Evolution in Sparse Neural Networks. CoRR Zahra Atashgahi, Xuhao Zhang, Neil Kichler, Shiwei Liu, Lu Yin, Mykola Pechenizkiy, Raymond N. J. Veldhuis, Decebal Constantin Mocanu
2023	J	`jnl`	Supervised Feature Selection with Neuron Evolution in Sparse Neural Networks. Trans. Mach. Learn. Res. Zahra Atashgahi, Xuhao Zhang, Neil Kichler, Shiwei Liu, Lu Yin, Mykola Pechenizkiy, Raymond N. J. Veldhuis, Decebal Constantin Mocanu
2023	J	`jnl`	Ten Lessons We Have Learned in the New "Sparseland": A Short Handbook for Sparse Neural Network Researchers. CoRR Shiwei Liu, Zhangyang Wang
2023	J	`jnl`	The Counterattack of CNNs in Self-Supervised Learning: Larger Kernel Size might be All You Need. CoRR Tianjin Huang, Tianlong Chen, Zhangyang Wang, Shiwei Liu
2023	A*	`conf`	The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter. NeurIPS Ajay Jaiswal, Shiwei Liu, Tianlong Chen, Zhangyang Wang
2023	J	`jnl`	The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter. CoRR Ajay Jaiswal, Shiwei Liu, Tianlong Chen, Zhangyang Wang
2023	A*	`conf`	Towards Data-Agnostic Pruning At Initialization: What Makes a Good Sparse Mask? NeurIPS Hoang Pham, The-Anh Ta, Shiwei Liu, Lichuan Xiang, Dung Le, Hongkai Wen, Long Tran-Thanh
2023	J	`jnl`	Visual Prompting Upgrades Neural Network Sparsification: A Data-Model Perspective. CoRR Can Jin, Tianjin Huang, Yihua Zhang, Mykola Pechenizkiy, Sijia Liu, Shiwei Liu, Tianlong Chen
2022	J	`jnl`	A brain-inspired algorithm for training highly sparse neural networks. Mach. Learn. Zahra Atashgahi, Joost Pieterse, Shiwei Liu, Decebal Constantin Mocanu, Raymond N. J. Veldhuis, Mykola Pechenizkiy
2022	J	`jnl`	Achieving Personalized Federated Learning with Sparse Local Models. CoRR Tiansheng Huang, Shiwei Liu, Li Shen, Fengxiang He, Weiwei Lin, Dacheng Tao
2022	A*	`conf`	Deep Ensembling with No Overhead for either Training or Testing: The All-Round Blessings of Dynamic Sparsity. ICLR Shiwei Liu, Tianlong Chen, Zahra Atashgahi, Xiaohan Chen, Ghada Sokar, Elena Mocanu, Mykola Pechenizkiy, Zhangyang Wang, Decebal Constantin Mocanu
2022	J	`jnl`	Don't Be So Dense: Sparse-to-Sparse GAN Training Without Sacrificing Performance. CoRR Shiwei Liu, Yuesong Tian, Tianlong Chen, Li Shen
2022	A*	`conf`	Dynamic Sparse Network for Time Series Classification: Learning What to "See". NeurIPS Qiao Xiao, Boqian Wu, Yu Zhang, Shiwei Liu, Mykola Pechenizkiy, Elena Mocanu, Decebal Constantin Mocanu
2022	J	`jnl`	Dynamic Sparse Network for Time Series Classification: Learning What to "see". CoRR Qiao Xiao, Boqian Wu, Yu Zhang, Shiwei Liu, Mykola Pechenizkiy, Elena Mocanu, Decebal Constantin Mocanu
2022	J	`jnl`	Lottery Pools: Winning More by Interpolating Tickets without Increasing Training or Inference Cost. CoRR Lu Yin, Shiwei Liu, Meng Fang, Tianjin Huang, Vlado Menkovski, Mykola Pechenizkiy
2022	J	`jnl`	More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity. CoRR Shiwei Liu, Tianlong Chen, Xiaohan Chen, Xuxi Chen, Qiao Xiao, Boqian Wu, Mykola Pechenizkiy, Decebal Constantin Mocanu, Zhangyang Wang
2022	J	`jnl`	Superposing Many Tickets into One: A Performance Booster for Sparse Neural Network Training. CoRR Lu Yin, Vlado Menkovski, Meng Fang, Tianjin Huang, Yulong Pei, Mykola Pechenizkiy, Decebal Constantin Mocanu, Shiwei Liu
2022	A*	`conf`	The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training. ICLR Shiwei Liu, Tianlong Chen, Xiaohan Chen, Li Shen, Decebal Constantin Mocanu, Zhangyang Wang, Mykola Pechenizkiy
2022	J	`jnl`	The Unreasonable Effectiveness of Random Pruning: Return of the Most Naive Baseline for Sparse Training. CoRR Shiwei Liu, Tianlong Chen, Xiaohan Chen, Li Shen, Decebal Constantin Mocanu, Zhangyang Wang, Mykola Pechenizkiy
2022	—	`conf`	You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets. LoG Tianjin Huang, Tianlong Chen, Meng Fang, Vlado Menkovski, Jiaxu Zhao, Lu Yin, Yulong Pei, Decebal Constantin Mocanu, Zhangyang Wang, Mykola Pechenizkiy, Shiwei Liu
2022	J	`jnl`	You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets. CoRR Tianjin Huang, Tianlong Chen, Meng Fang, Vlado Menkovski, Jiaxu Zhao, Lu Yin, Yulong Pei, Decebal Constantin Mocanu, Zhangyang Wang, Mykola Pechenizkiy, Shiwei Liu
2021	A*	`conf`	Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training. ICML Shiwei Liu, Lu Yin, Decebal Constantin Mocanu, Mykola Pechenizkiy
2021	J	`jnl`	Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training. CoRR Shiwei Liu, Lu Yin, Decebal Constantin Mocanu, Mykola Pechenizkiy
2021	J	`jnl`	Efficient and effective training of sparse recurrent neural networks. Neural Comput. Appl. Shiwei Liu, Iftitahu Ni'mah, Vlado Menkovski, Decebal Constantin Mocanu, Mykola Pechenizkiy
2021	J	`jnl`	FreeTickets: Accurate, Robust and Efficient Deep Ensemble by Training with Dynamic Sparsity. CoRR Shiwei Liu, Tianlong Chen, Zahra Atashgahi, Xiaohan Chen, Ghada Sokar, Elena Mocanu, Mykola Pechenizkiy, Zhangyang Wang, Decebal Constantin Mocanu
2021	C	`conf`	Hierarchical Semantic Segmentation using Psychometric Learning. ACML Lu Yin, Vlado Menkovski, Shiwei Liu, Mykola Pechenizkiy
2021	J	`jnl`	Hierarchical Semantic Segmentation using Psychometric Learning. CoRR Lu Yin, Vlado Menkovski, Shiwei Liu, Mykola Pechenizkiy
2021	A*	`conf`	Selfish Sparse RNN Training. ICML Shiwei Liu, Decebal Constantin Mocanu, Yulong Pei, Mykola Pechenizkiy
2021	J	`jnl`	Selfish Sparse RNN Training. CoRR Shiwei Liu, Decebal Constantin Mocanu, Yulong Pei, Mykola Pechenizkiy
2021	A*	`conf`	Sparse Training via Boosting Pruning Plasticity with Neuroregeneration. NeurIPS Shiwei Liu, Tianlong Chen, Xiaohan Chen, Zahra Atashgahi, Lu Yin, Huanyu Kou, Li Shen, Mykola Pechenizkiy, Zhangyang Wang, Decebal Constantin Mocanu
2021	J	`jnl`	Sparse Training via Boosting Pruning Plasticity with Neuroregeneration. CoRR Shiwei Liu, Tianlong Chen, Xiaohan Chen, Zahra Atashgahi, Lu Yin, Huanyu Kou, Li Shen, Mykola Pechenizkiy, Zhangyang Wang, Decebal Constantin Mocanu
2021	J	`jnl`	Sparse evolutionary deep learning with over one million artificial neurons on commodity hardware. Neural Comput. Appl. Shiwei Liu, Decebal Constantin Mocanu, Amarsagar Reddy Ramapuram Matavalam, Yulong Pei, Mykola Pechenizkiy
2020	A*	`conf`	Learning Sparse Neural Networks for Better Generalization. IJCAI Shiwei Liu
2020	—	`conf`	Network Performance Optimization with Real Time Traffic Prediction in Data Center Network. ECOC Fulong Yan, Shiwei Liu, Nicola Calabretta
2020	J	`jnl`	Topological Insights in Sparse Neural Networks. CoRR Shiwei Liu, Tim van der Lee, Anil Yaman, Zahra Atashgahi, Davide Ferraro, Ghada Sokar, Mykola Pechenizkiy, Decebal Constantin Mocanu
2020	—	`conf`	Topological Insights into Sparse Neural Networks. ECML/PKDD (3) Shiwei Liu, Tim van der Lee, Anil Yaman, Zahra Atashgahi, Davide Ferraro, Ghada Sokar, Mykola Pechenizkiy, Decebal Constantin Mocanu
2019	J	`jnl`	Intrinsically Sparse Long Short-Term Memory Networks. CoRR Shiwei Liu, Decebal Constantin Mocanu, Mykola Pechenizkiy
2019	J	`jnl`	On improving deep learning generalization with adaptive sparse connectivity. CoRR Shiwei Liu, Decebal Constantin Mocanu, Mykola Pechenizkiy
2019	J	`jnl`	Sparse evolutionary Deep Learning with over one million artificial neurons on commodity hardware. CoRR Shiwei Liu, Decebal Constantin Mocanu, Amarsagar Reddy Ramapuram Matavalam, Yulong Pei, Mykola Pechenizkiy