Xiaozhe Ren

39 papers A* 8A 1Misc 3Journal 22Unranked 5
YearRankTypeTitle / Venue / Authors
2026 A* conf
BitDP: Ultra-low-bit Communication for Data Parallelism in LLM Training.
AAAI
Xiaozhe Ren, Qiong Luo
2025 J jnl
A Survey of Reasoning with Foundation Models: Concepts, Methodologies, and Outlook.
ACM Comput. Surv.
Jiankai Sun, Chuanyang Zheng, Enze Xie, Zhengying Liu, Ruihang Chu, Jianing Qiu, Jiaqi Xu, Mingyu Ding, Hongyang Li, Mengzhe Geng, Yue Wu, Wenhai Wang, Junsong Chen, Zhangyue Yin, Xiaozhe Ren, Jie Fu, Junxian He, Wu Yuan, Qi Liu, Xihui Liu, Yu Li, Hao Dong, Yu Cheng, Ming Zhang, Pheng-Ann Heng, Jifeng Dai, Ping Luo, Jingdong Wang, Ji-Rong Wen, Xipeng Qiu, Yike Guo, Hui Xiong, Qun Liu, Zhenguo Li
2025 — conf
DAPE V2: Process Attention Score as Feature Map for Length Extrapolation.
ACL (1)
Chuanyang Zheng, Yihang Gao, Han Shi, Jing Xiong, Jiankai Sun, Jingyao Li, Minbin Huang, Xiaozhe Ren, Michael Ng, Xin Jiang, Zhenguo Li, Yu Li
2025 J jnl
Pangu DeepDiver: Adaptive Search Intensity Scaling via Open-Web Reinforcement Learning.
CoRR
Wenxuan Shi, Haochen Tan, Chuqiao Kuang, Xiaoguang Li, Xiaozhe Ren, Chen Zhang, Hanting Chen, Yasheng Wang, Lifeng Shang, Fisher Yu, Yunhe Wang
2025 A* conf
Self-Adjust Softmax.
EMNLP
Chuanyang Zheng, Yihang Gao, Guoxuan Chen, Han Shi, Jing Xiong, Xiaozhe Ren, Chao Huang, Zhenguo Li, Yu Li
2025 J jnl
Self-Adjust Softmax.
CoRR
Chuanyang Zheng, Yihang Gao, Guoxuan Chen, Han Shi, Jing Xiong, Xiaozhe Ren, Chao Huang, Xin Jiang, Zhenguo Li, Yu Li
2025 A* conf
SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator.
ICML
Guoxuan Chen, Han Shi, Jiawei Li, Yihang Gao, Xiaozhe Ren, Yimeng Chen, Xin Jiang, Zhenguo Li, Weiyang Liu, Chao Huang
2025 Misc conf
TaskSense: A Translation-like Approach for Tasking Heterogeneous Sensor Systems with LLMs.
SenSys
Kaiwei Liu, Bufang Yang, Lilin Xu, Yunqi Guo, Guoliang Xing, Xian Shuai, Xiaozhe Ren, Xin Jiang, Zhenyu Yan
2024 J jnl
CAPE: Context-Adaptive Positional Encoding for Length Extrapolation.
CoRR
Chuanyang Zheng, Yihang Gao, Han Shi, Minbin Huang, Jingyao Li, Jing Xiong, Xiaozhe Ren, Michael K. Ng, Xin Jiang, Zhenguo Li, Yu Li
2024 J jnl
DAPE V2: Process Attention Score as Feature Map for Length Extrapolation.
CoRR
Chuanyang Zheng, Yihang Gao, Han Shi, Jing Xiong, Jiankai Sun, Jingyao Li, Minbin Huang, Xiaozhe Ren, Michael K. Ng, Xin Jiang, Zhenguo Li, Yu Li
2024 A* conf
DAPE: Data-Adaptive Positional Encoding for Length Extrapolation.
NeurIPS
Chuanyang Zheng, Yihang Gao, Han Shi, Minbin Huang, Jingyao Li, Jing Xiong, Xiaozhe Ren, Michael K. Ng, Xin Jiang, Zhenguo Li, Yu Li
2024 — conf
PIXART-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation.
ECCV (32)
Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, Zhenguo Li
2024 J jnl
PixArt-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation.
CoRR
Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, Zhenguo Li
2024 Misc conf
Poster Abstract: Tasking Heterogeneous Sensor Systems with LLMs.
SenSys
Kaiwei Liu, Bufang Yang, Lilin Xu, Yunqi Guo, Neiwen Ling, Zhihe Zhao, Guoliang Xing, Xian Shuai, Xiaozhe Ren, Xin Jiang, Zhenyu Yan
2024 J jnl
Scaling Law for Language Models Training Considering Batch Size.
CoRR
Xian Shuai, Yiding Wang, Yimeng Wu, Xin Jiang, Xiaozhe Ren
2024 A conf
ScheMoE: An Extensible Mixture-of-Experts Distributed Training System with Tasks Scheduling.
EuroSys
Shaohuai Shi, Xinglin Pan, Qiang Wang, Chengjian Liu, Xiaozhe Ren, Zhongzhe Hu, Yu Yang, Bo Li, Xiaowen Chu
2024 J jnl
SepLLM: Accelerate Large Language Models by Compressing One Segment into One Separator.
CoRR
Guoxuan Chen, Han Shi, Jiawei Li, Yihang Gao, Xiaozhe Ren, Yimeng Chen, Xin Jiang, Zhenguo Li, Weiyang Liu, Chao Huang
2023 A* conf
A Study on Transformer Configuration and Training Objective.
ICML
Fuzhao Xue, Jianghai Chen, Aixin Sun, Xiaozhe Ren, Zangwei Zheng, Xiaoxin He, Yongming Chen, Xin Jiang, Yang You
2023 J jnl
A Survey of Reasoning with Foundation Models.
CoRR
Jiankai Sun, Chuanyang Zheng, Enze Xie, Zhengying Liu, Ruihang Chu, Jianing Qiu, Jiaqi Xu, Mingyu Ding, Hongyang Li, Mengzhe Geng, Yue Wu, Wenhai Wang, Junsong Chen, Zhangyue Yin, Xiaozhe Ren, Jie Fu, Junxian He, Wu Yuan, Qi Liu, Xihui Liu, Yu Li, Hao Dong, Yu Cheng, Ming Zhang, Pheng-Ann Heng, Jifeng Dai, Ping Luo, Jingdong Wang, Ji-Rong Wen, Xipeng Qiu, Yike Guo, Hui Xiong, Qun Liu, Zhenguo Li
2023 — conf
CAME: Confidence-guided Adaptive Memory Efficient Optimization.
ACL (1)
Yang Luo, Xiaozhe Ren, Zangwei Zheng, Zhuo Jiang, Xin Jiang, Yang You
2023 J jnl
CAME: Confidence-guided Adaptive Memory Efficient Optimization.
CoRR
Yang Luo, Xiaozhe Ren, Zangwei Zheng, Zhuo Jiang, Xin Jiang, Yang You
2023 Misc conf
EdgeFM: Leveraging Foundation Model for Open-set Learning on the Edge.
SenSys
Bufang Yang, Lixing He, Neiwen Ling, Zhenyu Yan, Guoliang Xing, Xian Shuai, Xiaozhe Ren, Xin Jiang
2023 J jnl
EdgeFM: Leveraging Foundation Model for Open-set Learning on the Edge.
CoRR
Bufang Yang, Lixing He, Neiwen Ling, Zhenyu Yan, Guoliang Xing, Xian Shuai, Xiaozhe Ren, Xin Jiang
2023 — conf
One Student Knows All Experts Know: From Sparse to Dense.
Tiny Papers @ ICLR
Fuzhao Xue, Xiaoxin He, Xiaozhe Ren, Yuxuan Lou, Yang You
2023 J jnl
PanGu-Σ: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computing.
CoRR
Xiaozhe Ren, Pingyi Zhou, Xinfan Meng, Xinjing Huang, Yadao Wang, Weichao Wang, Pengfei Li, Xiaoda Zhang, Alexander Podolskiy, Grigory Arshinov, Andrey Bout, Irina Piontkovskaya, Jiansheng Wei, Xin Jiang, Teng Su, Qun Liu, Jun Yao
2023 A* conf
Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline.
NeurIPS
Zangwei Zheng, Xiaozhe Ren, Fuzhao Xue, Yang Luo, Xin Jiang, Yang You
2023 J jnl
Response Length Perception and Sequence Scheduling: An LLM-Empowered LLM Inference Pipeline.
CoRR
Zangwei Zheng, Xiaozhe Ren, Fuzhao Xue, Yang Luo, Xin Jiang, Yang You
2022 A* conf
AutoBERT-Zero: Evolving BERT Backbone from Scratch.
AAAI
Jiahui Gao, Hang Xu, Han Shi, Xiaozhe Ren, Philip L. H. Yu, Xiaodan Liang, Xin Jiang, Zhenguo Li
2022 J jnl
Deeper vs Wider: A Revisit of Transformer Configuration.
CoRR
Fuzhao Xue, Jianghai Chen, Aixin Sun, Xiaozhe Ren, Zangwei Zheng, Xiaoxin He, Xin Jiang, Yang You
2022 J jnl
One Student Knows All Experts Know: From Sparse to Dense.
CoRR
Fuzhao Xue, Xiaoxin He, Xiaozhe Ren, Yuxuan Lou, Yang You
2021 J jnl
AutoBERT-Zero: Evolving BERT Backbone from Scratch.
CoRR
Jiahui Gao, Hang Xu, Han Shi, Xiaozhe Ren, Philip L. H. Yu, Xiaodan Liang, Xin Jiang, Zhenguo Li
2021 — conf
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation.
EMNLP (Findings)
Chenhe Dong, Guangrun Wang, Hang Xu, Jiefeng Peng, Xiaozhe Ren, Xiaodan Liang
2021 J jnl
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation.
CoRR
Chenhe Dong, Guangrun Wang, Hang Xu, Jiefeng Peng, Xiaozhe Ren, Xiaodan Liang
2021 J jnl
Large-Scale Deep Learning Optimizations: A Comprehensive Survey.
CoRR
Xiaoxin He, Fuzhao Xue, Xiaozhe Ren, Yang You
2021 J jnl
NumGPT: Improving Numeracy Ability of Generative Pre-trained Models.
CoRR
Zhihua Jin, Xin Jiang, Xingbo Wang, Qun Liu, Yong Wang, Xiaozhe Ren, Huamin Qu
2021 J jnl
PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation.
CoRR
Wei Zeng, Xiaozhe Ren, Teng Su, Hui Wang, Yi Liao, Zhiwei Wang, Xin Jiang, ZhenZhang Yang, Kaisheng Wang, Xiaoda Zhang, Chen Li, Ziyan Gong, Yifan Yao, Xinjing Huang, Jun Wang, Jianfeng Yu, Qi Guo, Yue Yu, Yan Zhang, Jin Wang, Hengtao Tao, Dasen Yan, Zexuan Yi, Fang Peng, Fangqing Jiang, Han Zhang, Lingfeng Deng, Yehong Zhang, Zhe Lin, Chao Zhang, Shaojie Zhang, Mingyue Guo, Shanzhi Gu, Gaojun Fan, Yaowei Wang, Xuefeng Jin, Qun Liu, Yonghong Tian
2021 A* conf
SparseBERT: Rethinking the Importance Analysis in Self-attention.
ICML
Han Shi, Jiahui Gao, Xiaozhe Ren, Hang Xu, Xiaodan Liang, Zhenguo Li, James Tin-Yau Kwok
2021 J jnl
SparseBERT: Rethinking the Importance Analysis in Self-attention.
CoRR
Han Shi, Jiahui Gao, Xiaozhe Ren, Hang Xu, Xiaodan Liang, Zhenguo Li, James T. Kwok
2019 J jnl
NEZHA: Neural Contextualized Representation for Chinese Language Understanding.
CoRR
Junqiu Wei, Xiaozhe Ren, Xiaoguang Li, Wenyong Huang, Yi Liao, Yasheng Wang, Jiashu Lin, Xin Jiang, Xiao Chen, Qun Liu