Kaiyue Wen - RankMe

36 papers A* 10Journal 23Unranked 3

Year	Rank	Type	Title / Venue / Authors
2026	J	`jnl`	A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training. CoRR Zihan Qiu, Zeyu Huang, Kaiyue Wen, Peng Jin, Bo Zheng, Yuxin Zhou, Haofeng Huang, Zekun Wang, Xiao Li, Huaqing Zhang, Yang Xu, Haoran Lian, Siqi Zhang, Rui Men, Jianwei Zhang, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin
2026	J	`jnl`	Configuration-to-Performance Scaling Law with Neural Ansatz. CoRR Huaqing Zhang, Kaiyue Wen, Tengyu Ma
2026	J	`jnl`	Divide-and-Conquer CoT: RL for Reducing Latency via Parallel Reasoning. CoRR Arvind V. Mahankali, Kaiyue Wen, Tengyu Ma
2026	J	`jnl`	Enhancing underwater images: a dual-constraint latent diffusion approach with multi-view contrastive learning. Expert Syst. Appl. Peng Liu, Kaiyue Wen, Guoyu Wang, Yong-Chao Li, Junyu Dong
2025	—	`conf`	Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models. ACL (1) Zihan Qiu, Zeyu Huang, Bo Zheng, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin
2025	J	`jnl`	Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models. CoRR Zihan Qiu, Zeyu Huang, Bo Zheng, Kaiyue Wen, Zekun Wang, Rui Men, Ivan Titov, Dayiheng Liu, Jingren Zhou, Junyang Lin
2025	J	`jnl`	Fantastic Pretraining Optimizers and Where to Find Them. CoRR Kaiyue Wen, David Hall, Tengyu Ma, Percy Liang
2025	A*	`conf`	From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency. ICLR Kaiyue Wen, Huaqing Zhang, Hongzhou Lin, Jingzhao Zhang
2025	J	`jnl`	Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free. CoRR Zihan Qiu, Zekun Wang, Bo Zheng, Zeyu Huang, Kaiyue Wen, Songlin Yang, Rui Men, Le Yu, Fei Huang, Suozhi Huang, Dayiheng Liu, Jingren Zhou, Junyang Lin
2025	A*	`conf`	Overtrained Language Models Are Harder to Fine-Tune. ICML Jacob Mitchell Springer, Sachin Goyal, Kaiyue Wen, Tanishq Kumar, Xiang Yue, Sadhika Malladi, Graham Neubig, Aditi Raghunathan
2025	J	`jnl`	Overtrained Language Models Are Harder to Fine-Tune. CoRR Jacob Mitchell Springer, Sachin Goyal, Kaiyue Wen, Tanishq Kumar, Xiang Yue, Sadhika Malladi, Graham Neubig, Aditi Raghunathan
2025	J	`jnl`	PaTH Attention: Position Encoding via Accumulating Householder Transformations. CoRR Songlin Yang, Yikang Shen, Kaiyue Wen, Shawn Tan, Mayank Mishra, Liliang Ren, Rameswar Panda, Yoon Kim
2025	J	`jnl`	QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation. CoRR Jiazheng Li, Hong Lu, Kaiyue Wen, Zaiwen Yang, Jiaxuan Gao, Hongzhou Lin, Yi Wu, Jingzhao Zhang
2025	A*	`conf`	RNNs are not Transformers (Yet): The Key Bottleneck on In-Context Retrieval. ICLR Kaiyue Wen, Xingyu Dang, Kaifeng Lyu
2025	—	`conf`	Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images. ACL (1) Shengguang Wu, Fan-Yun Sun, Kaiyue Wen, Nick Haber
2025	J	`jnl`	Symmetrical Visual Contrastive Optimization: Aligning Vision-Language Models with Minimal Contrastive Images. CoRR Shengguang Wu, Fan-Yun Sun, Kaiyue Wen, Nick Haber
2025	J	`jnl`	Task Generalization With AutoRegressive Compositional Structure: Can Learning From CoRR Amirhesam Abedsoltan, Huaqing Zhang, Kaiyue Wen, Hongzhou Lin, Jingzhao Zhang, Mikhail Belkin
2025	A*	`conf`	Task Generalization with Autoregressive Compositional Structure: Can Learning from D Tasks Generalize to DT Tasks? ICML Amirhesam Abedsoltan, Huaqing Zhang, Kaiyue Wen, Hongzhou Lin, Jingzhao Zhang, Mikhail Belkin
2025	A*	`conf`	Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape View. ICLR Kaiyue Wen, Zhiyuan Li, Jason S. Wang, David Leo Wright Hall, Percy Liang, Tengyu Ma
2025	J	`jnl`	Weight Ensembling Improves Reasoning in Language Models. CoRR Xingyu Dang, Christina Baek, Kaiyue Wen, Zico Kolter, Aditi Raghunathan
2024	J	`jnl`	From Sparse Dependence to Sparse Attention: Unveiling How Chain-of-Thought Enhances Transformer Sample Efficiency. CoRR Kaiyue Wen, Huaqing Zhang, Hongzhou Lin, Jingzhao Zhang
2024	J	`jnl`	RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval. CoRR Kaiyue Wen, Xingyu Dang, Kaifeng Lyu
2024	J	`jnl`	Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective. CoRR Kaiyue Wen, Zhiyuan Li, Jason S. Wang, David Hall, Percy Liang, Tengyu Ma
2023	A*	`conf`	Benign Overfitting in Classification: Provably Counter Label Noise with Larger Models. ICLR Kaiyue Wen, Jiaye Teng, Jingzhao Zhang
2023	A*	`conf`	How Sharpness-Aware Minimization Minimizes Sharpness? ICLR Kaiyue Wen, Tengyu Ma, Zhiyuan Li
2023	J	`jnl`	Practically Solving LPN in High Noise Regimes Faster Using Neural Networks. CoRR Haozhe Jiang, Kaiyue Wen, Yilei Chen
2023	J	`jnl`	Practically Solving LPN in High Noise Regimes Faster Using Neural Networks. IACR Cryptol. ePrint Arch. Haozhe Jiang, Kaiyue Wen, Yilei Chen
2023	A*	`conf`	Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization. NeurIPS Kaiyue Wen, Zhiyuan Li, Tengyu Ma
2023	J	`jnl`	Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization. CoRR Kaiyue Wen, Zhiyuan Li, Tengyu Ma
2023	A*	`conf`	Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammars. NeurIPS Kaiyue Wen, Yuchen Li, Bingbin Liu, Andrej Risteski
2023	J	`jnl`	Transformers are uninterpretable with myopic methods: a case study with bounded Dyck grammars. CoRR Kaiyue Wen, Yuchen Li, Bingbin Liu, Andrej Risteski
2022	A*	`conf`	Finding Skill Neurons in Pre-trained Transformer-based Language Models. EMNLP Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, Juanzi Li
2022	J	`jnl`	Finding Skill Neurons in Pre-trained Transformer-based Language Models. CoRR Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, Lei Hou, Zhiyuan Liu, Juanzi Li
2022	J	`jnl`	How Does Sharpness-Aware Minimization Minimize Sharpness? CoRR Kaiyue Wen, Tengyu Ma, Zhiyuan Li
2022	—	`conf`	On Transferability of Prompt Tuning for Natural Language Processing. NAACL-HLT Yusheng Su, Xiaozhi Wang, Yujia Qin, Chi-Min Chan, Yankai Lin, Huadong Wang, Kaiyue Wen, Zhiyuan Liu, Peng Li, Juanzi Li, Lei Hou, Maosong Sun, Jie Zhou
2022	J	`jnl`	Realistic Deep Learning May Not Fit Benignly. CoRR Kaiyue Wen, Jiaye Teng, Jingzhao Zhang