Soufiane Hayou

44 papers A* 10A 1Journal 31Unranked 1

Year	Rank	Type	Title / Venue / Authors
2026	J	`jnl`	Learning Rate Scaling across LoRA Ranks and Transfer to Full Finetuning. CoRR Nan Chen, Soledad Villar, Soufiane Hayou
2026	J	`jnl`	μpscaling small models: Principled warm starts and hyperparameter transfer. CoRR Yuxin Ma, Nan Chen, Mateo Díaz, Soufiane Hayou, Dmitriy Kunisky, Soledad Villar
2025	J	`jnl`	A Proof of Learning Rate Transfer under μP. CoRR Soufiane Hayou
2025	A*	`conf`	Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory. ICLR Aymane El Firdoussi, Mohamed El Amine Seddik, Soufiane Hayou, Réda Alami, Ahmed Alzubaidi, Hakim Hacid
2025	J	`jnl`	On the Stability of the Jacobian Matrix in Deep Neural Networks. CoRR Benjamin Dadoun, Soufiane Hayou, Hanan Salam, Mohamed El Amine Seddik, Pierre Youssef
2025	J	`jnl`	Optimal Embedding Learning Rate in LLMs: The Effect of Vocabulary Size. CoRR Soufiane Hayou, Liyuan Liu
2025	J	`jnl`	PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models. CoRR Soufiane Hayou, Nikhil Ghosh, Bin Yu
2024	J	`jnl`	Commutative Scaling of Width and Depth in Deep Neural Networks. J. Mach. Learn. Res. Soufiane Hayou
2024	J	`jnl`	How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse. CoRR Mohamed El Amine Seddik, Suei-Wen Chen, Soufiane Hayou, Pierre Youssef, Mérouane Debbah
2024	A*	`conf`	Leave-one-out Distinguishability in Machine Learning. ICLR Jiayuan Ye, Anastasia Borovykh, Soufiane Hayou, Reza Shokri
2024	A*	`conf`	LoRA+: Efficient Low Rank Adaptation of Large Models. ICML Soufiane Hayou, Nikhil Ghosh, Bin Yu
2024	J	`jnl`	LoRA+: Efficient Low Rank Adaptation of Large Models. CoRR Soufiane Hayou, Nikhil Ghosh, Bin Yu
2024	J	`jnl`	Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory. CoRR Aymane El Firdoussi, Mohamed El Amine Seddik, Soufiane Hayou, Réda Alami, Ahmed Alzubaidi, Hakim Hacid
2024	A*	`conf`	Tensor Programs VI: Feature Learning in Infinite Depth Neural Networks. ICLR Greg Yang, Dingli Yu, Chen Zhu, Soufiane Hayou
2024	A*	`conf`	The Impact of Initialization on LoRA Finetuning Dynamics. NeurIPS Soufiane Hayou, Nikhil Ghosh, Bin Yu
2024	J	`jnl`	The Impact of Initialization on LoRA Finetuning Dynamics. CoRR Soufiane Hayou, Nikhil Ghosh, Bin Yu
2024	J	`jnl`	Visualising Feature Learning in Deep Neural Networks by Diagonalizing the Forward Feature Map. CoRR Yoonsoo Nam, Chris Mingard, Seok Hyeong Lee, Soufiane Hayou, Ard A. Louis
2023	J	`jnl`	Commutative Width and Depth Scaling in Deep Neural Networks. CoRR Soufiane Hayou
2023	J	`jnl`	Data pruning and neural scaling laws: fundamental limitations of score-based algorithms. CoRR Fadhel Ayed, Soufiane Hayou
2023	J	`jnl`	Data pruning and neural scaling laws: fundamental limitations of score-based algorithms. Trans. Mach. Learn. Res. Fadhel Ayed, Soufiane Hayou
2023	J	`jnl`	Leave-one-out Distinguishability in Machine Learning. CoRR Jiayuan Ye, Anastasia Borovykh, Soufiane Hayou, Reza Shokri
2023	J	`jnl`	On the Connection Between Riemann Hypothesis and a Special Class of Neural Networks. CoRR Soufiane Hayou
2023	J	`jnl`	On the infinite-depth limit of finite-width neural networks. Trans. Mach. Learn. Res. Soufiane Hayou
2023	J	`jnl`	Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks. CoRR Greg Yang, Dingli Yu, Chen Zhu, Soufiane Hayou
2023	A*	`conf`	Width and Depth Limits Commute in Residual Networks. ICML Soufiane Hayou, Greg Yang
2023	J	`jnl`	Width and Depth Limits Commute in Residual Networks. CoRR Soufiane Hayou, Greg Yang
2022	J	`jnl`	Connecting Optimization and Generalization via Gradient Flow Path Length. CoRR Fusheng Liu, Haizhao Yang, Soufiane Hayou, Qianxiao Li
2022	A*	`conf`	Feature Learning and Signal Propagation in Deep Neural Networks. ICML Yizhang Lou, Chris E. Mingard, Soufiane Hayou
2022	J	`jnl`	From Optimization Dynamics to Generalization Bounds via Łojasiewicz Gradient Inequality. Trans. Mach. Learn. Res. Fusheng Liu, Haizhao Yang, Soufiane Hayou, Qianxiao Li
2022	J	`jnl`	On the infinite-depth limit of finite-width neural networks. CoRR Soufiane Hayou
2021	J	`jnl`	Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning. CoRR Soufiane Hayou, Bobby He, Gintare Karolina Dziugaite
2021	A*	`conf`	Regularization in ResNet with Stochastic Depth. NeurIPS Soufiane Hayou, Fadhel Ayed
2021	J	`jnl`	Regularization in ResNet with Stochastic Depth. CoRR Soufiane Hayou, Fadhel Ayed
2021	A*	`conf`	Robust Pruning at Initialization. ICLR Soufiane Hayou, Jean-Francois Ton, Arnaud Doucet, Yee Whye Teh
2021	A	`conf`	Stable ResNet. AISTATS Soufiane Hayou, Eugenio Clerico, Bobby He, George Deligiannidis, Arnaud Doucet, Judith Rousseau
2021	—	`conf`	The Curse of Depth in Kernel Regime. ICBINB@NeurIPS Soufiane Hayou, Arnaud Doucet, Judith Rousseau
2021	J	`jnl`	The Equilibrium Hypothesis: Rethinking implicit regularization in Deep Neural Networks. CoRR Yizhang Lou, Chris Mingard, Soufiane Hayou
2021	—	`—`	Wide deep neural networks. Soufiane Hayou
2020	J	`jnl`	Pruning untrained neural networks: Principles and Analysis. CoRR Soufiane Hayou, Jean-Francois Ton, Arnaud Doucet, Yee Whye Teh
2020	J	`jnl`	Stable ResNet. CoRR Soufiane Hayou, Eugenio Clerico, Bobby He, George Deligiannidis, Arnaud Doucet, Judith Rousseau
2019	J	`jnl`	On the Impact of the Activation Function on Deep Neural Networks Training. CoRR Soufiane Hayou, Arnaud Doucet, Judith Rousseau
2019	A*	`conf`	On the Impact of the Activation function on Deep Neural Networks Training. ICML Soufiane Hayou, Arnaud Doucet, Judith Rousseau
2019	J	`jnl`	Training Dynamics of Deep Networks using Stochastic Gradient Descent via Neural Tangent Kernel. CoRR Soufiane Hayou, Arnaud Doucet, Judith Rousseau
2018	J	`jnl`	On the Selection of Initialization and Activation Function for Deep Neural Networks. CoRR Soufiane Hayou, Arnaud Doucet, Judith Rousseau