Nathan Godey

15 papers A* 1Journal 10Unranked 3
YearRankTypeTitle / Venue / Authors
2026 J jnl
Lost in Backpropagation: The LM Head is a Gradient Bottleneck.
CoRR
Nathan Godey, Yoav Artzi
2025 J jnl
Biomed-Enriched: A Biomedical Dataset Enriched with LLMs for Pretraining and Extracting Rare and Hidden Content.
CoRR
Rian Touchent, Nathan Godey, Éric de la Clergerie
2025 J jnl
Gaperon: A Peppered English-French Generative Language Model Suite.
CoRR
Nathan Godey, Wissam Antoun, Rian Touchent, Rachel Bawden, Éric de la Clergerie, Benoît Sagot, Djamé Seddah
2025 J jnl
Q-Filters: Leveraging QK Geometry for Efficient KV Cache Compression.
CoRR
Nathan Godey, Alessio Devoto, Yu Zhao, Simone Scardapane, Pasquale Minervini, Éric de la Clergerie, Benoît Sagot
2024 — conf
Anisotropy Is Inherent to Self-Attention in Transformers.
EACL (1)
Nathan Godey, Éric Villemonte de la Clergerie, Benoît Sagot
2024 J jnl
Anisotropy Is Inherent to Self-Attention in Transformers.
CoRR
Nathan Godey, Éric de la Clergerie, Benoît Sagot
2024 A* conf
Headless Language Models: Learning without Predicting with Contrastive Weight Tying.
ICLR
Nathan Godey, Éric Villemonte de la Clergerie, Benoît Sagot
2024 — —
Improving Representations for Language Modeling. (Amélioration des représentations pour la modélisation du langage).
Nathan Godey
2024 — conf
On the Scaling Laws of Geographical Representation in Language Models.
LREC/COLING
Nathan Godey, Éric de la Clergerie, Benoît Sagot
2024 J jnl
On the Scaling Laws of Geographical Representation in Language Models.
CoRR
Nathan Godey, Éric de la Clergerie, Benoît Sagot
2024 J jnl
Why do small language models underperform? Studying Language Model Saturation via the Softmax Bottleneck.
CoRR
Nathan Godey, Éric de la Clergerie, Benoît Sagot
2023 J jnl
Headless Language Models: Learning without Predicting with Contrastive Weight Tying.
CoRR
Nathan Godey, Éric de la Clergerie, Benoît Sagot
2023 J jnl
Is Anisotropy Inherent to Transformers?
CoRR
Nathan Godey, Éric de la Clergerie, Benoît Sagot
2022 — conf
MANTa: Efficient Gradient-Based Tokenization for End-to-End Robust Language Modeling.
EMNLP (Findings)
Nathan Godey, Roman Castagné, Éric de la Clergerie, Benoît Sagot
2022 J jnl
MANTa: Efficient Gradient-Based Tokenization for Robust End-to-End Language Modeling.
CoRR
Nathan Godey, Roman Castagné, Éric de la Clergerie, Benoît Sagot