Katherine Lee

50 papers A* 10B 3Misc 1Journal 29Unranked 7
YearRankTypeTitle / Venue / Authors
2026 J jnl
Estimating near-verbatim extraction risk in language models with decoding-constrained beam search.
CoRR
A. Feder Cooper, Mark A. Lemley, Christopher De Sa, Lea Duesterwald, Allison Casasola, Jamie Hayes, Katherine Lee, Daniel E. Ho, Percy Liang
2025 A* conf
Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards.
ICML
Yangsibo Huang, Milad Nasr, Anastasios Nikolas Angelopoulos, Nicholas Carlini, Wei-Lin Chiang, Christopher A. Choquette-Choo, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Ken Liu, Ion Stoica, Florian Tramèr, Chiyuan Zhang
2025 J jnl
Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards.
CoRR
Yangsibo Huang, Milad Nasr, Anastasios Angelopoulos, Nicholas Carlini, Wei-Lin Chiang, Christopher A. Choquette-Choo, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Ken Ziyu Liu, Ion Stoica, Florian Tramèr, Chiyuan Zhang
2025 — conf
Measuring memorization in language models via probabilistic extraction.
NAACL (Long Papers)
Jamie Hayes, Marika Swanberg, Harsh Chaudhari, Itay Yona, Ilia Shumailov, Milad Nasr, Christopher A. Choquette-Choo, Katherine Lee, A. Feder Cooper
2025 — conf
Privacy Ripple Effects from Adding or Removing Personal Information in Language Model Training.
ACL (Findings)
Jaydeep Borkar, Matthew Jagielski, Katherine Lee, Niloofar Mireshghallah, David A. Smith, Christopher A. Choquette-Choo
2025 J jnl
Privacy Ripple Effects from Adding or Removing Personal Information in Language Model Training.
CoRR
Jaydeep Borkar, Matthew Jagielski, Katherine Lee, Niloofar Mireshghallah, David A. Smith, Christopher A. Choquette-Choo
2025 A* conf
Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon.
ICLR
USVSN Sai Prashanth, Alvin Deng, Kyle O'Brien, Jyothir S. V, Mohammad Aflah Khan, Jaydeep Borkar, Christopher A. Choquette-Choo, Jacob Ray Fuehne, Stella Biderman, Tracy Ke, Katherine Lee, Naomi Saphra
2025 A* conf
Scalable Extraction of Training Data from Aligned, Production Language Models.
ICLR
Milad Nasr, Javier Rando, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito, Christopher A. Choquette-Choo, Florian Tramèr, Katherine Lee
2025 J jnl
Strong Membership Inference Attacks on Massive Datasets and (Moderately) Large Language Models.
CoRR
Jamie Hayes, Ilia Shumailov, Christopher A. Choquette-Choo, Matthew Jagielski, Georgios Kaissis, Katherine Lee, Milad Nasr, Sahra Ghalebikesabi, Niloofar Mireshghallah, Meenatchi Sundaram Mutu Selva Annamalai, Igor Shilov, Matthieu Meeus, Yves-Alexandre de Montjoye, Franziska Boenisch, Adam Dziedzic, A. Feder Cooper
2024 — conf
A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity.
NAACL-HLT
Shayne Longpre, Gregory Yauney, Emily Reif, Katherine Lee, Adam Roberts, Barret Zoph, Denny Zhou, Jason Wei, Kevin Robinson, David Mimno, Daphne Ippolito
2024 J jnl
An Abundance of Katherines: The Game Theory of Baby Naming.
CoRR
Katy Blumer, Kate Donahue, Katie Fritz, Kate Ivanovich, Katherine Lee, Katie Luo, Cathy Meng, Katie Van Koevering
2024 A* conf
Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classification.
AAAI
A. Feder Cooper, Katherine Lee, Madiha Zahrah Choksi, Solon Barocas, Christopher De Sa, James Grimmelmann, Jon M. Kleinberg, Siddhartha Sen, Baobao Zhang
2024 J jnl
LMD3: Language Model Data Density Dependence.
CoRR
John Kirchenbauer, Garrett Honke, Gowthami Somepalli, Jonas Geiping, Daphne Ippolito, Katherine Lee, Tom Goldstein, David Andre
2024 J jnl
Machine Unlearning Doesn't Do What You Think: Lessons for Generative AI Policy, Research, and Practice.
CoRR
A. Feder Cooper, Christopher A. Choquette-Choo, Miranda Bogen, Matthew Jagielski, Katja Filippova, Ken Ziyu Liu, Alexandra Chouldechova, Jamie Hayes, Yangsibo Huang, Niloofar Mireshghallah, Ilia Shumailov, Eleni Triantafillou, Peter Kairouz, Nicole Mitchell, Percy Liang, Daniel E. Ho, Yejin Choi, Sanmi Koyejo, Fernando A. Delgado, James Grimmelmann, Vitaly Shmatikov, Christopher De Sa, Solon Barocas, Amy Cyphert, Mark A. Lemley, danah boyd, Jennifer Wortman Vaughan, Miles Brundage, David Bau, Seth Neel, Abigail Z. Jacobs, Andreas Terzis, Hanna M. Wallach, Nicolas Papernot, Katherine Lee
2024 J jnl
Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon.
CoRR
USVSN Sai Prashanth, Alvin Deng, Kyle O'Brien, Jyothir S. V, Mohammad Aflah Khan, Jaydeep Borkar, Christopher A. Choquette-Choo, Jacob Ray Fuehne, Stella Biderman, Tracy Ke, Katherine Lee, Naomi Saphra
2024 J jnl
Stealing Part of a Production Language Model.
CoRR
Nicholas Carlini, Daniel Paleka, Krishnamurthy (Dj) Dvijotham, Thomas Steinke, Jonathan Hayase, A. Feder Cooper, Katherine Lee, Matthew Jagielski, Milad Nasr, Arthur Conmy, Eric Wallace, David Rolnick, Florian Tramèr
2024 A* conf
Stealing part of a production language model.
ICML
Nicholas Carlini, Daniel Paleka, Krishnamurthy Dj Dvijotham, Thomas Steinke, Jonathan Hayase, A. Feder Cooper, Katherine Lee, Matthew Jagielski, Milad Nasr, Arthur Conmy, Eric Wallace, David Rolnick, Florian Tramèr
2024 — conf
Talkin' 'Bout AI Generation: Copyright and the Generative-AI Supply Chain (The Short Version).
CSLAW
Katherine Lee, A. Feder Cooper, James Grimmelmann
2023 J jnl
A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicity.
CoRR
Shayne Longpre, Gregory Yauney, Emily Reif, Katherine Lee, Adam Roberts, Barret Zoph, Denny Zhou, Jason Wei, Kevin Robinson, David Mimno, Daphne Ippolito
2023 J jnl
Are aligned neural networks adversarially aligned?
CoRR
Nicholas Carlini, Milad Nasr, Christopher A. Choquette-Choo, Matthew Jagielski, Irena Gao, Anas Awadalla, Pang Wei Koh, Daphne Ippolito, Katherine Lee, Florian Tramèr, Ludwig Schmidt
2023 A* conf
Counterfactual Memorization in Neural Language Models.
NeurIPS
Chiyuan Zhang, Daphne Ippolito, Katherine Lee, Matthew Jagielski, Florian Tramèr, Nicholas Carlini
2023 J jnl
MADLAD-400: A Multilingual And Document-Level Large Audited Dataset.
CoRR
Sneha Kudugunta, Isaac Caswell, Biao Zhang, Xavier Garcia, Christopher A. Choquette-Choo, Katherine Lee, Derrick Xin, Aditya Kusupati, Romi Stella, Ankur Bapna, Orhan Firat
2023 A* conf
Measuring Forgetting of Memorized Training Examples.
ICLR
Matthew Jagielski, Om Thakkar, Florian Tramèr, Daphne Ippolito, Katherine Lee, Nicholas Carlini, Eric Wallace, Shuang Song, Abhradeep Guha Thakurta, Nicolas Papernot, Chiyuan Zhang
2023 J jnl
PaLM: Scaling Language Modeling with Pathways.
J. Mach. Learn. Res.
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, Noah Fiedel
2023 B conf
Preventing Generation of Verbatim Memorization in Language Models Gives a False Sense of Privacy.
INLG
Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A. Choquette-Choo, Nicholas Carlini
2023 A* conf
Quantifying Memorization Across Neural Language Models.
ICLR
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramèr, Chiyuan Zhang
2023 J jnl
Report of the 1st Workshop on Generative AI and Law.
CoRR
A. Feder Cooper, Katherine Lee, James Grimmelmann, Daphne Ippolito, Christopher Callison-Burch, Christopher A. Choquette-Choo, Niloofar Mireshghallah, Miles Brundage, David Mimno, Madiha Zahrah Choksi, Jack M. Balkin, Nicholas Carlini, Christopher De Sa, Jonathan Frankle, Deep Ganguli, Bryant Gipson, Andres Guadamuz, Swee Leng Harris, Abigail Z. Jacobs, Elizabeth Joh, Gautam Kamath, Mark A. Lemley, Cass Matthews, Christine McLeavey, Corynne McSherry, Milad Nasr, Paul Ohm, Adam Roberts, Tom Rubin, Pamela Samuelson, Ludwig Schubert, Kristen Vaccaro, Luis Villa, Felix Wu, Elana Zeide
2023 B conf
Reverse-Engineering Decoding Strategies Given Blackbox Access to a Language Generation System.
INLG
Daphne Ippolito, Nicholas Carlini, Katherine Lee, Milad Nasr, Yun William Yu
2023 J jnl
Reverse-Engineering Decoding Strategies Given Blackbox Access to a Language Generation System.
CoRR
Daphne Ippolito, Nicholas Carlini, Katherine Lee, Milad Nasr, Yun William Yu
2023 J jnl
Scalable Extraction of Training Data from (Production) Language Models.
CoRR
Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito, Christopher A. Choquette-Choo, Eric Wallace, Florian Tramèr, Katherine Lee
2023 A* conf
Students Parrot Their Teachers: Membership Inference on Model Distillation.
NeurIPS
Matthew Jagielski, Milad Nasr, Katherine Lee, Christopher A. Choquette-Choo, Nicholas Carlini, Florian Tramèr
2023 J jnl
Students Parrot Their Teachers: Membership Inference on Model Distillation.
CoRR
Matthew Jagielski, Milad Nasr, Christopher A. Choquette-Choo, Katherine Lee, Nicholas Carlini
2023 J jnl
Talkin' 'Bout AI Generation: Copyright and the Generative-AI Supply Chain.
CoRR
Katherine Lee, A. Feder Cooper, James Grimmelmann
2022 — conf
Deduplicating Training Data Makes Language Models Better.
ACL (1)
Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, Nicholas Carlini
2022 J jnl
Measuring Forgetting of Memorized Training Examples.
CoRR
Matthew Jagielski, Om Thakkar, Florian Tramèr, Daphne Ippolito, Katherine Lee, Nicholas Carlini, Eric Wallace, Shuang Song, Abhradeep Thakurta, Nicolas Papernot, Chiyuan Zhang
2022 J jnl
PaLM: Scaling Language Modeling with Pathways.
CoRR
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam Shazeer, Vinodkumar Prabhakaran, Emily Reif, Nan Du, Ben Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier Garcia, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph, Alexander Spiridonov, Ryan Sepassi, David Dohan, Shivani Agrawal, Mark Omernick, Andrew M. Dai, Thanumalayan Sankaranarayana Pillai, Marie Pellat, Aitor Lewkowycz, Erica Moreira, Rewon Child, Oleksandr Polozov, Katherine Lee, Zongwei Zhou, Xuezhi Wang, Brennan Saeta, Mark Diaz, Orhan Firat, Michele Catasta, Jason Wei, Kathy Meier-Hellstern, Douglas Eck, Jeff Dean, Slav Petrov, Noah Fiedel
2022 J jnl
Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy.
CoRR
Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A. Choquette-Choo, Nicholas Carlini
2022 J jnl
Quantifying Memorization Across Neural Language Models.
CoRR
Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramèr, Chiyuan Zhang
2022 — conf
What Does it Mean for a Language Model to Preserve Privacy?
FAccT
Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah, Reza Shokri, Florian Tramèr
2022 J jnl
What Does it Mean for a Language Model to Preserve Privacy?
CoRR
Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah, Reza Shokri, Florian Tramèr
2021 J jnl
Counterfactual Memorization in Neural Language Models.
CoRR
Chiyuan Zhang, Daphne Ippolito, Katherine Lee, Matthew Jagielski, Florian Tramèr, Nicholas Carlini
2021 J jnl
Deduplicating Training Data Makes Language Models Better.
CoRR
Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, Nicholas Carlini
2021 A* conf
Extracting Training Data from Large Language Models.
USENIX Security Symposium
Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B. Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, Colin Raffel
2021 Misc conf
Predictive Modeling of Healthcare Utilization Metrics Identifies Adult Patients at High Risk for Suicide Attempt in the Primary Care Setting.
AMIA
Katherine Lee, Colin G. Walsh
2020 J jnl
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.
J. Mach. Learn. Res.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu
2020 J jnl
Extracting Training Data from Large Language Models.
CoRR
Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B. Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, Colin Raffel
2020 J jnl
WT5?! Training Text-to-Text Models to Explain their Predictions.
CoRR
Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, Karishma Malkan
2019 J jnl
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer.
CoRR
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J. Liu
2015 — conf
A computer-aided diagnosis system to identify regions of pathologic change in temporal subtraction images of the chest.
Computer-Aided Diagnosis
Charles Ho, Katherine Lee, Samuel G. Armato
2004 B conf
Analysis and Detection of Reading Miscues for Interactive Literacy Tutors.
COLING
Katherine Lee, Andreas Hagen, Nicholas Romanyshyn, Sean Martin, Bryan L. Pellom