Iz Beltagy

72 papers A* 9Journal 40Unranked 22
YearRankTypeTitle / Venue / Authors
2024 — conf
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research.
ACL (1)
Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Raghavi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A. Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, Kyle Lo
2024 J jnl
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research.
CoRR
Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Raghavi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A. Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, Kyle Lo
2024 — conf
OLMo: Accelerating the Science of Language Models.
ACL (1)
Dirk Groeneveld, Iz Beltagy, Evan Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, Hannaneh Hajishirzi
2024 J jnl
OLMo: Accelerating the Science of Language Models.
CoRR
Dirk Groeneveld, Iz Beltagy, Pete Walsh, Akshita Bhagia, Rodney Kinney, Oyvind Tafjord, Ananya Harsh Jha, Hamish Ivison, Ian Magnusson, Yizhong Wang, Shane Arora, David Atkinson, Russell Authur, Khyathi Raghavi Chandu, Arman Cohan, Jennifer Dumas, Yanai Elazar, Yuling Gu, Jack Hessel, Tushar Khot, William Merrill, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Valentina Pyatkin, Abhilasha Ravichander, Dustin Schwenk, Saurabh Shah, Will Smith, Emma Strubell, Nishant Subramani, Mitchell Wortsman, Pradeep Dasigi, Nathan Lambert, Kyle Richardson, Luke Zettlemoyer, Jesse Dodge, Kyle Lo, Luca Soldaini, Noah A. Smith, Hannaneh Hajishirzi
2024 A* conf
Paloma: A Benchmark for Evaluating Language Model Fit.
NeurIPS
Ian Magnusson, Akshita Bhagia, Valentin Hofmann, Luca Soldaini, Ananya Harsh Jha, Oyvind Tafjord, Dustin Schwenk, Evan Pete Walsh, Yanai Elazar, Kyle Lo, Dirk Groeneveld, Iz Beltagy, Hanna Hajishirzi, Noah A. Smith, Kyle Richardson, Jesse Dodge
2024 J jnl
Source-Aware Training Enables Knowledge Attribution in Language Models.
CoRR
Muhammad Khalifa, David Wadden, Emma Strubell, Honglak Lee, Lu Wang, Iz Beltagy, Hao Peng
2024 — conf
TESS: Text-to-Text Self-Conditioned Simplex Diffusion.
EACL (1)
Rabeeh Karimi Mahabadi, Hamish Ivison, Jaesung Tae, James Henderson, Iz Beltagy, Matthew E. Peters, Arman Cohan
2023 J jnl
Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2.
CoRR
Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew E. Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A. Smith, Iz Beltagy, Hannaneh Hajishirzi
2023 J jnl
Catwalk: A Unified Language Model Evaluation Framework for Many Datasets.
CoRR
Dirk Groeneveld, Anas Awadalla, Iz Beltagy, Akshita Bhagia, Ian Magnusson, Hao Peng, Oyvind Tafjord, Pete Walsh, Kyle Richardson, Jesse Dodge
2023 J jnl
Efficiency Pentathlon: A Standardized Arena for Efficiency Evaluation.
CoRR
Hao Peng, Qingqing Cao, Jesse Dodge, Matthew E. Peters, Jared Fernandez, Tom Sherborne, Kyle Lo, Sam Skjonsberg, Emma Strubell, Darrell Plessas, Iz Beltagy, Evan Pete Walsh, Noah A. Smith, Hannaneh Hajishirzi
2023 — conf
FiD-ICL: A Fusion-in-Decoder Approach for Efficient In-Context Learning.
ACL (1)
Qinyuan Ye, Iz Beltagy, Matthew E. Peters, Xiang Ren, Hannaneh Hajishirzi
2023 A* conf
How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources.
NeurIPS
Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, Hannaneh Hajishirzi
2023 J jnl
How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resources.
CoRR
Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, Hannaneh Hajishirzi
2023 J jnl
Large Language Model Distillation Doesn't Need a Teacher.
CoRR
Ananya Harsh Jha, Dirk Groeneveld, Emma Strubell, Iz Beltagy
2023 J jnl
Paloma: A Benchmark for Evaluating Language Model Fit.
CoRR
Ian Magnusson, Akshita Bhagia, Valentin Hofmann, Luca Soldaini, Ananya Harsh Jha, Oyvind Tafjord, Dustin Schwenk, Evan Pete Walsh, Yanai Elazar, Kyle Lo, Dirk Groeneveld, Iz Beltagy, Hannaneh Hajishirzi, Noah A. Smith, Kyle Richardson, Jesse Dodge
2023 J jnl
TESS: Text-to-Text Self-Conditioned Simplex Diffusion.
CoRR
Rabeeh Karimi Mahabadi, Jaesung Tae, Hamish Ivison, James Henderson, Iz Beltagy, Matthew E. Peters, Arman Cohan
2023 J jnl
The Semantic Scholar Open Data Platform.
CoRR
Rodney Kinney, Chloe Anastasiades, Russell Authur, Iz Beltagy, Jonathan Bragg, Alexandra Buraczynski, Isabel Cachola, Stefan Candra, Yoganand Chandrasekhar, Arman Cohan, Miles Crawford, Doug Downey, Jason Dunkelberger, Oren Etzioni, Rob Evans, Sergey Feldman, Joseph Gorney, David Graham, Fangzhou Hu, Regan Huff, Daniel King, Sebastian Kohlmeier, Bailey Kuehl, Michael Langan, Daniel Lin, Haokun Liu, Kyle Lo, Jaron Lochner, Kelsey MacMillan, Tyler Murray, Chris Newell, Smita Rao, Shaurya Rohatgi, Paul Sayre, Zejiang Shen, Amanpreet Singh, Luca Soldaini, Shivashankar Subramanian, Amber Tanaka, Alex D. Wade, Linda Wagner, Lucy Lu Wang, Chris Wilhelm, Caroline Wu, Jiangjiang Yang, Angele Zamarron, Madeleine van Zuylen, Daniel S. Weld
2023 J jnl
Transparency Helps Reveal When Language Models Learn Meaning.
Trans. Assoc. Comput. Linguistics
Zhaofeng Wu, William Merrill, Hao Peng, Iz Beltagy, Noah A. Smith
2023 — conf
Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations.
ACL (1)
Xinxi Lyu, Sewon Min, Iz Beltagy, Luke Zettlemoyer, Hannaneh Hajishirzi
2022 A* conf
Continued Pretraining for Better Zero- and Few-Shot Promptability.
EMNLP
Zhaofeng Wu, Robert L. Logan IV, Pete Walsh, Akshita Bhagia, Dirk Groeneveld, Sameer Singh, Iz Beltagy
2022 J jnl
Continued Pretraining for Better Zero- and Few-Shot Promptability.
CoRR
Zhaofeng Wu, Robert L. Logan IV, Pete Walsh, Akshita Bhagia, Dirk Groeneveld, Sameer Singh, Iz Beltagy
2022 J jnl
Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search.
CoRR
Daniel King, Zejiang Shen, Nishant Subramani, Daniel S. Weld, Iz Beltagy, Doug Downey
2022 — conf
Few-Shot Self-Rationalization with Natural Language Prompts.
NAACL-HLT (Findings)
Ana Marasovic, Iz Beltagy, Doug Downey, Matthew E. Peters
2022 J jnl
Infrastructure for Rapid Open Knowledge Network Development.
AI Mag.
Michael J. Cafarella, Michael R. Anderson, Iz Beltagy, Arie Cattan, Sarah E. Chasins, Ido Dagan, Doug Downey, Oren Etzioni, Sergey Feldman, Tian Gao, Tom Hope, Kexin Huang, Sophie Johnson, Daniel King, Kyle Lo, Yuze Lou, Matthew D. Shapiro, Dinghao Shen, Shivashankar Subramanian, Lucy Lu Wang, Yuning Wang, Yitong Wang, Daniel S. Weld, Jenny M. Vo-Phamhi, Anna Zeng, Jiayun Zou
2022 — conf
MultiVerS: Improving scientific claim verification with weak supervision and full-document context.
NAACL-HLT (Findings)
David Wadden, Kyle Lo, Lucy Lu Wang, Arman Cohan, Iz Beltagy, Hannaneh Hajishirzi
2022 — conf
PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization.
ACL (1)
Wen Xiao, Iz Beltagy, Giuseppe Carenini, Arman Cohan
2022 — conf
SciFact-Open: Towards open-domain scientific claim verification.
EMNLP (Findings)
David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Iz Beltagy, Lucy Lu Wang, Hannaneh Hajishirzi
2022 J jnl
SciFact-Open: Towards open-domain scientific claim verification.
CoRR
David Wadden, Kyle Lo, Bailey Kuehl, Arman Cohan, Iz Beltagy, Lucy Lu Wang, Hannaneh Hajishirzi
2022 A* conf
Staged Training for Transformer Language Models.
ICML
Sheng Shen, Pete Walsh, Kurt Keutzer, Jesse Dodge, Matthew E. Peters, Iz Beltagy
2022 J jnl
Staged Training for Transformer Language Models.
CoRR
Sheng Shen, Pete Walsh, Kurt Keutzer, Jesse Dodge, Matthew E. Peters, Iz Beltagy
2022 J jnl
Transparency Helps Reveal When Language Models Learn Meaning.
CoRR
Zhaofeng Wu, William Merrill, Hao Peng, Iz Beltagy, Noah A. Smith
2022 J jnl
What Language Model Architecture and Pretraining Objective Work Best for Zero-Shot Generalization?
CoRR
Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay, Colin Raffel
2022 A* conf
What Language Model Architecture and Pretraining Objective Works Best for Zero-Shot Generalization?
ICML
Thomas Wang, Adam Roberts, Daniel Hesslow, Teven Le Scao, Hyung Won Chung, Iz Beltagy, Julien Launay, Colin Raffel
2022 — conf
What Language Model to Train if You Have One Million GPU Hours?
EMNLP (Findings)
Teven Le Scao, Thomas Wang, Daniel Hesslow, Lucile Saulnier, Stas Bekman, M. Saiful Bari, Stella Biderman, Hady Elsahar, Niklas Muennighoff, Jason Phang, Ofir Press, Colin Raffel, Victor Sanh, Sheng Shen, Lintang Sutawika, Jaesung Tae, Zheng Xin Yong, Julien Launay, Iz Beltagy
2022 J jnl
What Language Model to Train if You Have One Million GPU Hours?
CoRR
Teven Le Scao, Thomas Wang, Daniel Hesslow, Lucile Saulnier, Stas Bekman, M. Saiful Bari, Stella Biderman, Hady Elsahar, Niklas Muennighoff, Jason Phang, Ofir Press, Colin Raffel, Victor Sanh, Sheng Shen, Lintang Sutawika, Jaesung Tae, Zheng Xin Yong, Julien Launay, Iz Beltagy
2022 J jnl
Z-ICL: Zero-Shot In-Context Learning with Pseudo-Demonstrations.
CoRR
Xinxi Lyu, Sewon Min, Iz Beltagy, Luke Zettlemoyer, Hannaneh Hajishirzi
2022 — conf
Zero- and Few-Shot NLP with Pretrained Language Models.
ACL (tutorial)
Iz Beltagy, Arman Cohan, Robert L. Logan IV, Sewon Min, Sameer Singh
2021 — conf
A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers.
NAACL-HLT
Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, Matt Gardner
2021 J jnl
A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers.
CoRR
Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, Matt Gardner
2021 — conf
CDLM: Cross-Document Language Modeling.
EMNLP (Findings)
Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew E. Peters, Arie Cattan, Ido Dagan
2021 J jnl
Cross-Document Language Modeling.
CoRR
Avi Caciularu, Arman Cohan, Iz Beltagy, Matthew E. Peters, Arie Cattan, Ido Dagan
2021 A* conf
FLEX: Unifying Evaluation for Few-Shot NLP.
NeurIPS
Jonathan Bragg, Arman Cohan, Kyle Lo, Iz Beltagy
2021 J jnl
FLEX: Unifying Evaluation for Few-Shot NLP.
CoRR
Jonathan Bragg, Arman Cohan, Kyle Lo, Iz Beltagy
2021 J jnl
Few-Shot Self-Rationalization with Natural Language Prompts.
CoRR
Ana Marasovic, Iz Beltagy, Doug Downey, Matthew E. Peters
2021 J jnl
LongChecker: Improving scientific claim verification by modeling full-abstract context.
CoRR
David Wadden, Kyle Lo, Lucy Lu Wang, Arman Cohan, Iz Beltagy, Hannaneh Hajishirzi
2021 J jnl
MS2: Multi-Document Summarization of Medical Studies.
CoRR
Jay DeYoung, Iz Beltagy, Madeleine van Zuylen, Bailey Kuehl, Lucy Lu Wang
2021 — conf
MS\^2: Multi-Document Summarization of Medical Studies.
EMNLP (1)
Jay DeYoung, Iz Beltagy, Madeleine van Zuylen, Bailey Kuehl, Lucy Lu Wang
2021 J jnl
PRIMER: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization.
CoRR
Wen Xiao, Iz Beltagy, Giuseppe Carenini, Arman Cohan
2021 — ed.
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021
NAACL-HLT
Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, Yichao Zhou
2021 — conf
SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts.
AKBC
Arie Cattan, Sophie Johnson, Daniel S. Weld, Ido Dagan, Iz Beltagy, Doug Downey, Tom Hope
2021 J jnl
SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts.
CoRR
Arie Cattan, Sophie Johnson, Daniel S. Weld, Ido Dagan, Iz Beltagy, Doug Downey, Tom Hope
2021 — conf
Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study.
AKBC
Rahul Nadkarni, David Wadden, Iz Beltagy, Noah A. Smith, Hannaneh Hajishirzi, Tom Hope
2021 J jnl
Scientific Language Models for Biomedical Knowledge Base Completion: An Empirical Study.
CoRR
Rahul Nadkarni, David Wadden, Iz Beltagy, Noah A. Smith, Hannaneh Hajishirzi, Tom Hope
2020 A* conf
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks.
ACL
Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, Noah A. Smith
2020 J jnl
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks.
CoRR
Suchin Gururangan, Ana Marasovic, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, Noah A. Smith
2020 J jnl
Longformer: The Long-Document Transformer.
CoRR
Iz Beltagy, Matthew E. Peters, Arman Cohan
2020 A* conf
SPECTER: Document-level Representation Learning using Citation-informed Transformers.
ACL
Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, Daniel S. Weld
2020 J jnl
SPECTER: Document-level Representation Learning using Citation-informed Transformers.
CoRR
Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, Daniel S. Weld
2020 A* conf
SciREX: A Challenge Dataset for Document-Level Information Extraction.
ACL
Sarthak Jain, Madeleine van Zuylen, Hannaneh Hajishirzi, Iz Beltagy
2020 J jnl
SciREX: A Challenge Dataset for Document-Level Information Extraction.
CoRR
Sarthak Jain, Madeleine van Zuylen, Hannaneh Hajishirzi, Iz Beltagy
2019 — conf
Combining Distant and Direct Supervision for Neural Relation Extraction.
NAACL-HLT (1)
Iz Beltagy, Kyle Lo, Waleed Ammar
2019 — conf
Pretrained Language Models for Sequential Sentence Classification.
EMNLP/IJCNLP (1)
Arman Cohan, Iz Beltagy, Daniel King, Bhavana Dalvi, Daniel S. Weld
2019 J jnl
Pretrained Language Models for Sequential Sentence Classification.
CoRR
Arman Cohan, Iz Beltagy, Daniel King, Bhavana Dalvi, Daniel S. Weld
2019 — conf
SciBERT: A Pretrained Language Model for Scientific Text.
EMNLP/IJCNLP (1)
Iz Beltagy, Kyle Lo, Arman Cohan
2019 J jnl
SciBERT: Pretrained Contextualized Embeddings for Scientific Text.
CoRR
Iz Beltagy, Arman Cohan, Kyle Lo
2019 — conf
ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing.
BioNLP@ACL
Mark Neumann, Daniel King, Iz Beltagy, Waleed Ammar
2019 J jnl
ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing.
CoRR
Mark Neumann, Daniel King, Iz Beltagy, Waleed Ammar
2018 — conf
Construction of the Literature Graph in Semantic Scholar.
NAACL-HLT (3)
Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler Murray, Hsu-Han Ooi, Matthew E. Peters, Joanna Power, Sam Skjonsberg, Lucy Lu Wang, Chris Wilhelm, Zheng Yuan, Madeleine van Zuylen, Oren Etzioni
2018 J jnl
Construction of the Literature Graph in Semantic Scholar.
CoRR
Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler Murray, Hsu-Han Ooi, Matthew E. Peters, Joanna Power, Sam Skjonsberg, Lucy Lu Wang, Chris Wilhelm, Zheng Yuan, Madeleine van Zuylen, Oren Etzioni
2018 J jnl
Improving Distant Supervision with Maxpooled Attention and Sentence-Level Supervision.
CoRR
Iz Beltagy, Kyle Lo, Waleed Ammar
2018 — conf
Machine Learning for Reliable MMWAVE Systems: Blockage Prediction and Proactive Handoff.
GlobalSIP
Ahmed Alkhateeb, Iz Beltagy, Sam Alex
2018 J jnl
Machine Learning for Reliable mmWave Systems: Blockage Prediction and Proactive Handoff.
CoRR
Ahmed Alkhateeb, Iz Beltagy