Armen Aghajanyan

56 papers A* 11Journal 35Unranked 10

Year	Rank	Type	Title / Venue / Authors
2026	J	`jnl`	Improving MoE Compute Efficiency by Composing Weight and Data Sparsity. CoRR Maciej Kilian, Oleg Mkrtchyan, Luke Zettlemoyer, Akshat Shrivastava, Armen Aghajanyan
2026	J	`jnl`	SkillRater: Untangling Capabilities in Multimodal Data. CoRR Naveen Sahi, Jeremy Dohmann, Armen Aghajanyan, Akshat Shrivastava
2025	J	`jnl`	Text Quality-Based Pruning for Efficient Training of Language Models. J. Data-centric Mach. Learn. Res. Vasu Sharma, Karthik Padthe, Newsha Ardalani, Kushal Tirumala, Russell Howes, Hu Xu, Po-Yao Huang, Daniel Li Chen, Armen Aghajanyan, Gargi Ghosh, Luke Zettlemoyer
2024	J	`jnl`	BartSmiles: Generative Masked Language Models for Molecular Representations. J. Chem. Inf. Model. Gayane Chilingaryan, Hovhannes Tamoyan, Ani Tevosyan, Nelly Babayan, Karen Hambardzumyan, Zaven Navoyan, Armen Aghajanyan, Hrant Khachatrian, Lusine Khondkaryan
2024	A*	`conf`	Jointly Training Large Autoregressive Multimodal Models. ICLR Emanuele Aiello, Lili Yu, Yixin Nie, Armen Aghajanyan, Barlas Oguz
2024	J	`jnl`	MoMa: Efficient Early-Fusion Pre-training with Mixture of Modality-Aware Experts. CoRR Xi Victoria Lin, Akshat Shrivastava, Liang Luo, Srinivasan Iyer, Mike Lewis, Gargi Ghosh, Luke Zettlemoyer, Armen Aghajanyan
2024	J	`jnl`	Small Molecule Optimization with Large Language Models. CoRR Philipp Guevorguian, Menua Bedrosian, Tigran Fahradyan, Gayane Chilingaryan, Hrant Khachatrian, Armen Aghajanyan
2024	J	`jnl`	Text Quality-Based Pruning for Efficient Training of Language Models. CoRR Vasu Sharma, Karthik Padthe, Newsha Ardalani, Kushal Tirumala, Russell Howes, Hu Xu, Po-Yao Huang, Shang-wen Li, Armen Aghajanyan, Gargi Ghosh, Luke Zettlemoyer
2024	J	`jnl`	When Worse is Better: Navigating the compression-generation tradeoff in visual tokenization. CoRR Vivek Ramanujan, Kushal Tirumala, Armen Aghajanyan, Luke Zettlemoyer, Ali Farhadi
2023	A*	`conf`	D4: Improving LLM Pretraining via Document De-Duplication and Diversification. NeurIPS Kushal Tirumala, Daniel Simig, Armen Aghajanyan, Ari Morcos
2023	J	`jnl`	D4: Improving LLM Pretraining via Document De-Duplication and Diversification. CoRR Kushal Tirumala, Daniel Simig, Armen Aghajanyan, Ari S. Morcos
2023	J	`jnl`	DOMINO: A Dual-System for Multi-step Visual Language Reasoning. CoRR Peifang Wang, Olga Golovneva, Armen Aghajanyan, Xiang Ren, Muhao Chen, Asli Celikyilmaz, Maryam Fazel-Zarandi
2023	A*	`conf`	InCoder: A Generative Model for Code Infilling and Synthesis. ICLR Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Scott Yih, Luke Zettlemoyer, Mike Lewis
2023	J	`jnl`	Jointly Training Large Autoregressive Multimodal Models. CoRR Emanuele Aiello, Lili Yu, Yixin Nie, Armen Aghajanyan, Barlas Oguz
2023	A*	`conf`	MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers. NeurIPS Lili Yu, Daniel Simig, Colin Flaherty, Armen Aghajanyan, Luke Zettlemoyer, Mike Lewis
2023	J	`jnl`	MEGABYTE: Predicting Million-byte Sequences with Multiscale Transformers. CoRR Lili Yu, Daniel Simig, Colin Flaherty, Armen Aghajanyan, Luke Zettlemoyer, Mike Lewis
2023	A*	`conf`	Retrieval-Augmented Multimodal Language Modeling. ICML Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Richard James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer, Wen-tau Yih
2023	J	`jnl`	Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning. CoRR Lili Yu, Bowen Shi, Ramakanth Pasunuru, Benjamin Muller, Olga Golovneva, Tianlu Wang, Arun Babu, Binh Tang, Brian Karrer, Shelly Sheynin, Candace Ross, Adam Polyak, Russell Howes, Vasu Sharma, Puxin Xu, Hovhannes Tamoyan, Oron Ashual, Uriel Singer, Shang-wen Li, Susan Zhang, Richard James, Gargi Ghosh, Yaniv Taigman, Maryam Fazel-Zarandi, Asli Celikyilmaz, Luke Zettlemoyer, Armen Aghajanyan
2023	A*	`conf`	Scaling Laws for Generative Mixed-Modal Language Models. ICML Armen Aghajanyan, Lili Yu, Alexis Conneau, Wei-Ning Hsu, Karen Hambardzumyan, Susan Zhang, Stephen Roller, Naman Goyal, Omer Levy, Luke Zettlemoyer
2023	J	`jnl`	Scaling Laws for Generative Mixed-Modal Language Models. CoRR Armen Aghajanyan, Lili Yu, Alexis Conneau, Wei-Ning Hsu, Karen Hambardzumyan, Susan Zhang, Stephen Roller, Naman Goyal, Omer Levy, Luke Zettlemoyer
2022	J	`jnl`	BARTSmiles: Generative Masked Language Models for Molecular Representations. CoRR Gayane Chilingaryan, Hovhannes Tamoyan, Ani Tevosyan, Nelly Babayan, Lusine Khondkaryan, Karen Hambardzumyan, Zaven Navoyan, Hrant Khachatrian, Armen Aghajanyan
2022	—	`conf`	CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training. NAACL-HLT (Findings) Patrick Huber, Armen Aghajanyan, Barlas Oguz, Dmytro Okhonko, Scott Yih, Sonal Gupta, Xilun Chen
2022	J	`jnl`	CM3: A Causal Masked Multimodal Model of the Internet. CoRR Armen Aghajanyan, Bernie Huang, Candace Ross, Vladimir Karpukhin, Hu Xu, Naman Goyal, Dmytro Okhonko, Mandar Joshi, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer
2022	A*	`conf`	HTLM: Hyper-Text Pre-Training and Prompting of Language Models. ICLR Armen Aghajanyan, Dmytro Okhonko, Mike Lewis, Mandar Joshi, Hu Xu, Gargi Ghosh, Luke Zettlemoyer
2022	A*	`conf`	Improving Passage Retrieval with Zero-Shot Question Generation. EMNLP Devendra Singh Sachan, Mike Lewis, Mandar Joshi, Armen Aghajanyan, Wen-tau Yih, Joelle Pineau, Luke Zettlemoyer
2022	J	`jnl`	Improving Passage Retrieval with Zero-Shot Question Generation. CoRR Devendra Singh Sachan, Mike Lewis, Mandar Joshi, Armen Aghajanyan, Wen-tau Yih, Joelle Pineau, Luke Zettlemoyer
2022	J	`jnl`	InCoder: A Generative Model for Code Infilling and Synthesis. CoRR Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, Mike Lewis
2022	A*	`conf`	Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models. NeurIPS Kushal Tirumala, Aram H. Markosyan, Luke Zettlemoyer, Armen Aghajanyan
2022	J	`jnl`	Memorization Without Overfitting: Analyzing the Training Dynamics of Large Language Models. CoRR Kushal Tirumala, Aram H. Markosyan, Luke Zettlemoyer, Armen Aghajanyan
2022	J	`jnl`	Retrieval-Augmented Multimodal Language Modeling. CoRR Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer, Wen-tau Yih
2022	—	`conf`	RetroNLU: Retrieval Augmented Task-Oriented Semantic Parsing. ConvAI@ACL Vivek Gupta, Akshat Shrivastava, Adithya Sagar, Armen Aghajanyan, Denis Savenkov
2021	A*	`conf`	Better Fine-Tuning by Reducing Representational Collapse. ICLR Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, Sonal Gupta
2021	J	`jnl`	CCQA: A New Web-Scale Question Answering Dataset for Model Pre-Training. CoRR Patrick Huber, Armen Aghajanyan, Barlas Oguz, Dmytro Okhonko, Wen-tau Yih, Sonal Gupta, Xilun Chen
2021	J	`jnl`	HTLM: Hyper-Text Pre-Training and Prompting of Language Models. CoRR Armen Aghajanyan, Dmytro Okhonko, Mike Lewis, Mandar Joshi, Hu Xu, Gargi Ghosh, Luke Zettlemoyer
2021	—	`conf`	Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning. ACL/IJCNLP (1) Armen Aghajanyan, Sonal Gupta, Luke Zettlemoyer
2021	—	`conf`	Muppet: Massive Multi-task Representations with Pre-Finetuning. EMNLP (1) Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, Sonal Gupta
2021	J	`jnl`	Muppet: Massive Multi-task Representations with Pre-Finetuning. CoRR Armen Aghajanyan, Anchit Gupta, Akshat Shrivastava, Xilun Chen, Luke Zettlemoyer, Sonal Gupta
2021	—	`conf`	Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog. NAACL-HLT Arun Babu, Akshat Shrivastava, Armen Aghajanyan, Ahmed Aly, Angela Fan, Marjan Ghazvininejad
2021	J	`jnl`	Non-Autoregressive Semantic Parsing for Compositional Task-Oriented Dialog. CoRR Arun Babu, Akshat Shrivastava, Armen Aghajanyan, Ahmed Aly, Angela Fan, Marjan Ghazvininejad
2021	J	`jnl`	RETRONLU: Retrieval Augmented Task-Oriented Semantic Parsing. CoRR Vivek Gupta, Akshat Shrivastava, Adithya Sagar, Armen Aghajanyan, Denis Savenkov
2021	—	`conf`	VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding. EMNLP (1) Hu Xu, Gargi Ghosh, Po-Yao Huang, Dmytro Okhonko, Armen Aghajanyan, Florian Metze, Luke Zettlemoyer, Christoph Feichtenhofer
2021	J	`jnl`	VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding. CoRR Hu Xu, Gargi Ghosh, Po-Yao Huang, Dmytro Okhonko, Armen Aghajanyan, Florian Metze, Luke Zettlemoyer, Christoph Feichtenhofer
2020	J	`jnl`	Better Fine-Tuning by Reducing Representational Collapse. CoRR Armen Aghajanyan, Akshat Shrivastava, Anchit Gupta, Naman Goyal, Luke Zettlemoyer, Sonal Gupta
2020	—	`conf`	Conversational Semantic Parsing. EMNLP (1) Armen Aghajanyan, Jean Maillard, Akshat Shrivastava, Keith Diedrick, Michael Haeger, Haoran Li, Yashar Mehdad, Veselin Stoyanov, Anuj Kumar, Mike Lewis, Sonal Gupta
2020	J	`jnl`	Conversational Semantic Parsing. CoRR Armen Aghajanyan, Jean Maillard, Akshat Shrivastava, Keith Diedrick, Mike Haeger, Haoran Li, Yashar Mehdad, Ves Stoyanov, Anuj Kumar, Mike Lewis, Sonal Gupta
2020	J	`jnl`	Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning. CoRR Armen Aghajanyan, Luke Zettlemoyer, Sonal Gupta
2020	A*	`conf`	Pre-training via Paraphrasing. NeurIPS Mike Lewis, Marjan Ghazvininejad, Gargi Ghosh, Armen Aghajanyan, Sida Wang, Luke Zettlemoyer
2020	J	`jnl`	Pre-training via Paraphrasing. CoRR Mike Lewis, Marjan Ghazvininejad, Gargi Ghosh, Armen Aghajanyan, Sida I. Wang, Luke Zettlemoyer
2019	—	`conf`	Towards Language Agnostic Universal Representations. ACL (1) Armen Aghajanyan, Xia Song, Saurabh Tiwary
2018	J	`jnl`	Towards Language Agnostic Universal Representations. CoRR Armen Aghajanyan, Xia Song, Saurabh Tiwary
2017	—	`conf`	Charged Point Normalization: An Efficient Solution to the Saddle Point Problem. ICLR (Workshop) Armen Aghajanyan
2017	J	`jnl`	Convolution Aware Initialization. CoRR Armen Aghajanyan
2017	—	`conf`	SoftTarget Regularization: An Effective Technique to Reduce Over-Fitting in Neural Networks. CYBCONF Armen Aghajanyan
2016	J	`jnl`	Charged Point Normalization: An Efficient Solution to the Saddle Point Problem. CoRR Armen Aghajanyan
2016	J	`jnl`	SoftTarget Regularization: An Effective Technique to Reduce Over-Fitting in Neural Networks. CoRR Armen Aghajanyan
2015	J	`jnl`	Gravitational Clustering. CoRR Armen Aghajanyan