David Harwath - RankMe

130 papers A* 13A 20B 1C 4Misc 12Journal 63Unranked 17

Year	Rank	Type	Title / Venue / Authors
2026	J	`jnl`	Adapting Self-Supervised Speech Representations for Cross-lingual Dysarthria Detection in Parkinson's Disease. CoRR Abner Hernandez, Eunjung Yeo, Kwanghee Choi, Chin-Jou Li, Zhengjun Yue, Rohan Kumar Das, Jan Rusz, Mathew Magimai-Doss, Juan Rafael Orozco-Arroyave, Tomás Arias-Vergara, Andreas K. Maier, Elmar Nöth, David R. Mortensen, David Harwath, Paula Andrea Pérez-Toro
2026	J	`jnl`	Linear Script Representations in Speech Foundation Models Enable Zero-Shot Transliteration. CoRR Ryan Soh-Eun Shim, Kwanghee Choi, Kalvin Chang, Ming-Hao Hsu, Florian Eichin, Zhizheng Wu, Alane Suhr, Michael A. Hedderich, David Harwath, David R. Mortensen, Barbara Plank
2026	A*	`conf`	MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic Evaluation of Audio General Intelligence. AAAI Sonal Kumar, Simon Sedlácek, Vaibhavi Lokegaonkar, Fernando López, Wenyi Yu, Nishit Anand, Hyeonggon Ryu, Lichang Chen, Maxim Plicka, Miroslav Hlavácek, William Fineas Ellingwood, Sathvik Udupa, Siyuan Hou, Allison Ferner, Sara Barahona, Cecilia Bolaños, Satish Rahi, Laura Herrera-Alarcón, Satvik Dixit, Rupali S. Patil, Soham Deshmukh, Lasha Koroshinadze, Yao Liu, Leibny Paola García-Perera, Eleni Zanou, Themos Stafylakis, Joon Son Chung, David Harwath, Chao Zhang, Dinesh Manocha, Alicia Lozano-Diez, Santosh Kesiraju, Sreyan Ghosh, Ramani Duraiswami
2026	J	`jnl`	ParaSpeechCLAP: A Dual-Encoder Speech-Text Model for Rich Stylistic Language-Audio Pretraining. CoRR Anuj Diwan, Eunsol Choi, David Harwath
2026	J	`jnl`	Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspaces. CoRR Kwanghee Choi, Eunjung Yeo, Cheol Jun Cho, David R. Mortensen, David Harwath
2026	J	`jnl`	[b]=[d]-[t]+[p]: Self-supervised Speech Models Discover Phonological Vector Arithmetic. CoRR Kwanghee Choi, Eunjung Yeo, Cheol Jun Cho, David Harwath, David R. Mortensen
2025	C	`conf`	Codec2Vec: Self-Supervised Speech Representation Learning Using Neural Speech Codecs. ASRU Wei-Cheng Tseng, David Harwath
2025	J	`jnl`	Codec2Vec: Self-Supervised Speech Representation Learning Using Neural Speech Codecs. CoRR Wei-Cheng Tseng, David Harwath
2025	J	`jnl`	FacEDiT: Unified Talking Face Editing and Generation via Facial Motion Infilling. CoRR Sung-Bin Kim, Joohyun Chang, David Harwath, Tae-Hyun Oh
2025	J	`jnl`	MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic Evaluation of Audio General Intelligence. CoRR Sonal Kumar, Simon Sedlácek, Vaibhavi Lokegaonkar, Fernando López, Wenyi Yu, Nishit Anand, Hyeonggon Ryu, Lichang Chen, Maxim Plicka, Miroslav Hlavácek, William Fineas Ellingwood, Sathvik Udupa, Siyuan Hou, Allison Ferner, Sara Barahona, Cecilia Bolaños, Satish Rahi, Laura Herrera-Alarcón, Satvik Dixit, Rupali S. Patil, Soham Deshmukh, Lasha Koroshinadze, Yao Liu, Leibny Paola García-Perera, Eleni Zanou, Themos Stafylakis, Joon Son Chung, David Harwath, Chao Zhang, Dinesh Manocha, Alicia Lozano-Diez, Santosh Kesiraju, Sreyan Ghosh, Ramani Duraiswami
2025	A	`conf`	Probing the Robustness Properties of Neural Speech Codecs. INTERSPEECH Wei-Cheng Tseng, David Harwath
2025	J	`jnl`	Probing the Robustness Properties of Neural Speech Codecs. CoRR Wei-Cheng Tseng, David Harwath
2025	J	`jnl`	Rhapsody: A Dataset for Highlight Detection in Podcasts. CoRR Younghan Park, Anuj Diwan, David Harwath, Eunsol Choi
2025	J	`jnl`	RosettaSpeech: Zero-Shot Speech-to-Speech Translation from Monolingual Data. CoRR Zhisheng Zheng, Xiaohang Sun, Tuan Dinh, Abhishek Yanamandra, Abhinav Jain, Zhu Liu, Sunil Hadap, Vimal Bhat, Manoj Aggarwal, Gérard G. Medioni, David Harwath
2025	A*	`conf`	Scaling Rich Style-Prompted Text-to-Speech Datasets. EMNLP Anuj Diwan, Zhisheng Zheng, David Harwath, Eunsol Choi
2025	J	`jnl`	Scaling Rich Style-Prompted Text-to-Speech Datasets. CoRR Anuj Diwan, Zhisheng Zheng, David Harwath, Eunsol Choi
2025	A*	`conf`	SyllableLM: Learning Coarse Semantic Units for Speech Language Models. ICLR Alan Baade, Puyuan Peng, David Harwath
2025	A	`conf`	Temporally Streaming Audio-Visual Synchronization for Real-World Videos. WACV Jordan Voas, Wei-Cheng Tseng, Layne Berry, Xixi Hu, Puyuan Peng, James Stuedemann, David Harwath
2025	J	`jnl`	Towards Unsupervised Speech Recognition at the Syllable-Level. CoRR Liming Wang, Junrui Ni, Kai-Wei Chang, Saurabhchand Bhati, David Harwath, Mark Hasegawa-Johnson, James R. Glass
2025	C	`conf`	Unifying Model and Layer Fusion for Speech Foundation Models. ASRU Yi-Jen Shih, David Harwath
2025	J	`jnl`	Unifying Model and Layer Fusion for Speech Foundation Models. CoRR Yi-Jen Shih, David Harwath
2025	J	`jnl`	VoiceCraft-Dub: Automated Video Dubbing with Neural Codec Language Models. CoRR Sung-Bin Kim, Jeongsoo Choi, Puyuan Peng, Joon Son Chung, Tae-Hyun Oh, David Harwath
2025	A*	`conf`	VoiceCraft-X: Unifying Multilingual, Voice-Cloning Speech Synthesis and Speech Editing. EMNLP Zhisheng Zheng, Puyuan Peng, Anuj Diwan, Cong Phuoc Huynh, Xiaohang Sun, Zhu Liu, Vimal Bhat, David Harwath
2025	J	`jnl`	VoiceCraft-X: Unifying Multilingual, Voice-Cloning Speech Synthesis and Speech Editing. CoRR Zhisheng Zheng, Puyuan Peng, Anuj Diwan, Cong Phuoc Huynh, Xiaohang Sun, Zhu Liu, Vimal Bhat, David Harwath
2025	J	`jnl`	VoiceStar: Robust Zero-Shot Autoregressive TTS with Duration Control and Extrapolation. CoRR Puyuan Peng, Shang-wen Li, Abdelrahman Mohamed, David Harwath
2024	Misc	`conf`	AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models. ICASSP Yuan Tseng, Layne Berry, Yiting Chen, I-Hsiang Chiu, Hsuan-Hao Lin, Max Liu, Puyuan Peng, Yi-Jen Shih, Hung-Yu Wang, Haibin Wu, Poyao Huang, Chun-Mao Lai, Shang-wen Li, David Harwath, Yu Tsao, Abdelrahman Mohamed, Chi-Luen Feng, Hung-yi Lee
2024	—	`conf`	Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos. ECCV (70) Changan Chen, Puyuan Peng, Ami Baid, Zihui Xue, Wei-Ning Hsu, David Harwath, Kristen Grauman
2024	J	`jnl`	Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos. CoRR Changan Chen, Puyuan Peng, Ami Baid, Zihui Xue, Wei-Ning Hsu, David Harwath, Kristen Grauman
2024	A*	`conf`	BAT: Learning to Reason about Spatial Sounds with Large Language Models. ICML Zhisheng Zheng, Puyuan Peng, Ziyang Ma, Xie Chen, Eunsol Choi, David Harwath
2024	J	`jnl`	BAT: Learning to Reason about Spatial Sounds with Large Language Models. CoRR Zhisheng Zheng, Puyuan Peng, Ziyang Ma, Xie Chen, Eunsol Choi, David Harwath
2024	A	`conf`	Direct Speech Synthesis from Non-Invasive, Neuromagnetic Signals. INTERSPEECH Jinuk Kwon, David Harwath, Debadatta Dash, Paul Ferrari, Jun Wang
2024	J	`jnl`	Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks. CoRR Chien-yu Huang, Wei-Chih Chen, Shu-Wen Yang, Andy T. Liu, Chen-An Li, Yu-Xiang Lin, Wei-Cheng Tseng, Anuj Diwan, Yi-Jen Shih, Jiatong Shi, William Chen, Xuanjun Chen, Chi-Yuan Hsiao, Puyuan Peng, Shih-Heng Wang, Chun-Yi Kuan, Ke-Han Lu, Kai-Wei Chang, Chih-Kai Yang, Fabian Ritter Gutierrez, Ming To Chuang, Kuan-Po Huang, Siddhant Arora, You-Kuan Lin, Eunjung Yeo, Kalvin Chang, Chung-Ming Chien, Kwanghee Choi, Cheng-Hsiu Hsieh, Yi-Cheng Lin, Chee-En Yu, I-Hsiang Chiu, Heitor R. Guimarães, Jionghao Han, Tzu-Quan Lin, Tzu-Yuan Lin, Homu Chang, Ting-Wu Chang, Chun Wei Chen, Shou-Jen Chen, Yu-Hua Chen, Hsi-Chun Cheng, Kunal Dhawan, Jia-Lin Fang, Shi-Xin Fang, Kuan-Yu Fang Chiang, Chi An Fu, Hsien-Fu Hsiao, Ching Yu Hsu, Shao-Syuan Huang, Lee Chen Wei, Hsi-Che Lin, Hsuan-Hao Lin, Hsuan-Ting Lin, Jian-Ren Lin, Ting-Chun Liu, Li-Chun Lu, Tsung-Min Pai, Ankita Pasad, Shih-Yun Shan Kuan, Suwon Shon, Yuxun Tang, Yun-Shao Tsai, Jui-Chiang Wei, Tzu-Chieh Wei, Chengxi Wu, Dien-Ruei Wu, Chao-Han Huck Yang, Chieh-Chi Yang, Jia Qi Yip, Shao-Xiang Yuan, Vahid Noroozi, Zhehuai Chen, Haibin Wu, Karen Livescu, David Harwath, Shinji Watanabe, Hung-yi Lee
2024	J	`jnl`	How to Learn a New Language? An Efficient Solution for Self-Supervised Learning Models Unseen Languages Adaption in Low-Resource Scenario. CoRR Shih-Heng Wang, Zih-Ching Chen, Jiatong Shi, Ming-To Chuang, Guan-Ting Lin, Kuan-Po Huang, David Harwath, Shang-wen Li, Hung-yi Lee
2024	A	`conf`	Improving Audio Classification with Low-Sampled Microphone Input: An Empirical Study Using Model Self-Distillation. INTERSPEECH Dawei Liang, Alice Zhang, David Harwath, Edison Thomaz
2024	—	`conf`	Integrating Self-Supervised Speech Model with Pseudo Word-Level Targets from Visually-Grounded Speech Model. ICASSP Workshops Hung-Chieh Fang, Nai-Xuan Ye, Yi-Jen Shih, Puyuan Peng, Hsuan-Fu Wang, Layne Berry, Hung-yi Lee, David Harwath
2024	J	`jnl`	Integrating Self-supervised Speech Model with Pseudo Word-level Targets from Visually-grounded Speech Model. CoRR Hung-Chieh Fang, Nai-Xuan Ye, Yi-Jen Shih, Puyuan Peng, Hsuan-Fu Wang, Layne Berry, Hung-yi Lee, David Harwath
2024	A	`conf`	Interface Design for Self-Supervised Speech Models. INTERSPEECH Yi-Jen Shih, David Harwath
2024	J	`jnl`	Interface Design for Self-Supervised Speech Models. CoRR Yi-Jen Shih, David Harwath
2024	—	`conf`	Measuring Sound Symbolism In Audio-Visual Models. SLT Wei-Cheng Tseng, Yi-Jen Shih, David Harwath, Raymond Mooney
2024	—	`conf`	Multimodal Contextualized Semantic Parsing from Speech. ACL (1) Jordan Voas, David Harwath, Raymond Mooney
2024	J	`jnl`	Multimodal Contextualized Semantic Parsing from Speech. CoRR Jordan Voas, Raymond Mooney, David Harwath
2024	A	`conf`	Neural Codec Language Models for Disentangled and Textless Voice Conversion. INTERSPEECH Alan Baade, Puyuan Peng, David Harwath
2024	—	`conf`	Self-Supervised Speech Models For Word-Level Stuttered Speech Detection. SLT Yi-Jen Shih, Zoi Gkalitsiou, Alexandros G. Dimakis, David Harwath
2024	J	`jnl`	Self-supervised Speech Models for Word-Level Stuttered Speech Detection. CoRR Yi-Jen Shih, Zoi Gkalitsiou, Alexandros G. Dimakis, David Harwath
2024	A*	`conf`	SoundingActions: Learning How Actions Sound from Narrated Egocentric Videos. CVPR Changan Chen, Kumar Ashutosh, Rohit Girdhar, David Harwath, Kristen Grauman
2024	J	`jnl`	SoundingActions: Learning How Actions Sound from Narrated Egocentric Videos. CoRR Changan Chen, Kumar Ashutosh, Rohit Girdhar, David Harwath, Kristen Grauman
2024	—	`conf`	SpeechCLIP+: Self-Supervised Multi-Task Representation Learning for Speech Via Clip and Speech-Image Data. ICASSP Workshops Hsuan-Fu Wang, Yi-Jen Shih, Heng-Jui Chang, Layne Berry, Puyuan Peng, Hung-yi Lee, Hsin-Min Wang, David Harwath
2024	J	`jnl`	SpeechCLIP+: Self-supervised multi-task representation learning for speech via CLIP and speech-image data. CoRR Hsuan-Fu Wang, Yi-Jen Shih, Heng-Jui Chang, Layne Berry, Puyuan Peng, Hung-yi Lee, Hsin-Min Wang, David Harwath
2024	J	`jnl`	SyllableLM: Learning Coarse Semantic Units for Speech Language Models. CoRR Alan Baade, Puyuan Peng, David Harwath
2024	—	`conf`	Textless Speech-to-Speech Translation With Limited Parallel Data. EMNLP (Findings) Anuj Diwan, Anirudh Srinivasan, David Harwath, Eunsol Choi
2024	—	`conf`	VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild. ACL (1) Puyuan Peng, Po-Yao Huang, Shang-wen Li, Abdelrahman Mohamed, David Harwath
2024	J	`jnl`	VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild. CoRR Puyuan Peng, Po-Yao Huang, Daniel Li, Abdelrahman Mohamed, David Harwath
2023	—	`conf`	A Dataset for Foreground Speech Analysis With Smartwatches In Everyday Home Environments. ICASSP Workshops Dawei Liang, Zifan Xu, Yinuo Chen, Rebecca Adaimi, David Harwath, Edison Thomaz
2023	J	`jnl`	AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models. CoRR Yuan Tseng, Layne Berry, Yi-Ting Chen, I-Hsiang Chiu, Hsuan-Hao Lin, Max Liu, Puyuan Peng, Yi-Jen Shih, Hung-Yu Wang, Haibin Wu, Po-Yao Huang, Chun-Mao Lai, Shang-wen Li, David Harwath, Yu Tsao, Shinji Watanabe, Abdelrahman Mohamed, Chi-Luen Feng, Hung-yi Lee
2023	C	`conf`	Audio-Visual Neural Syntax Acquisition. ASRU Cheng-I Jeff Lai, Freda Shi, Puyuan Peng, Yoon Kim, Kevin Gimpel, Shiyu Chang, Yung-Sung Chuang, Saurabhchand Bhati, David D. Cox, David Harwath, Yang Zhang, Karen Livescu, James R. Glass
2023	J	`jnl`	Audio-Visual Neural Syntax Acquisition. CoRR Cheng-I Jeff Lai, Freda Shi, Puyuan Peng, Yoon Kim, Kevin Gimpel, Shiyu Chang, Yung-Sung Chuang, Saurabhchand Bhati, David D. Cox, David Harwath, Yang Zhang, Karen Livescu, James R. Glass
2023	Misc	`conf`	C2KD: Cross-Lingual Cross-Modal Knowledge Distillation for Multilingual Text-Video Retrieval. ICASSP Andrew Rouditchenko, Yung-Sung Chuang, Nina Shvetsova, Samuel Thomas, Rogério Feris, Brian Kingsbury, Leonid Karlinsky, David Harwath, Hilde Kuehne, James R. Glass
2023	A	`conf`	Comparison of Multilingual Self-Supervised and Weakly-Supervised Speech Pre-Training for Adaptation to Unseen Languages. INTERSPEECH Andrew Rouditchenko, Sameer Khurana, Samuel Thomas, Rogério Feris, Leonid Karlinsky, Hilde Kuehne, David Harwath, Brian Kingsbury, James R. Glass
2023	J	`jnl`	Comparison of Multilingual Self-Supervised and Weakly-Supervised Speech Pre-Training for Adaptation to Unseen Languages. CoRR Andrew Rouditchenko, Sameer Khurana, Samuel Thomas, Rogério Feris, Leonid Karlinsky, Hilde Kuehne, David Harwath, Brian Kingsbury, James R. Glass
2023	Misc	`conf`	Continual Learning for On-Device Speech Recognition Using Disentangled Conformers. ICASSP Anuj Diwan, Ching-Feng Yeh, Wei-Ning Hsu, Paden Tomasello, Eunsol Choi, David Harwath, Abdelrahman Mohamed
2023	A*	`conf`	Contrastive Audio-Visual Masked Autoencoder. ICLR Yuan Gong, Andrew Rouditchenko, Alexander H. Liu, David Harwath, Leonid Karlinsky, Hilde Kuehne, James R. Glass
2023	Misc	`conf`	Learning Audio-Visual Dereverberation. ICASSP Changan Chen, Wei Sun, David Harwath, Kristen Grauman
2023	A	`conf`	Learning to Map Efficiently by Active Echolocation. IROS Xixi Hu, Senthil Purushwalkam, David Harwath, Kristen Grauman
2023	Misc	`conf`	M-SpeechCLIP: Leveraging Large-Scale, Pre-Trained Models for Multilingual Speech to Image Retrieval. ICASSP Layne Berry, Yi-Jen Shih, Hsuan-Fu Wang, Heng-Jui Chang, Hung-yi Lee, David Harwath
2023	A	`conf`	Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalization. INTERSPEECH Puyuan Peng, Brian Yan, Shinji Watanabe, David Harwath
2023	J	`jnl`	Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalization. CoRR Puyuan Peng, Brian Yan, Shinji Watanabe, David Harwath
2023	J	`jnl`	Style-transfer based Speech and Audio-visual Scene Understanding for Robot Action Sequence Acquisition from Videos. CoRR Chiori Hori, Puyuan Peng, David Harwath, Xinyu Liu, Kei Ota, Siddarth Jain, Radu Corcodel, Devesh K. Jha, Diego Romeres, Jonathan Le Roux
2023	A	`conf`	Style-transfer based Speech and Audio-visual Scene understanding for Robot Action Sequence Acquisition from Videos. INTERSPEECH Chiori Hori, Puyuan Peng, David Harwath, Xinyu Liu, Kei Ota, Siddarth Jain, Radu Corcodel, Devesh K. Jha, Diego Romeres, Jonathan Le Roux
2023	—	`conf`	Subject Generalization in Classifying Imagined and Spoken Speech with MEG. NER Debadatta Dash, Paul Ferrari, Abbas Babajani-Feremi, David Harwath, Amir Borna, Jun Wang
2023	J	`jnl`	Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Mode. CoRR Puyuan Peng, Shang-wen Li, Okko Räsänen, Abdelrahman Mohamed, David Harwath
2023	A	`conf`	Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model. INTERSPEECH Puyuan Peng, Shang-wen Li, Okko Räsänen, Abdelrahman Mohamed, David Harwath
2023	J	`jnl`	Unit-based Speech-to-Speech Translation Without Parallel Data. CoRR Anuj Diwan, Anirudh Srinivasan, David Harwath, Eunsol Choi
2023	Misc	`conf`	Unsupervised Fine-Tuning Data Selection for ASR Using Self-Supervised Speech Models. ICASSP Reem Gody, David Harwath
2023	—	`conf`	When to Use Efficient Self Attention? Profiling Text, Speech and Image Transformer Variants. ACL (2) Anuj Diwan, Eunsol Choi, David Harwath
2023	J	`jnl`	When to Use Efficient Self Attention? Profiling Text, Speech and Image Transformer Variants. CoRR Anuj Diwan, Eunsol Choi, David Harwath
2022	Misc	`conf`	Adversarial Input Ablation for Audio-Visual Learning. ICASSP David Xu, David Harwath
2022	J	`jnl`	Automated detection of foreground speech with wearable sensing in everyday home environments: A transfer learning approach. CoRR Dawei Liang, Zifan Xu, Yinuo Chen, Rebecca Adaimi, David Harwath, Edison Thomaz
2022	J	`jnl`	C2KD: Cross-Lingual Cross-Modal Knowledge Distillation for Multilingual Text-Video Retrieval. CoRR Andrew Rouditchenko, Yung-Sung Chuang, Nina Shvetsova, Samuel Thomas, Rogério Feris, Brian Kingsbury, Leonid Karlinsky, David Harwath, Hilde Kuehne, James R. Glass
2022	J	`jnl`	Continual Learning for On-Device Speech Recognition using Disentangled Conformers. CoRR Anuj Diwan, Ching-Feng Yeh, Wei-Ning Hsu, Paden Tomasello, Eunsol Choi, David Harwath, Abdelrahman Mohamed
2022	J	`jnl`	Contrastive Audio-Visual Masked Autoencoder. CoRR Yuan Gong, Andrew Rouditchenko, Alexander H. Liu, David Harwath, Leonid Karlinsky, Hilde Kuehne, James R. Glass
2022	A*	`conf`	Everything at Once - Multi-modal Fusion Transformer for Video Retrieval. CVPR Nina Shvetsova, Brian Chen, Andrew Rouditchenko, Samuel Thomas, Brian Kingsbury, Rogério Feris, David Harwath, James R. Glass, Hilde Kuehne
2022	A	`conf`	Exploring Few-Shot Fine-Tuning Strategies for Models of Visually Grounded Speech. INTERSPEECH Tyler Miller, David Harwath
2022	Misc	`conf`	Fast-Slow Transformer for Visually Grounding Speech. ICASSP Puyuan Peng, David Harwath
2022	J	`jnl`	M-SpeechCLIP: Leveraging Large-Scale, Pre-Trained Models for Multilingual Speech to Image Retrieval. CoRR Layne Berry, Yi-Jen Shih, Hsuan-Fu Wang, Heng-Jui Chang, Hung-yi Lee, David Harwath
2022	A	`conf`	MAE-AST: Masked Autoencoding Audio Spectrogram Transformer. INTERSPEECH Alan Baade, Puyuan Peng, David Harwath
2022	J	`jnl`	MAE-AST: Masked Autoencoding Audio Spectrogram Transformer. CoRR Alan Baade, Puyuan Peng, David Harwath
2022	—	`conf`	Phoneme Segmentation Using Self-Supervised Speech Models. SLT Luke Strgar, David Harwath
2022	J	`jnl`	Phoneme Segmentation Using Self-Supervised Speech Models. CoRR Luke Strgar, David Harwath
2022	J	`jnl`	Self-Supervised Representation Learning for Speech Using Visual Grounding and Masked Language Modeling. CoRR Puyuan Peng, David Harwath
2022	B	`conf`	Speak: A Toolkit Using Amazon Mechanical Turk to Collect and Validate Speech Audio Recordings. LREC Christopher Song, David Harwath, Tuka Alhanai, James R. Glass
2022	—	`conf`	SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Model. SLT Yi-Jen Shih, Hsuan-Fu Wang, Heng-Jui Chang, Layne Berry, Hung-yi Lee, David Harwath
2022	J	`jnl`	SpeechCLIP: Integrating Speech with Pre-Trained Vision and Language Model. CoRR Yi-Jen Shih, Hsuan-Fu Wang, Heng-Jui Chang, Layne Berry, Hung-yi Lee, David Harwath
2022	J	`jnl`	Unsupervised Fine-Tuning Data Selection for ASR Using Self-Supervised Speech Models. CoRR Reem Gody, David Harwath
2022	A*	`conf`	Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionality. EMNLP Anuj Diwan, Layne Berry, Eunsol Choi, David Harwath, Kyle Mahowald
2022	J	`jnl`	Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionality. CoRR Anuj Diwan, Layne Berry, Eunsol Choi, David Harwath, Kyle Mahowald
2022	A	`conf`	Word Discovery in Visually Grounded, Self-Supervised Speech Models. INTERSPEECH Puyuan Peng, David Harwath
2022	J	`jnl`	Word Discovery in Visually Grounded, Self-Supervised Speech Models. CoRR Puyuan Peng, David Harwath
2021	A	`conf`	AVLnet: Learning Audio-Visual Language Representations from Instructional Videos. Interspeech Andrew Rouditchenko, Angie W. Boggust, David Harwath, Brian Chen, Dhiraj Joshi, Samuel Thomas, Kartik Audhkhasi, Hilde Kuehne, Rameswar Panda, Rogério Schmidt Feris, Brian Kingsbury, Michael Picheny, Antonio Torralba, James R. Glass
2021	A	`conf`	Cascaded Multilingual Audio-Visual Learning from Videos. Interspeech Andrew Rouditchenko, Angie W. Boggust, David Harwath, Samuel Thomas, Hilde Kuehne, Brian Chen, Rameswar Panda, Rogério Feris, Brian Kingsbury, Michael Picheny, James R. Glass
2021	J	`jnl`	Cascaded Multilingual Audio-Visual Learning from Videos. CoRR Andrew Rouditchenko, Angie W. Boggust, David Harwath, Samuel Thomas, Hilde Kuehne, Brian Chen, Rameswar Panda, Rogério Feris, Brian Kingsbury, Michael Picheny, James R. Glass
2021	J	`jnl`	Everything at Once - Multi-modal Fusion Transformer for Video Retrieval. CoRR Nina Shvetsova, Brian Chen, Andrew Rouditchenko, Samuel Thomas, Brian Kingsbury, Rogério Feris, David Harwath, James R. Glass, Hilde Kuehne
2021	J	`jnl`	Fast-Slow Transformer for Visually Grounding Speech. CoRR Puyuan Peng, David Harwath
2021	J	`jnl`	Learning Audio-Visual Dereverberation. CoRR Changan Chen, Wei Sun, David Harwath, Kristen Grauman
2021	A*	`conf`	Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videos. ICCV Brian Chen, Andrew Rouditchenko, Kevin Duarte, Hilde Kuehne, Samuel Thomas, Angie W. Boggust, Rameswar Panda, Brian Kingsbury, Rogério Feris, David Harwath, James R. Glass, Michael Picheny, Shih-Fu Chang
2021	J	`jnl`	Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videos. CoRR Brian Chen, Andrew Rouditchenko, Kevin Duarte, Hilde Kuehne, Samuel Thomas, Angie W. Boggust, Rameswar Panda, Brian Kingsbury, Rogério Schmidt Feris, David Harwath, James R. Glass, Michael Picheny, Shih-Fu Chang
2021	J	`jnl`	Routing with Self-Attention for Multimodal Capsule Networks. CoRR Kevin Duarte, Brian Chen, Nina Shvetsova, Andrew Rouditchenko, Samuel Thomas, Alexander H. Liu, David Harwath, James R. Glass, Hilde Kuehne, Mubarak Shah
2021	A*	`conf`	Spoken Moments: Learning Joint Audio-Visual Representations From Video Descriptions. CVPR Mathew Monfort, SouYoung Jin, Alexander H. Liu, David Harwath, Rogério Feris, James R. Glass, Aude Oliva
2021	J	`jnl`	Spoken Moments: Learning Joint Audio-Visual Representations from Video Descriptions. CoRR Mathew Monfort, SouYoung Jin, Alexander H. Liu, David Harwath, Rogério Feris, James R. Glass, Aude Oliva
2021	—	`conf`	Text-Free Image-to-Speech Synthesis Using Learned Segmental Units. ACL/IJCNLP (1) Wei-Ning Hsu, David Harwath, Tyler Miller, Christopher Song, James R. Glass
2020	J	`jnl`	AVLnet: Learning Audio-Visual Language Representations from Instructional Videos. CoRR Andrew Rouditchenko, Angie W. Boggust, David Harwath, Brian Chen, Dhiraj Joshi, Samuel Thomas, Kartik Audhkhasi, Hilde Kuehne, Rameswar Panda, Rogério Feris, Brian Kingsbury, Michael Picheny, Antonio Torralba, James R. Glass
2020	J	`jnl`	Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input. Int. J. Comput. Vis. David Harwath, Adrià Recasens, Dídac Surís, Galen Chuang, Antonio Torralba, James R. Glass
2020	A*	`conf`	Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech. ICLR David Harwath, Wei-Ning Hsu, James R. Glass
2020	A	`conf`	Pair Expansion for Learning Multilingual Semantic Embeddings Using Disjoint Visually-Grounded Speech Audio Datasets. INTERSPEECH Yasunori Ohishi, Akisato Kimura, Takahito Kawanishi, Kunio Kashino, David Harwath, James R. Glass
2020	J	`jnl`	Text-Free Image-to-Speech Synthesis Using Learned Segmental Units. CoRR Wei-Ning Hsu, David Harwath, Christopher Song, James R. Glass
2020	Misc	`conf`	Trilingual Semantic Embeddings of Visually Grounded Speech with Self-Attention Mechanisms. ICASSP Yasunori Ohishi, Akisato Kimura, Takahito Kawanishi, Kunio Kashino, David Harwath, James R. Glass
2019	—	`conf`	Grounding Spoken Words in Unlabeled Video. CVPR Workshops Angie W. Boggust, Kartik Audhkhasi, Dhiraj Joshi, David Harwath, Samuel Thomas, Rogério Schmidt Feris, Danny Gutfreund, Yang Zhang, Antonio Torralba, Michael Picheny, James R. Glass
2019	J	`jnl`	Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech. CoRR David Harwath, Wei-Ning Hsu, James R. Glass
2019	A*	`conf`	Learning Words by Drawing Images. CVPR Didac Suris, Adrià Recasens, David Bau, David Harwath, James R. Glass, Antonio Torralba
2019	A	`conf`	Towards Bilingual Lexicon Discovery From Visually Grounded Speech Audio. INTERSPEECH Emmanuel Azuh, David Harwath, James R. Glass
2019	J	`jnl`	Towards Visually Grounded Sub-Word Speech Unit Discovery. CoRR David Harwath, James R. Glass
2019	Misc	`conf`	Towards Visually Grounded Sub-word Speech Unit Discovery. ICASSP David Harwath, James R. Glass
2019	A	`conf`	Transfer Learning from Audio-Visual Grounding to Speech Recognition. INTERSPEECH Wei-Ning Hsu, David Harwath, James R. Glass
2018	—	`conf`	Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input. ECCV (6) David Harwath, Adrià Recasens, Dídac Surís, Galen Chuang, Antonio Torralba, James R. Glass
2018	J	`jnl`	Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input. CoRR David Harwath, Adrià Recasens, Dídac Surís, Galen Chuang, Antonio Torralba, James R. Glass
2018	Misc	`conf`	Vision as an Interlingua: Learning Multilingual Semantic Embeddings of Untranscribed Speech. ICASSP David Harwath, Galen Chuang, James R. Glass
2017	J	`jnl`	Learning Modality-Invariant Representations for Speech and Images. CoRR Kenneth Leidal, David Harwath, James R. Glass
2017	—	`conf`	Learning Word-Like Units from Joint Audio-Visual Analysis. ACL (1) David Harwath, James R. Glass
2017	C	`conf`	Learning modality-invariant representations for speech and images. ASRU Kenneth Leidal, David Harwath, James R. Glass
2014	A	`conf`	Choosing useful word alternates for automatic speech recognition correction interfaces. INTERSPEECH David Harwath, Alexander Gruenstein, Ian McGraw
2013	Misc	`conf`	A summary of the 2012 JHU CLSP workshop on zero resource speech technologies and models of early language acquisition. ICASSP Aren Jansen, Emmanuel Dupoux, Sharon Goldwater, Mark Johnson, Sanjeev Khudanpur, Kenneth Church, Naomi Feldman, Hynek Hermansky, Florian Metze, Richard C. Rose, Mike Seltzer, Pascal Clark, Ian McGraw, Balakrishnan Varadarajan, Erin Bennett, Benjamin Börschinger, Justin T. Chiu, Ewan Dunbar, Abdellah Fourtassi, David Harwath, Chia-ying Lee, Keith D. Levin, Atta Norouzian, Vijayaditya Peddinti, Rachael Richardson, Thomas Schatz, Samuel Thomas