Rafael Valle

68 papers A* 12A 1Misc 6Journal 41Unranked 8
YearRankTypeTitle / Venue / Authors
2026 J jnl
Speech-Hands: A Self-Reflection Voice Agentic Approach to Speech Recognition and Audio Reasoning with Omni Perception.
CoRR
Zhen Wan, Chao-Han Huck Yang, Jinchuan Tian, Hanrong Ye, Ankita Pasad, Szu-Wei Fu, Arushi Goel, Ryo Hachiuma, Shizhe Diao, Kunal Dhawan, Sreyan Ghosh, Yusuke Hirota, Zhehuai Chen, Rafael Valle, Ehsan Hosseini-Asl, Chenhui Chu, Shinji Watanabe, Yu-Chiang Frank Wang, Boris Ginsburg
2025 J jnl
A2SB: Audio-to-Audio Schrodinger Bridges.
CoRR
Zhifeng Kong, Kevin J. Shih, Weili Nie, Arash Vahdat, Sang-gil Lee, João Felipe Santos, Ante Jukic, Rafael Valle, Bryan Catanzaro
2025 A* conf
Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities.
ICML
Sreyan Ghosh, Zhifeng Kong, Sonal Kumar, S. Sakshi, Jaehyeon Kim, Wei Ping, Rafael Valle, Dinesh Manocha, Bryan Catanzaro
2025 J jnl
Audio Flamingo 2: An Audio-Language Model with Long-Audio Understanding and Expert Reasoning Abilities.
CoRR
Sreyan Ghosh, Zhifeng Kong, Sonal Kumar, S. Sakshi, Jaehyeon Kim, Wei Ping, Rafael Valle, Dinesh Manocha, Bryan Catanzaro
2025 J jnl
Audio Flamingo 3: Advancing Audio Intelligence with Fully Open Large Audio Language Models.
CoRR
Arushi Goel, Sreyan Ghosh, Jaehyeon Kim, Sonal Kumar, Zhifeng Kong, Sang-gil Lee, Chao-Han Huck Yang, Ramani Duraiswami, Dinesh Manocha, Rafael Valle, Bryan Catanzaro
2025 J jnl
Audio Flamingo Sound-CoT Technical Report: Improving Chain-of-Thought Reasoning in Sound Understanding.
CoRR
Zhifeng Kong, Arushi Goel, João Felipe Santos, Sreyan Ghosh, Rafael Valle, Wei Ping, Bryan Catanzaro
2025 A* conf
ETTA: Elucidating the Design Space of Text-to-Audio Models.
ICML
Sang-gil Lee, Zhifeng Kong, Arushi Goel, Sungwon Kim, Rafael Valle, Bryan Catanzaro
2025 A* conf
Fugatto 1: Foundational Generative Audio Transformer Opus 1.
ICLR
Rafael Valle, Rohan Badlani, Zhifeng Kong, Sang-gil Lee, Arushi Goel, Sungwon Kim, João Felipe Santos, Shuqi Dai, Siddharth Gururani, Aya Aljafari, Alexander H. Liu, Kevin J. Shih, Ryan Prenger, Wei Ping, Chao-Han Huck Yang, Bryan Catanzaro
2025 A* conf
Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidance.
EMNLP
Shehzeen Samarah Hussain, Paarth Neekhara, Xuesong Yang, Edresson Casanova, Subhankar Ghosh, Roy Fejgin, Mikyas T. Desta, Rafael Valle, Jason Li
2025 J jnl
Koel-TTS: Enhancing LLM based Speech Generation with Preference Alignment and Classifier Free Guidance.
CoRR
Shehzeen Hussain, Paarth Neekhara, Xuesong Yang, Edresson Casanova, Subhankar Ghosh, Mikyas T. Desta, Roy Fejgin, Rafael Valle, Jason Li
2025 J jnl
Multi-Domain Audio Question Answering Toward Acoustic Content Reasoning in The DCASE 2025 Challenge.
CoRR
Chao-Han Huck Yang, Sreyan Ghosh, Qing Wang, Jaeyeon Kim, Hengyi Hong, Sonal Kumar, Guirui Zhong, Zhifeng Kong, Sakshi Singh, Vaibhavi Lokegaonkar, Oriol Nieto, Ramani Duraiswami, Dinesh Manocha, Gunhee Kim, Jun Du, Rafael Valle, Bryan Catanzaro
2025 J jnl
OmniVinci: Enhancing Architecture and Data for Omni-Modal Understanding LLM.
CoRR
Hanrong Ye, Chao-Han Huck Yang, Arushi Goel, Wei Huang, Ligeng Zhu, Yuanhang Su, Sean Lin, An-Chieh Cheng, Zhen Wan, Jinchuan Tian, Yuming Lou, Dong Yang, Zhijian Liu, Yukang Chen, Ambrish Dantrey, Ehsan Jahangiri, Sreyan Ghosh, Daguang Xu, Ehsan Hosseini-Asl, Danial Mohseni-Taheri, Vidya Murali, Sifei Liu, Yao Lu, Oluwatobi Olabiyi, Yu-Chiang Frank Wang, Rafael Valle, Bryan Catanzaro, Andrew Tao, Song Han, Jan Kautz, Hongxu Yin, Pavlo Molchanov
2025 A* conf
Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Data.
ICLR
Sreyan Ghosh, Sonal Kumar, Zhifeng Kong, Rafael Valle, Bryan Catanzaro, Dinesh Manocha
2025 J jnl
UALM: Unified Audio Language Model for Understanding, Generation and Reasoning.
CoRR
Jinchuan Tian, Sang-gil Lee, Zhifeng Kong, Sreyan Ghosh, Arushi Goel, Chao-Han Huck Yang, Wenliang Dai, Zihan Liu, Hanrong Ye, Shinji Watanabe, Mohammad Shoeybi, Bryan Catanzaro, Rafael Valle, Wei Ping
2025 A* conf
UniWav: Towards Unified Pre-training for Speech Representation Learning and Generation.
ICLR
Alexander H. Liu, Sang-gil Lee, Chao-Han Huck Yang, Yuan Gong, Yu-Chiang Frank Wang, James R. Glass, Rafael Valle, Bryan Catanzaro
2025 J jnl
UniWav: Towards Unified Pre-training for Speech Representation Learning and Generation.
CoRR
Alexander H. Liu, Sang-gil Lee, Chao-Han Huck Yang, Yuan Gong, Yu-Chiang Frank Wang, James R. Glass, Rafael Valle, Bryan Catanzaro
2024 J jnl
Audio Dialogues: Dialogues dataset for audio and music understanding.
CoRR
Arushi Goel, Zhifeng Kong, Rafael Valle, Bryan Catanzaro
2024 A* conf
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities.
ICML
Zhifeng Kong, Arushi Goel, Rohan Badlani, Wei Ping, Rafael Valle, Bryan Catanzaro
2024 J jnl
Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilities.
CoRR
Zhifeng Kong, Arushi Goel, Rohan Badlani, Wei Ping, Rafael Valle, Bryan Catanzaro
2024 J jnl
ETTA: Elucidating the Design Space of Text-to-Audio Models.
CoRR
Sang-gil Lee, Zhifeng Kong, Arushi Goel, Sungwon Kim, Rafael Valle, Bryan Catanzaro
2024 A* conf
ExpressiveSinger: Multilingual and Multi-Style Score-based Singing Voice Synthesis with Expressive Performance Control.
ACM Multimedia
Shuqi Dai, Ming-Yu Liu, Rafael Valle, Siddharth Gururani
2024 J jnl
Improving Robustness of LLM-based Speech Synthesis by Learning Monotonic Alignment.
CoRR
Paarth Neekhara, Shehzeen Hussain, Subhankar Ghosh, Jason Li, Rafael Valle, Rohan Badlani, Boris Ginsburg
2024 J jnl
Improving Text-To-Audio Models with Synthetic Captions.
CoRR
Zhifeng Kong, Sang-gil Lee, Deepanway Ghosal, Navonil Majumder, Ambuj Mehrish, Rafael Valle, Soujanya Poria, Bryan Catanzaro
2024 J jnl
OMCAT: Omni Context Aware Transformer.
CoRR
Arushi Goel, Karan Sapra, Matthieu Le, Rafael Valle, Andrew Tao, Bryan Catanzaro
2024 J jnl
Scaling NVIDIA's Multi-speaker Multi-lingual TTS Systems with Zero-Shot TTS to Indic Languages.
CoRR
Akshit Arora, Rohan Badlani, Sungwon Kim, Rafael Valle, Bryan Catanzaro
2024 — conf
Scaling Nvidia's Multi-Speaker Multi-Lingual TTS Systems With Zero-Shot TTS to Indic Languages.
ICASSP Workshops
Akshit Arora, Rohan Badlani, Sungwon Kim, Rafael Valle, Bryan Catanzaro
2024 A* conf
SelfVC: Voice Conversion With Iterative Refinement using Self Transformations.
ICML
Paarth Neekhara, Shehzeen Samarah Hussain, Rafael Valle, Boris Ginsburg, Rishabh Ranjan, Shlomo Dubnov, Farinaz Koushanfar, Julian J. McAuley
2024 J jnl
Storage Standards and Solutions, Data Storage, Sharing, and Structuring in Digital Health: A Brazilian Case Study.
Inf.
Nicollas Rodrigues de Oliveira, Yago de Rezende dos Santos, Ana Carolina Rocha Mendes, Guilherme Nunes Nasseh Barbosa, Marcela Tuler de Oliveira, Rafael Valle, Dianne Scherly Varela de Medeiros, Diogo M. F. Mattos
2024 J jnl
Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Data.
CoRR
Sreyan Ghosh, Sonal Kumar, Zhifeng Kong, Rafael Valle, Bryan Catanzaro, Dinesh Manocha
2024 J jnl
TangoFlux: Super Fast and Faithful Text to Audio Generation with Flow Matching and Clap-Ranked Preference Optimization.
CoRR
Chia-Yu Hung, Navonil Majumder, Zhifeng Kong, Ambuj Mehrish, Rafael Valle, Bryan Catanzaro, Soujanya Poria
2023 Misc conf
Any-to-Any Voice Conversion with F0 and Timbre Disentanglement and Novel Timbre Conditioning.
ICASSP
Sudheer Kovela, Rafael Valle, Ambrish Dantrey, Bryan Catanzaro
2023 Misc conf
High-Acoustic Fidelity Text To Speech Synthesis With Fine-Grained Control Of Speech Attributes.
ICASSP
Rafael Valle, João Felipe Santos, Kevin J. Shih, Rohan Badlani, Bryan Catanzaro
2023 J jnl
Multilingual Multiaccented Multispeaker TTS with RADTTS.
CoRR
Rohan Badlani, Rafael Valle, Kevin J. Shih, João Felipe Santos, Siddharth Gururani, Bryan Catanzaro
2023 A* conf
P-Flow: A Fast and Data-Efficient Zero-Shot TTS through Speech Prompting.
NeurIPS
Sungwon Kim, Kevin J. Shih, Rohan Badlani, João Felipe Santos, Evelina Bakhturina, Mikyas T. Desta, Rafael Valle, Sungroh Yoon, Bryan Catanzaro
2023 A conf
RAD-MMM: Multilingual Multiaccented Multispeaker Text To Speech.
INTERSPEECH
Rohan Badlani, Rafael Valle, Kevin J. Shih, João Felipe Santos, Siddharth Gururani, Bryan Catanzaro
2023 A* conf
SPACE: Speech-driven Portrait Animation with Controllable Expression.
ICCV
Siddharth Gururani, Arun Mallya, Ting-Chun Wang, Rafael Valle, Ming-Yu Liu
2023 J jnl
SelfVC: Voice Conversion With Iterative Refinement using Self Transformations.
CoRR
Paarth Neekhara, Shehzeen Hussain, Rafael Valle, Boris Ginsburg, Rishabh Ranjan, Shlomo Dubnov, Farinaz Koushanfar, Julian J. McAuley
2023 J jnl
VANI: Very-lightweight Accent-controllable TTS for Native and Non-native speakers with Identity Preservation.
CoRR
Rohan Badlani, Akshit Arora, Subhankar Ghosh, Rafael Valle, Kevin J. Shih, João Felipe Santos, Boris Ginsburg, Bryan Catanzaro
2023 Misc conf
Vani: Very-Lightweight Accent-Controllable TTS for Native And Non-Native Speakers With Identity Preservation.
ICASSP
Rohan Badlani, Akshit Arora, Subhankar Ghosh, Rafael Valle, Kevin J. Shih, João Felipe Santos, Boris Ginsburg, Bryan Catanzaro
2022 J jnl
Generative Modeling for Low Dimensional Speech Attributes with Neural Spline Flows.
CoRR
Kevin J. Shih, Rafael Valle, Rohan Badlani, João Felipe Santos, Bryan Catanzaro
2022 Misc conf
One TTS Alignment to Rule Them All.
ICASSP
Rohan Badlani, Adrian Lancucki, Kevin J. Shih, Rafael Valle, Wei Ping, Bryan Catanzaro
2022 J jnl
SPACEx: Speech-driven Portrait Animation with Controllable Expression.
CoRR
Siddharth Gururani, Arun Mallya, Ting-Chun Wang, Rafael Valle, Ming-Yu Liu
2021 J jnl
Character-based handwritten text transcription with attention networks.
Neural Comput. Appl.
Jason Poulos, Rafael Valle
2021 A* conf
Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis.
ICLR
Rafael Valle, Kevin J. Shih, Ryan Prenger, Bryan Catanzaro
2021 J jnl
One TTS Alignment To Rule Them All.
CoRR
Rohan Badlani, Adrian Lancucki, Kevin J. Shih, Rafael Valle, Wei Ping, Bryan Catanzaro
2020 J jnl
Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis.
CoRR
Rafael Valle, Kevin J. Shih, Ryan Prenger, Bryan Catanzaro
2020 Misc conf
Mellotron: Multispeaker Expressive Voice Synthesis by Conditioning on Rhythm, Pitch and Global Style Tokens.
ICASSP
Rafael Valle, Jason Li, Ryan Prenger, Bryan Catanzaro
2019 J jnl
Mellotron: Multispeaker expressive voice synthesis by conditioning on rhythm, pitch and global style tokens.
CoRR
Rafael Valle, Jason Li, Ryan Prenger, Bryan Catanzaro
2019 J jnl
Neural ODEs for Image Segmentation with Level Sets.
CoRR
Rafael Valle, Fitsum A. Reda, Mohammad Shoeybi, Patrick LeGresley, Andrew Tao, Bryan Catanzaro
2019 Misc conf
Waveglow: A Flow-based Generative Network for Speech Synthesis.
ICASSP
Ryan Prenger, Rafael Valle, Bryan Catanzaro
2018 J jnl
Attacking Speaker Recognition With Deep Generative Models.
CoRR
Wilson Cai, Anish Doshi, Rafael Valle
2018 J jnl
Missing Data Imputation for Supervised Learning.
Appl. Artif. Intell.
Jason Poulos, Rafael Valle
2018 J jnl
TequilaGAN: How to easily identify GAN samples.
CoRR
Rafael Valle, Wilson Cai, Anish Doshi
2018 J jnl
Visual Display and Retrieval of Music Information.
CoRR
Rafael Valle
2018 J jnl
WaveGlow: A Flow-based Generative Network for Speech Synthesis.
CoRR
Ryan Prenger, Rafael Valle, Bryan Catanzaro
2017 J jnl
Attention networks for image-to-text.
CoRR
Jason Poulos, Rafael Valle
2016 J jnl
ABROA : Audio-Based Room-Occupancy Analysis using Gaussian Mixtures and Hidden Markov Models.
CoRR
Rafael Valle
2016 — conf
ABROA: Audio-Based Room-Occupancy Analysis Using Gaussian Mixtures and Hidden Markov Models.
DCASE
Rafael Valle
2016 — conf
Control Improvisation with Probabilistic Temporal Specifications.
IoTDI
Ilge Akkaya, Daniel J. Fremont, Rafael Valle, Alexandre Donzé, Edward A. Lee, Sanjit A. Seshia
2016 — conf
Learning and Visualizing Music Specifications Using Pattern Graphs.
ISMIR
Rafael Valle, Daniel J. Fremont, Ilge Akkaya, Alexandre Donzé, Adrian Freed, Sanjit A. Seshia
2016 J jnl
Missing Data Imputation for Supervised Learning.
CoRR
Jason Poulos, Rafael Valle
2016 J jnl
Specification Mining for Machine Improvisation with Formal Specifications.
Comput. Entertain.
Rafael Valle, Alexandre Donzé, Daniel J. Fremont, Ilge Akkaya, Sanjit A. Seshia, Adrian Freed, David Wessel
2015 J jnl
Control Improvisation with Probabilistic Temporal Specifications.
CoRR
Ilge Akkaya, Daniel J. Fremont, Rafael Valle, Alexandre Donzé, Edward A. Lee, Sanjit A. Seshia
2015 — conf
Symbolic Music Similarity Using Neuronal Periodicity and Dynamic Programming.
MCM
Rafael Valle, Adrian Freed
2014 — conf
Machine Improvisation with Formal Specifications.
ICMC
Alexandre Donzé, Rafael Valle, Ilge Akkaya, Sophie Libkind, Sanjit A. Seshia, David Wessel
2013 — conf
Gradual control of harmonicity in the Context of frequency modulation.
ICMC
Rafael Valle
2013 — conf
Towards a Dynamic, Inclusive and Equalitarian Augmented Activity Space.
ICMC
Rafael Valle
1993 J jnl
Multi-protocol communications controller.
Microprocess. Microprogramming
Juan Carlos Calderón, JoséRamón Salvador, Rafael Valle, Luis París, Carles Ferrer