Zeyu Jin

92 papers A* 11A 8B 1Misc 18Journal 41Unranked 12
YearRankTypeTitle / Venue / Authors
2026 J jnl
AudioChat: Unified Audio Storytelling, Editing, and Understanding with Transfusion Forcing.
CoRR
William Chen, Prem Seetharaman, Rithesh Kumar, Oriol Nieto, Shinji Watanabe, Justin Salamon, Zeyu Jin
2026 J jnl
From Natural Alignment to Conditional Controllability in Multimodal Dialogue.
CoRR
Zeyu Jin, Songtao Zhou, Haoyu Wang, Minghao Tian, Kaifeng Yun, Zhuo Chen, Xiaoyu Qin, Jia Jia
2026 J jnl
Gencho: Room Impulse Response Generation from Reverberant Speech and Text via Diffusion Transformers.
CoRR
Jackie Lin, Jiaqi Su, Nishit Anand, Zeyu Jin, Minje Kim, Paris Smaragdis
2026 J jnl
TAC: Timestamped Audio Captioning.
CoRR
Sonal Kumar, Prem Seetharaman, Ke Chen, Oriol Nieto, Jiaqi Su, Zhepei Wang, Rithesh Kumar, Dinesh Manocha, Nicholas J. Bryan, Zeyu Jin, Justin Salamon
2026 J jnl
Transfer learning-driven adaptive physics-informed neural networks acceleration method for aerodynamic identification.
Eng. Appl. Artif. Intell.
Ming Yan, Kai Liu, Shuaibin An, Guan Wang, Zeyu Jin, Jianwen Zang
2025 J jnl
Audiovisual emotion recognition based on bi-layer LSTM and multi-head attention mechanism on RAVDESS dataset.
J. Supercomput.
Zeyu Jin, Wenjiao Zai
2025 Misc conf
Code Drift: Towards Idempotent Neural Audio Codecs.
ICASSP
Patrick O'Reilly, Prem Seetharaman, Jiaqu Su, Zeyu Jin, Bryan Pardo
2025 A* conf
DMOSpeech: Direct Metric Optimization via Distilled Diffusion Model in Zero-Shot Speech Synthesis.
ICML
Yinghao Aaron Li, Rithesh Kumar, Zeyu Jin
2025 J jnl
Deep Audio Watermarks are Shallow: Limitations of Post-Hoc Watermarking Techniques for Speech.
CoRR
Patrick O'Reilly, Zeyu Jin, Jiaqi Su, Bryan Pardo
2025 J jnl
DiTSE: High-Fidelity Generative Speech Enhancement via Latent Diffusion Transformers.
CoRR
Heitor R. Guimarães, Jiaqi Su, Rithesh Kumar, Tiago H. Falk, Zeyu Jin
2025 — conf
DiTVC: One-Shot Voice Conversion via Diffusion Transformer with Environment and Speaking Rate Cloning.
WASPAA
Yunyun Wang, Jiaqi Su, Adam Finkelstein, Rithesh Kumar, Ke Chen, Zeyu Jin
2025 J jnl
Efficient Long-Time Simulations of Multiscale Systems via High-Order Numerical Homogenization.
CoRR
Bojin Chen, Zeyu Jin, Ruo Li
2025 Misc conf
Everyone-Can-Sing: Zero-Shot Singing Voice Synthesis and Conversion with Speech Reference.
ICASSP
Shuqi Dai, Yunyun Wang, Roger B. Dannenberg, Zeyu Jin
2025 J jnl
Everyone-Can-Sing: Zero-Shot Singing Voice Synthesis and Conversion with Speech Reference.
CoRR
Shuqi Dai, Yunyun Wang, Roger B. Dannenberg, Zeyu Jin
2025 A* conf
HarmoniVox: Painting Voices to Match the Avatar's Soul.
ACM Multimedia
Songtao Zhou, Xiaoyu Qin, Yixuan Zhou, Qixin Wang, Zeyu Jin, Zixuan Wang, Zhiyong Wu, Jia Jia
2025 J jnl
PromptSep: Generative Audio Separation via Multimodal Prompting.
CoRR
Yutong Wen, Ke Chen, Prem Seetharaman, Oriol Nieto, Jiaqi Su, Rithesh Kumar, Minje Kim, Paris Smaragdis, Zeyu Jin, Justin Salamon
2025 A* conf
SpeakEasy: Enhancing Text-to-Speech Interactions for Expressive Content Creation.
CHI
Stephen Brade, Sam Anderson, Rithesh Kumar, Zeyu Jin, Anh Truong
2025 J jnl
SpeakEasy: Enhancing Text-to-Speech Interactions for Expressive Content Creation.
CoRR
Stephen Brade, Sam Anderson, Rithesh Kumar, Zeyu Jin, Anh Truong
2025 J jnl
SpeechOp: Inference-Time Task Composition for Generative Speech Processing.
CoRR
Justin Lovelace, Rithesh Kumar, Jiaqi Su, Ke Chen, Kilian Q. Weinberger, Zeyu Jin
2025 B conf
V-CASS: Vision-context-aware Expressive Speech Synthesis for Enhancing User Understanding of Videos.
IJCNN
Qixin Wang, Songtao Zhou, Zeyu Jin, Chenglin Guo, Shikun Sun, Xiaoyu Qin
2025 J jnl
V-CASS: Vision-context-aware Expressive Speech Synthesis for Enhancing User Understanding of Videos.
CoRR
Qixin Wang, Songtao Zhou, Zeyu Jin, Chenglin Guo, Shikun Sun, Xiaoyu Qin
2025 A* conf
Visual Description Grounding Reduces Hallucinations and Boosts Reasoning in LVLMs.
ICLR
Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Utkarsh Tyagi, Oriol Nieto, Zeyu Jin, Dinesh Manocha
2024 A* conf
A Closer Look at the Limitations of Instruction Tuning.
ICML
Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Ramaneswaran S., Deepali Aneja, Zeyu Jin, Ramani Duraiswami, Dinesh Manocha
2024 J jnl
A Closer Look at the Limitations of Instruction Tuning.
CoRR
Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Ramaneswaran S., Deepali Aneja, Zeyu Jin, Ramani Duraiswami, Dinesh Manocha
2024 J jnl
Code Drift: Towards Idempotent Neural Audio Codecs.
CoRR
Patrick O'Reilly, Prem Seetharaman, Jiaqi Su, Zeyu Jin, Bryan Pardo
2024 J jnl
DMDSpeech: Distilled Diffusion Model Surpassing The Teacher in Zero-shot Speech Synthesis via Direct Metric Optimization.
CoRR
Yingahao Aaron Li, Rithesh Kumar, Zeyu Jin
2024 Misc conf
GR0: Self-Supervised Global Representation Learning for Zero-Shot Voice Conversion.
ICASSP
Yunyun Wang, Jiaqi Su, Adam Finkelstein, Zeyu Jin
2024 A conf
Genhancer: High-Fidelity Speech Enhancement via Generative Modeling on Discrete Codec Tokens.
INTERSPEECH
Haici Yang, Jiaqi Su, Minje Kim, Zeyu Jin
2024 A conf
Improving Generalization of Speech Separation in Real-World Scenarios: Strategies in Simulation, Optimization, and Evaluation.
INTERSPEECH
Ke Chen, Jiaqi Su, Taylor Berg-Kirkpatrick, Shlomo Dubnov, Zeyu Jin
2024 J jnl
Improving Generalization of Speech Separation in Real-World Scenarios: Strategies in Simulation, Optimization, and Evaluation.
CoRR
Ke Chen, Jiaqi Su, Taylor Berg-Kirkpatrick, Shlomo Dubnov, Zeyu Jin
2024 Misc conf
MDX-GAN: Enhancing Perceptual Quality in Multi-Class Source Separation Via Adversarial Training.
ICASSP
Ke Chen, Jiaqi Su, Zeyu Jin
2024 Misc conf
Maskmark: Robust Neuralwatermarking for Real and Synthetic Speech.
ICASSP
Patrick O'Reilly, Zeyu Jin, Jiaqi Su, Bryan Pardo
2024 — conf
SoulSkipper: A Voice-Controlled Emotional Adaptive Game to Complement Therapy for Social Anxiety Disorder.
CHI Extended Abstracts
Xiaohan Li, Qixin Wang, Zishan Wang, Zeyu Jin, Jia Jia
2024 A* conf
SpeechCraft: A Fine-Grained Expressive Speech Dataset with Natural Language Description.
ACM Multimedia
Zeyu Jin, Jia Jia, Qixin Wang, Kehan Li, Shuoyi Zhou, Songtao Zhou, Xiaoyu Qin, Zhiyong Wu
2024 J jnl
SpeechCraft: A Fine-grained Expressive Speech Dataset with Natural Language Description.
CoRR
Zeyu Jin, Jia Jia, Qixin Wang, Kehan Li, Shuoyi Zhou, Songtao Zhou, Xiaoyu Qin, Zhiyong Wu
2024 J jnl
VDGD: Mitigating LVLM Hallucinations in Cognitive Prompts by Bridging the Visual Perception Gap.
CoRR
Sreyan Ghosh, Chandra Kiran Reddy Evuru, Sonal Kumar, Utkarsh Tyagi, Oriol Nieto, Zeyu Jin, Dinesh Manocha
2024 A* conf
VoxInstruct: Expressive Human Instruction-to-Speech Generation with Unified Multilingual Codec Language Modelling.
ACM Multimedia
Yixuan Zhou, Xiaoyu Qin, Zeyu Jin, Shuoyi Zhou, Shun Lei, Songtao Zhou, Zhiyong Wu, Jia Jia
2024 J jnl
VoxInstruct: Expressive Human Instruction-to-Speech Generation with Unified Multilingual Codec Language Modelling.
CoRR
Yixuan Zhou, Xiaoyu Qin, Zeyu Jin, Shuoyi Zhou, Shun Lei, Songtao Zhou, Zhiyong Wu, Jia Jia
2023 A conf
Efficient Spoken Language Recognition via Multilabel Classification.
INTERSPEECH
Oriol Nieto, Zeyu Jin, Franck Dernoncourt, Justin Salamon
2023 J jnl
Efficient Spoken Language Recognition via Multilabel Classification.
CoRR
Oriol Nieto, Zeyu Jin, Franck Dernoncourt, Justin Salamon
2023 A* conf
HoloSinger: Semantics and Music Driven Motion Generation with Octahedral Holographic Projection.
ACM Multimedia
Zeyu Jin, Zixuan Wang, Qixin Wang, Jia Jia, Ye Bai, Yi Zhao, Hao Li, Xiaorui Wang
2023 — conf
White Box Search Over Audio Synthesizer Parameters.
ISMIR
Yuting Yang, Zeyu Jin, Connelly Barnes, Adam Finkelstein
2022 A* conf
AI Carpet: Automatic Generation of Aesthetic Carpet Pattern.
ACM Multimedia
Ziyi Wang, Xingqi Wang, Zeyu Jin, Xiaohan Li, Shikun Sun, Jia Jia
2022 A conf
Audio Similarity is Unreliable as a Proxy for Audio Quality.
INTERSPEECH
Pranay Manocha, Zeyu Jin, Adam Finkelstein
2022 J jnl
Audio Similarity is Unreliable as a Proxy for Audio Quality.
CoRR
Pranay Manocha, Zeyu Jin, Adam Finkelstein
2022 Misc conf
Controllable Speech Representation Learning Via Voice Conversion and AIC Loss.
ICASSP
Yunyun Wang, Jiaqi Su, Adam Finkelstein, Zeyu Jin
2022 J jnl
HEAR 2021: Holistic Evaluation of Audio Representations.
CoRR
Joseph Turian, Jordie Shier, Humair Raj Khan, Bhiksha Raj, Björn W. Schuller, Christian J. Steinmetz, Colin Malloy, George Tzanetakis, Gissel Velarde, Kirk McNally, Max Henry, Nicolas Pinto, Camille Noufi, Christian Clough, Dorien Herremans, Eduardo Fonseca, Jesse H. Engel, Justin Salamon, Philippe Esling, Pranay Manocha, Shinji Watanabe, Zeyu Jin, Yonatan Bisk
2022 J jnl
High-order Numerical Homogenization for Dissipative Ordinary Differential Equations.
Multiscale Model. Simul.
Zeyu Jin, Ruo Li
2022 Misc conf
Music Enhancement via Image Translation and Vocoding.
ICASSP
Nikhil Kandpal, Oriol Nieto, Zeyu Jin
2022 J jnl
Music Enhancement via Image Translation and Vocoding.
CoRR
Nikhil Kandpal, Oriol Nieto, Zeyu Jin
2022 A* conf
Record Once, Post Everywhere: Automatic Shortening of Audio Stories for Social Media.
UIST
Bryan Wang, Zeyu Jin, Gautham J. Mysore
2022 Misc conf
SQAPP: No-Reference Speech Quality Assessment Via Pairwise Preference.
ICASSP
Pranay Manocha, Zeyu Jin, Adam Finkelstein
2022 J jnl
Stochastic Augmented Projected Gradient Methods for the Large-Scale Precoding Matrix Indicator Selection Problem.
IEEE Trans. Wirel. Commun.
Jiaqi Zhang, Zeyu Jin, Bo Jiang, Zaiwen Wen
2021 Misc conf
Bandwidth Extension is All You Need.
ICASSP
Jiaqi Su, Yunyun Wang, Adam Finkelstein, Zeyu Jin
2021 Misc conf
CDPAM: Contrastive Learning for Perceptual Audio Similarity.
ICASSP
Pranay Manocha, Zeyu Jin, Richard Zhang, Adam Finkelstein
2021 J jnl
CDPAM: Contrastive learning for perceptual audio similarity.
CoRR
Pranay Manocha, Zeyu Jin, Richard Zhang, Adam Finkelstein
2021 — conf
Compare Machine Learning Models in Text Classification Using Steam User Reviews.
ICSED
Youchen Miao, Zeyu Jin, Yumeng Zhang, Yuchen Chen, Junren Lai
2021 Misc conf
Context-Aware Prosody Correction for Text-Based Speech Editing.
ICASSP
Max Morrison, Lucas Rencker, Zeyu Jin, Nicholas J. Bryan, Juan Pablo Cáceres, Bryan Pardo
2021 J jnl
Context-Aware Prosody Correction for Text-Based Speech Editing.
CoRR
Max Morrison, Lucas Rencker, Zeyu Jin, Nicholas J. Bryan, Juan Pablo Cáceres, Bryan Pardo
2021 — conf
Controllable deep melody generation via hierarchical music structure representation.
ISMIR
Shuqi Dai, Zeyu Jin, Celso Gomes, Roger B. Dannenberg
2021 J jnl
Controllable deep melody generation via hierarchical music structure representation.
CoRR
Shuqi Dai, Zeyu Jin, Celso Gomes, Roger B. Dannenberg
2021 — conf
HEAR: Holistic Evaluation of Audio Representations.
NeurIPS (Competition and Demos)
Joseph Turian, Jordie Shier, Humair Raj Khan, Bhiksha Raj, Björn W. Schuller, Christian J. Steinmetz, Colin Malloy, George Tzanetakis, Gissel Velarde, Kirk McNally, Max Henry, Nicolas Pinto, Camille Noufi, Christian Clough, Dorien Herremans, Eduardo Fonseca, Jesse H. Engel, Justin Salamon, Philippe Esling, Pranay Manocha, Shinji Watanabe, Zeyu Jin, Yonatan Bisk
2021 — conf
HiFi-GAN-2: Studio-Quality Speech Enhancement via Generative Adversarial Networks Conditioned on Acoustic Features.
WASPAA
Jiaqi Su, Zeyu Jin, Adam Finkelstein
2021 J jnl
High Order Numerical Homogenization for Dissipative Ordinary Differential Equations.
CoRR
Zeyu Jin, Ruo Li
2021 J jnl
Neural Pitch-Shifting and Time-Stretching with Controllable LPCNet.
CoRR
Max Morrison, Zeyu Jin, Nicholas J. Bryan, Juan Pablo Cáceres, Bryan Pardo
2020 A conf
A Differentiable Perceptual Audio Metric Learned from Just Noticeable Differences.
INTERSPEECH
Pranay Manocha, Adam Finkelstein, Richard Zhang, Nicholas J. Bryan, Gautham J. Mysore, Zeyu Jin
2020 J jnl
A Differentiable Perceptual Audio Metric Learned from Just Noticeable Differences.
CoRR
Pranay Manocha, Adam Finkelstein, Zeyu Jin, Nicholas J. Bryan, Richard Zhang, Gautham J. Mysore
2020 Misc conf
Acoustic Matching By Embedding Impulse Responses.
ICASSP
Jiaqi Su, Zeyu Jin, Adam Finkelstein
2020 A conf
Controllable Neural Prosody Synthesis.
INTERSPEECH
Max Morrison, Zeyu Jin, Justin Salamon, Nicholas J. Bryan, Gautham J. Mysore
2020 J jnl
Controllable Neural Prosody Synthesis.
CoRR
Max Morrison, Zeyu Jin, Justin Salamon, Nicholas J. Bryan, Gautham J. Mysore
2020 Misc conf
Disentangled Multidimensional Metric Learning for Music Similarity.
ICASSP
Jongpil Lee, Nicholas J. Bryan, Justin Salamon, Zeyu Jin, Juhan Nam
2020 J jnl
Disentangled Multidimensional Metric Learning for Music Similarity.
CoRR
Jongpil Lee, Nicholas J. Bryan, Justin Salamon, Zeyu Jin, Juhan Nam
2020 Misc conf
F0-Consistent Many-To-Many Non-Parallel Voice Conversion Via Conditional Autoencoder.
ICASSP
Kaizhi Qian, Zeyu Jin, Mark Hasegawa-Johnson, Gautham J. Mysore
2020 J jnl
F0-consistent many-to-many non-parallel voice conversion via conditional autoencoder.
CoRR
Kaizhi Qian, Zeyu Jin, Mark Hasegawa-Johnson, Gautham J. Mysore
2020 A conf
HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks.
INTERSPEECH
Jiaqi Su, Zeyu Jin, Adam Finkelstein
2020 J jnl
HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networks.
CoRR
Jiaqi Su, Zeyu Jin, Adam Finkelstein
2020 J jnl
Metric Learning vs Classification for Disentangled Music Representation Learning.
CoRR
Jongpil Lee, Nicholas J. Bryan, Justin Salamon, Zeyu Jin, Juhan Nam
2020 — conf
Metric learning vs classification for disentangled music representation learning.
ISMIR
Jongpil Lee, Nicholas J. Bryan, Justin Salamon, Zeyu Jin, Juhan Nam
2020 A* conf
Music Creation by Example.
CHI
Emma Frid, Celso Gomes, Zeyu Jin
2020 A conf
Pose2Pose: pose selection and transfer for 2D character animation.
IUI
Nora S. Willett, Hijung Valentina Shin, Zeyu Jin, Wilmot Li, Adam Finkelstein
2019 Misc conf
Learning Bandwidth Expansion Using Perceptually-motivated Loss.
ICASSP
Berthy Feng, Zeyu Jin, Jiaqi Su, Adam Finkelstein
2019 Misc conf
Perceptually-motivated Environment-specific Speech Enhancement.
ICASSP
Jiaqi Su, Adam Finkelstein, Zeyu Jin
2019 J jnl
Text-based Editing of Talking-head Video.
CoRR
Ohad Fried, Ayush Tewari, Michael Zollhöfer, Adam Finkelstein, Eli Shechtman, Dan B. Goldman, Kyle Genova, Zeyu Jin, Christian Theobalt, Maneesh Agrawala
2019 J jnl
Text-based editing of talking-head video.
ACM Trans. Graph.
Ohad Fried, Ayush Tewari, Michael Zollhöfer, Adam Finkelstein, Eli Shechtman, Dan B. Goldman, Kyle Genova, Zeyu Jin, Christian Theobalt, Maneesh Agrawala
2018 Misc conf
Fftnet: A Real-Time Speaker-Dependent Neural Vocoder.
ICASSP
Zeyu Jin, Adam Finkelstein, Gautham J. Mysore, Jingwan Lu
2018 — —
Speech Synthesis for Text-Based Editing of Audio Narration
Zeyu Jin
2017 J jnl
VoCo: text-based insertion and replacement in audio narration.
ACM Trans. Graph.
Zeyu Jin, Gautham J. Mysore, Stephen DiVerdi, Jingwan Lu, Adam Finkelstein
2016 Misc conf
Cute: A concatenative method for voice conversion using exemplar-based unit selection.
ICASSP
Zeyu Jin, Adam Finkelstein, Stephen DiVerdi, Jingwan Lu, Gautham J. Mysore
2015 — conf
Mallo: a distributed synchronized musical instrument designed for internet performance.
NIME
Zeyu Jin, Reid Oda, Adam Finkelstein, Rebecca Fiebrink
2014 — conf
AudioQuilt: 2D Arrangements of Audio Samples using Metric Learning and Kernelized Sorting.
NIME
Ohad Fried, Zeyu Jin, Reid Oda, Adam Finkelstein
2013 — conf
Formal Semantics for Music Notation control Flow.
ICMC
Zeyu Jin, Roger B. Dannenberg
2013 — conf
Formal Semantics for Music Notation control Flow.
ICMC
Zeyu Jin, Roger B. Dannenberg