Xize Cheng

77 papers A* 24A 2B 1Misc 2Journal 32Unranked 16
YearRankTypeTitle / Venue / Authors
2026 J jnl
Modeling and Benchmarking Spoken Dialogue Rewards with Modality and Colloquialness.
CoRR
Jingyu Lu, Yuhan Wang, Fan Zhuo, Xize Cheng, Changhao Pan, Xueyi Pu, Yifu Chen, Chenyuhao Wen, Tianle Liang, Zhou Zhao
2025 A* conf
A Wander Through the Multimodal Landscape: Efficient Transfer Learning via Low-rank Sequence Multimodal Adapter.
AAAI
Zirun Guo, Xize Cheng, Yangyang Wu, Tao Jin
2025 — conf
CART: A Generative Cross-Modal Retrieval Framework With Coarse-To-Fine Semantic Modeling.
ACL (1)
Minghui Fang, Shengpeng Ji, Jialong Zuo, Hai Huang, Yan Xia, Jieming Zhu, Xize Cheng, Xiaoda Yang, Wenrui Liu, Gang Wang, Zhenhua Dong, Zhou Zhao
2025 — conf
ControlSpeech: Towards Simultaneous and Independent Zero-shot Speaker Cloning and Zero-shot Language Style Control.
ACL (1)
Shengpeng Ji, Qian Chen, Wen Wang, Jialong Zuo, Minghui Fang, Ziyue Jiang, Hai Huang, Zehan Wang, Xize Cheng, Siqi Zheng, Zhou Zhao
2025 Misc conf
Curriculum Learning aided Audio-Visual Speech Recognition with Arbitrary Speaker Number.
ICASSP
Yuxiao Lin, Tao Jin, Xize Cheng, Zhou Zhao, Fei Wu
2025 Misc conf
Enhancing Expressive Voice Conversion with Discrete Pitch-Conditioned Flow Matching Model.
ICASSP
Jialong Zuo, Shengpeng Ji, Minghui Fang, Ziyue Jiang, Xize Cheng, Qian Yang, Wenrui Liu, Guangyan Zhang, Zehai Tu, Yiwen Guo, Zhou Zhao
2025 J jnl
Enhancing Expressive Voice Conversion with Discrete Pitch-Conditioned Flow Matching Model.
CoRR
Jialong Zuo, Shengpeng Ji, Minghui Fang, Ziyue Jiang, Xize Cheng, Qian Yang, Wenrui Liu, Guangyan Zhang, Zehai Tu, Yiwen Guo, Zhou Zhao
2025 A conf
GTA: Towards Generative Text-To-Audio Retrieval via Multi-Scale Tokenizer.
INTERSPEECH
Minghui Fang, Shengpeng Ji, Jialong Zuo, Xize Cheng, Wenrui Liu, Xiaoda Yang, Ruofan Hu, Jieming Zhu, Zhou Zhao
2025 J jnl
MARS-Sep: Multimodal-Aligned Reinforced Sound Separation.
CoRR
Zihan Zhang, Xize Cheng, Zhennan Jiang, Dongjie Fu, Jingyuan Chen, Zhou Zhao, Tao Jin
2025 J jnl
Multi-talker audio-visual speech recognition towards diverse scenarios.
Frontiers Inf. Technol. Electron. Eng.
Yuxiao Lin, Tao Jin, Xize Cheng, Zhou Zhao, Fei Wu
2025 — conf
Multimodal Conditional Retrieval with High Controllability.
KDD (2)
Xiaoda Yang, Xize Cheng, Minghui Fang, Hongshun Qiu, Yuhang Ma, Junyu Lu, Jiaqi Duan, Sihang Cai, Zehan Wang, Ruofan Hu, Dongjie Fu, Zhou Zhao, Tao Jin
2025 A* conf
OmniBind: Large-scale Omni Multimodal Representation via Binding Spaces.
ICLR
Zehan Wang, Ziang Zhang, Minjie Hong, Hang Zhang, Luping Liu, Rongjie Huang, Xize Cheng, Shengpeng Ji, Tao Jin, Hengshuang Zhao, Zhou Zhao
2025 J jnl
OmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarios.
CoRR
Xize Cheng, Dongjie Fu, Xiaoda Yang, Minghui Fang, Ruofan Hu, Jingyu Lu, Jionghao Bai, Zehan Wang, Shengpeng Ji, Rongjie Huang, Linjun Li, Yu Chen, Tao Jin, Zhou Zhao
2025 A* conf
OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup.
ICLR
Xize Cheng, Siqi Zheng, Zehan Wang, Minghui Fang, Ziang Zhang, Rongjie Huang, Shengpeng Ji, Jialong Zuo, Tao Jin, Zhou Zhao
2025 A* conf
PACHAT: Persona-Aware Speech Assistant for Multi-party Dialogue.
EMNLP
Dongjie Fu, Xize Cheng, Linjun Li, Xiaoda Yang, Lujia Yang, Tao Jin
2025 — conf
Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching.
ACL (1)
Jialong Zuo, Shengpeng Ji, Minghui Fang, Mingze Li, Ziyue Jiang, Xize Cheng, Xiaoda Yang, Feiyang Chen, Xinyu Duan, Zhou Zhao
2025 J jnl
Rhythm Controllable and Efficient Zero-Shot Voice Conversion via Shortcut Flow Matching.
CoRR
Jialong Zuo, Shengpeng Ji, Minghui Fang, Mingze Li, Ziyue Jiang, Xize Cheng, Xiaoda Yang, Feiyang Chen, Xinyu Duan, Zhou Zhao
2025 A* conf
SpatialCLIP: Learning 3D-aware Image Representations from Spatially Discriminative Language.
CVPR
Zehan Wang, Sashuai Zhou, Shaoxuan He, Haifeng Huang, Lihe Yang, Ziang Zhang, Xize Cheng, Shengpeng Ji, Tao Jin, Hengshuang Zhao, Zhou Zhao
2025 — conf
T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback.
ACL (1)
Zehan Wang, Ke Lei, Chen Zhu, Jiawei Huang, Sashuai Zhou, Luping Liu, Xize Cheng, Shengpeng Ji, Zhenhui Ye, Tao Jin, Zhou Zhao
2025 J jnl
T2A-Feedback: Improving Basic Capabilities of Text-to-Audio Generation via Fine-grained AI Feedback.
CoRR
Zehan Wang, Ke Lei, Chen Zhu, Jiawei Huang, Sashuai Zhou, Luping Liu, Xize Cheng, Shengpeng Ji, Zhenhui Ye, Tao Jin, Zhou Zhao
2025 J jnl
Unleashing the Power of Natural Audio Featuring Multiple Sound Sources.
CoRR
Xize Cheng, Slytherin Wang, Zehan Wang, Rongjie Huang, Tao Jin, Zhou Zhao
2025 A* conf
VoxDialogue: Can Spoken Dialogue Systems Understand Information Beyond Words?
ICLR
Xize Cheng, Ruofan Hu, Xiaoda Yang, Jingyu Lu, Dongjie Fu, Zehan Wang, Shengpeng Ji, Rongjie Huang, Boyang Zhang, Tao Jin, Zhou Zhao
2025 B conf
VoxpopuliTTS: a large-scale multilingual TTS corpus for zero-shot speech generation.
COLING
Wenrui Liu, Jionghao Bai, Xize Cheng, Jialong Zuo, Ziyue Jiang, Shengpeng Ji, Minghui Fang, Xiaoda Yang, Qian Yang, Zhou Zhao
2025 J jnl
WavReward: Spoken Dialogue Models With Generalist Reward Evaluators.
CoRR
Shengpeng Ji, Tianle Liang, Yangzhuo Li, Jialong Zuo, Minghui Fang, Jinzheng He, Yifu Chen, Zhengqing Liu, Ziyue Jiang, Xize Cheng, Siqi Zheng, Jin Xu, Junyang Lin, Zhou Zhao
2025 A* conf
WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling.
ICLR
Shengpeng Ji, Ziyue Jiang, Wen Wang, Yifu Chen, Minghui Fang, Jialong Zuo, Qian Yang, Xize Cheng, Zehan Wang, Ruiqi Li, Ziang Zhang, Xiaoda Yang, Rongjie Huang, Yidi Jiang, Qian Chen, Siqi Zheng, Zhou Zhao
2024 J jnl
A Wander Through the Multimodal Landscape: Efficient Transfer Learning via Low-rank Sequence Multimodal Adapter.
CoRR
Zirun Guo, Xize Cheng, Yangyang Wu, Tao Jin
2024 J jnl
ACE: A Generative Cross-Modal Retrieval Framework with Coarse-To-Fine Semantic Modeling.
CoRR
Minghui Fang, Shengpeng Ji, Jialong Zuo, Hai Huang, Yan Xia, Jieming Zhu, Xize Cheng, Xiaoda Yang, Wenrui Liu, Gang Wang, Zhenhua Dong, Zhou Zhao
2024 A* conf
AudioLCM: Efficient and High-Quality Text-to-Audio Generation with Minimal Inference Steps.
ACM Multimedia
Huadai Liu, Rongjie Huang, Yang Liu, Hengyuan Cao, Jialei Wang, Xize Cheng, Siqi Zheng, Zhou Zhao
2024 J jnl
AudioLCM: Text-to-Audio Generation with Latent Consistency Models.
CoRR
Huadai Liu, Rongjie Huang, Yang Liu, Hengyuan Cao, Jialei Wang, Xize Cheng, Siqi Zheng, Zhou Zhao
2024 A* conf
AudioVSR: Enhancing Video Speech Recognition with Audio Data.
EMNLP
Xiaoda Yang, Xize Cheng, Jiaqi Duan, Hongshun Qiu, Minjie Hong, Minghui Fang, Shengpeng Ji, Jialong Zuo, Zhiqing Hong, Zhimeng Zhang, Tao Jin
2024 A* conf
Boosting Speech Recognition Robustness to Modality-Distortion with Contrast-Augmented Prompts.
ACM Multimedia
Dongjie Fu, Xize Cheng, Xiaoda Yang, Hanting Wang, Zhou Zhao, Tao Jin
2024 A* conf
Chat-Scene: Bridging 3D Scene and Large Language Models with Object Identifiers.
NeurIPS
Haifeng Huang, Yilun Chen, Zehan Wang, Rongjie Huang, Runsen Xu, Tai Wang, Luping Liu, Xize Cheng, Yang Zhao, Jiangmiao Pang, Zhou Zhao
2024 J jnl
ControlSpeech: Towards Simultaneous Zero-shot Speaker Cloning and Zero-shot Language Style Control With Decoupled Codec.
CoRR
Shengpeng Ji, Jialong Zuo, Minghui Fang, Siqi Zheng, Qian Chen, Wen Wang, Ziyue Jiang, Hai Huang, Xize Cheng, Rongjie Huang, Zhou Zhao
2024 A* conf
Extending Multi-modal Contrastive Representations.
NeurIPS
Ziang Zhang, Zehan Wang, Luping Liu, Rongjie Huang, Xize Cheng, Zhenhui Ye, Wang Lin, Huadai Liu, Haifeng Huang, Yang Zhao, Tao Jin, Siqi Zheng, Zhou Zhao
2024 A* conf
FreeBind: Free Lunch in Unified Multimodal Space via Knowledge Fusion.
ICML
Zehan Wang, Ziang Zhang, Xize Cheng, Rongjie Huang, Luping Liu, Zhenhui Ye, Haifeng Huang, Yang Zhao, Tao Jin, Peng Gao, Zhou Zhao
2024 J jnl
FreeBind: Free Lunch in Unified Multimodal Space via Knowledge Fusion.
CoRR
Zehan Wang, Ziang Zhang, Xize Cheng, Rongjie Huang, Luping Liu, Zhenhui Ye, Haifeng Huang, Yang Zhao, Tao Jin, Peng Gao, Zhou Zhao
2024 A* conf
InstructSpeech: Following Speech Editing Instructions via Large Language Models.
ICML
Rongjie Huang, Ruofan Hu, Yongqi Wang, Zehan Wang, Xize Cheng, Ziyue Jiang, Zhenhui Ye, Dongchao Yang, Luping Liu, Peng Gao, Zhou Zhao
2024 A conf
Landmark-guided Diffusion Model for High-fidelity and Temporally Coherent Talking Head Generation.
ICME
Jintao Tan, Xize Cheng, Lingyu Xiong, Lei Zhu, Xiandong Li, Xianjia Wu, Kai Gong, Minglei Li, Yi Cai
2024 J jnl
Landmark-guided Diffusion Model for High-fidelity and Temporally Coherent Talking Head Generation.
CoRR
Jintao Tan, Xize Cheng, Lingyu Xiong, Lei Zhu, Xiandong Li, Xianjia Wu, Kai Gong, Minglei Li, Yi Cai
2024 A* conf
MimicTalk: Mimicking a personalized and expressive 3D talking face in minutes.
NeurIPS
Zhenhui Ye, Tianyun Zhong, Yi Ren, Ziyue Jiang, Jiawei Huang, Rongjie Huang, Jinglin Liu, Jinzheng He, Chen Zhang, Zehan Wang, Xize Cheng, Xiang Yin, Zhou Zhao
2024 J jnl
MuVi: Video-to-Music Generation with Semantic Alignment and Rhythmic Synchronization.
CoRR
Ruiqi Li, Siqi Zheng, Xize Cheng, Ziang Zhang, Shengpeng Ji, Zhou Zhao
2024 J jnl
OmniBind: Large-scale Omni Multimodal Representation via Binding Spaces.
CoRR
Zehan Wang, Ziang Zhang, Hang Zhang, Luping Liu, Rongjie Huang, Xize Cheng, Hengshuang Zhao, Zhou Zhao
2024 J jnl
OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup.
CoRR
Xize Cheng, Siqi Zheng, Zehan Wang, Minghui Fang, Ziang Zhang, Rongjie Huang, Ziyang Ma, Shengpeng Ji, Jialong Zuo, Tao Jin, Zhou Zhao
2024 — conf
Rethinking the Multimodal Correlation of Multimodal Sequential Learning via Generalizable Attentional Results Alignment.
ACL (1)
Tao Jin, Wang Lin, Ye Wang, Linjun Li, Xize Cheng, Zhou Zhao
2024 A* conf
SegTalker: Segmentation-based Talking Face Generation with Mask-guided Local Editing.
ACM Multimedia
Lingyu Xiong, Xize Cheng, Jintao Tan, Xianjia Wu, Xiandong Li, Lei Zhu, Fei Ma, Minglei Li, Huang Xu, Zhihui Hu
2024 J jnl
SegTalker: Segmentation-based Talking Face Generation with Mask-guided Local Editing.
CoRR
Lingyu Xiong, Xize Cheng, Jintao Tan, Xianjia Wu, Xiandong Li, Lei Zhu, Fei Ma, Minglei Li, Huang Xu, Zhihu Hu
2024 A* conf
SyncTalklip: Highly Synchronized Lip-Readable Speaker Generation with Multi-Task Learning.
ACM Multimedia
Xiaoda Yang, Xize Cheng, Dongjie Fu, Minghui Fang, Jialong Zuo, Shengpeng Ji, Zhou Zhao, Tao Jin
2024 — conf
Text-to-Song: Towards Controllable Music Generation Incorporating Vocal and Accompaniment.
ACL (1)
Zhiqing Hong, Rongjie Huang, Xize Cheng, Yongqi Wang, Ruiqi Li, Fuming You, Zhou Zhao, Zhimeng Zhang
2024 J jnl
Text-to-Song: Towards Controllable Music Generation Incorporating Vocals and Accompaniment.
CoRR
Zhiqing Hong, Rongjie Huang, Xize Cheng, Yongqi Wang, Ruiqi Li, Fuming You, Zhou Zhao, Zhimeng Zhang
2024 — conf
TransFace: Unit-Based Audio-Visual Speech Synthesizer for Talking Head Translation.
ACL (Findings)
Xize Cheng, Rongjie Huang, Linjun Li, Zehan Wang, Tao Jin, Aoxiong Yin, Feiyang Chen, Xinyu Duan, Baoxing Huai, Zhou Zhao
2024 — conf
Uni-Dubbing: Zero-Shot Speech Synthesis from Visual Articulation.
ACL (1)
Songju Lei, Xize Cheng, Mengjiao Lyu, Jianqiao Hu, Jintao Tan, Runlin Liu, Lingyu Xiong, Tao Jin, Xiandong Li, Zhou Zhao
2024 A* conf
VoiceTuner: Self-Supervised Pre-training and Efficient Fine-tuning For Voice Generation.
ACM Multimedia
Rongjie Huang, Yongqi Wang, Ruofan Hu, Xiaoshan Xu, Zhiqing Hong, Dongchao Yang, Xize Cheng, Zehan Wang, Ziyue Jiang, Zhenhui Ye, Luping Liu, Siqi Zheng, Zhou Zhao
2024 — conf
Wav2SQL: Direct Generalizable Speech-To-SQL Parsing.
ACL (Findings)
Huadai Liu, Rongjie Huang, Jinzheng He, Gang Sun, Ran Shen, Xize Cheng, Zhou Zhao
2024 J jnl
WavChat: A Survey of Spoken Dialogue Models.
CoRR
Shengpeng Ji, Yifu Chen, Minghui Fang, Jialong Zuo, Jingyu Lu, Hanting Wang, Ziyue Jiang, Long Zhou, Shujie Liu, Xize Cheng, Xiaoda Yang, Zehan Wang, Qian Yang, Jian Li, Yidi Jiang, Jingzhen He, Yunfei Chu, Jin Xu, Zhou Zhao
2024 J jnl
WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modeling.
CoRR
Shengpeng Ji, Ziyue Jiang, Xize Cheng, Yifu Chen, Minghui Fang, Jialong Zuo, Qian Yang, Ruiqi Li, Ziang Zhang, Xiaoda Yang, Rongjie Huang, Yidi Jiang, Qian Chen, Siqi Zheng, Wen Wang, Zhou Zhao
2023 A* conf
3DRP-Net: 3D Relative Position-aware Network for 3D Visual Grounding.
EMNLP
Zehan Wang, Haifeng Huang, Yang Zhao, Linjun Li, Xize Cheng, Yichen Zhu, Aoxiong Yin, Zhou Zhao
2023 J jnl
3DRP-Net: 3D Relative Position-aware Network for 3D Visual Grounding.
CoRR
Zehan Wang, Haifeng Huang, Yang Zhao, Linjun Li, Xize Cheng, Yichen Zhu, Aoxiong Yin, Zhou Zhao
2023 — conf
AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation.
ACL (1)
Rongjie Huang, Huadai Liu, Xize Cheng, Yi Ren, Linjun Li, Zhenhui Ye, Jinzheng He, Lichao Zhang, Jinglin Liu, Xiang Yin, Zhou Zhao
2023 J jnl
AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation.
CoRR
Rongjie Huang, Huadai Liu, Xize Cheng, Yi Ren, Linjun Li, Zhenhui Ye, Jinzheng He, Lichao Zhang, Jinglin Liu, Xiang Yin, Zhou Zhao
2023 J jnl
Chat-3D v2: Bridging 3D Scene and Large Language Models with Object Identifiers.
CoRR
Haifeng Huang, Zehan Wang, Rongjie Huang, Luping Liu, Xize Cheng, Yang Zhao, Tao Jin, Zhou Zhao
2023 A* conf
Connecting Multi-modal Contrastive Representations.
NeurIPS
Zehan Wang, Yang Zhao, Xize Cheng, Haifeng Huang, Jiageng Liu, Aoxiong Yin, Li Tang, Linjun Li, Yongqi Wang, Ziang Zhang, Zhou Zhao
2023 J jnl
Connecting Multi-modal Contrastive Representations.
CoRR
Zehan Wang, Yang Zhao, Xize Cheng, Haifeng Huang, Jiageng Liu, Li Tang, Linjun Li, Yongqi Wang, Aoxiong Yin, Ziang Zhang, Zhou Zhao
2023 — conf
Contrastive Token-Wise Meta-Learning for Unseen Performer Visual Temporal-Aligned Translation.
ACL (Findings)
Linjun Li, Tao Jin, Xize Cheng, Ye Wang, Wang Lin, Rongjie Huang, Zhou Zhao
2023 A* conf
Distilling Coarse-to-Fine Semantic Matching Knowledge for Weakly Supervised 3D Visual Grounding.
ICCV
Zehan Wang, Haifeng Huang, Yang Zhao, Linjun Li, Xize Cheng, Yichen Zhu, Aoxiong Yin, Zhou Zhao
2023 J jnl
Distilling Coarse-to-Fine Semantic Matching Knowledge for Weakly Supervised 3D Visual Grounding.
CoRR
Zehan Wang, Haifeng Huang, Yang Zhao, Linjun Li, Xize Cheng, Yichen Zhu, Aoxiong Yin, Zhou Zhao
2023 A* conf
Exploring Group Video Captioning with Efficient Relational Approximation.
ICCV
Wang Lin, Tao Jin, Ye Wang, Wenwen Pan, Linjun Li, Xize Cheng, Zhou Zhao
2023 A* conf
MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition.
ICCV
Xize Cheng, Tao Jin, Rongjie Huang, Linjun Li, Wang Lin, Zehan Wang, Ye Wang, Huadai Liu, Aoxiong Yin, Zhou Zhao
2023 J jnl
MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition.
CoRR
Xize Cheng, Linjun Li, Tao Jin, Rongjie Huang, Wang Lin, Zehan Wang, Huangdai Liu, Ye Wang, Aoxiong Yin, Zhou Zhao
2023 — conf
OpenSR: Open-Modality Speech Recognition via Maintaining Multi-Modality Alignment.
ACL (1)
Xize Cheng, Tao Jin, Linjun Li, Wang Lin, Xinyu Duan, Zhou Zhao
2023 J jnl
OpenSR: Open-Modality Speech Recognition via Maintaining Multi-Modality Alignment.
CoRR
Xize Cheng, Tao Jin, Linjun Li, Wang Lin, Xinyu Duan, Zhou Zhao
2023 A* conf
Rethinking Missing Modality Learning from a Decoding Perspective.
ACM Multimedia
Tao Jin, Xize Cheng, Linjun Li, Wang Lin, Ye Wang, Zhou Zhao
2023 — conf
Semantic-conditioned Dual Adaptation for Cross-domain Query-based Visual Segmentation.
ACL (Findings)
Ye Wang, Tao Jin, Wang Lin, Xize Cheng, Linjun Li, Zhou Zhao
2023 — conf
TAVT: Towards Transferable Audio-Visual Text Generation.
ACL (1)
Wang Lin, Tao Jin, Wenwen Pan, Linjun Li, Xize Cheng, Ye Wang, Zhou Zhao
2023 J jnl
TransFace: Unit-Based Audio-Visual Speech Synthesizer for Talking Head Translation.
CoRR
Xize Cheng, Rongjie Huang, Linjun Li, Tao Jin, Zehan Wang, Aoxiong Yin, Minglei Li, Xinyu Duan, Changpeng Yang, Zhou Zhao
2023 J jnl
Wav2SQL: Direct Generalizable Speech-To-SQL Parsing.
CoRR
Huadai Liu, Rongjie Huang, Jinzheng He, Gang Sun, Ran Shen, Xize Cheng, Zhou Zhao
2023 — conf
Weakly-Supervised Spoken Video Grounding via Semantic Interaction Learning.
ACL (1)
Ye Wang, Wang Lin, Shengyu Zhang, Tao Jin, Linjun Li, Xize Cheng, Zhou Zhao
2022 J jnl
Diffusion Denoising Process for Perceptron Bias in Out-of-distribution Detection.
CoRR
Luping Liu, Yi Ren, Xize Cheng, Zhou Zhao