Masatoshi Uehara

85 papers A* 29A 3Journal 53

Year	Rank	Type	Title / Venue / Authors
2025	A*	`conf`	Adding Conditional Control to Diffusion Models with Reinforcement Learning. ICLR Yulai Zhao, Masatoshi Uehara, Gabriele Scalia, Sun-Yuan Kung, Tommaso Biancalani, Sergey Levine, Ehsan Hajiramezanali
2025	J	`jnl`	Dynamic Search for Inference-Time Alignment in Diffusion Models. CoRR Xiner Li, Masatoshi Uehara, Xingyu Su, Gabriele Scalia, Tommaso Biancalani, Aviv Regev, Sergey Levine, Shuiwang Ji
2025	J	`jnl`	Fast Rates for the Regret of Offline Reinforcement Learning. Math. Oper. Res. Yichun Hu, Nathan Kallus, Masatoshi Uehara
2025	A*	`conf`	Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design. ICLR Chenyu Wang, Masatoshi Uehara, Yichun He, Amy Wang, Avantika Lal, Tommi S. Jaakkola, Sergey Levine, Aviv Regev, Hanchen Wang, Tommaso Biancalani
2025	J	`jnl`	Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Review. CoRR Masatoshi Uehara, Yulai Zhao, Chenyu Wang, Xiner Li, Aviv Regev, Sergey Levine, Tommaso Biancalani
2025	J	`jnl`	Iterative Distillation for Reward-Guided Fine-Tuning of Diffusion Models in Biomolecular Design. CoRR Xingyu Su, Xiner Li, Masatoshi Uehara, Sunwoo Kim, Yulai Zhao, Gabriele Scalia, Ehsan Hajiramezanali, Tommaso Biancalani, Degui Zhi, Shuiwang Ji
2025	A*	`conf`	Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Design. ICML Masatoshi Uehara, Xingyu Su, Yulai Zhao, Xiner Li, Aviv Regev, Shuiwang Ji, Sergey Levine, Tommaso Biancalani
2025	J	`jnl`	Reward-Guided Iterative Refinement in Diffusion Models at Test-Time with Applications to Protein and DNA Design. CoRR Masatoshi Uehara, Xingyu Su, Yulai Zhao, Xiner Li, Aviv Regev, Shuiwang Ji, Sergey Levine, Tommaso Biancalani
2024	J	`jnl`	Adding Conditional Control to Diffusion Models with Reinforcement Learning. CoRR Yulai Zhao, Masatoshi Uehara, Gabriele Scalia, Tommaso Biancalani, Sergey Levine, Ehsan Hajiramezanali
2024	A*	`conf`	Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models. NeurIPS Masatoshi Uehara, Yulai Zhao, Ehsan Hajiramezanali, Gabriele Scalia, Gökcen Eraslan, Avantika Lal, Sergey Levine, Tommaso Biancalani
2024	J	`jnl`	Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models. CoRR Masatoshi Uehara, Yulai Zhao, Ehsan Hajiramezanali, Gabriele Scalia, Gökcen Eraslan, Avantika Lal, Sergey Levine, Tommaso Biancalani
2024	J	`jnl`	Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decoding. CoRR Xiner Li, Yulai Zhao, Chenyu Wang, Gabriele Scalia, Gökcen Eraslan, Surag Nair, Tommaso Biancalani, Aviv Regev, Sergey Levine, Masatoshi Uehara
2024	A*	`conf`	Feedback Efficient Online Fine-Tuning of Diffusion Models. ICML Masatoshi Uehara, Yulai Zhao, Kevin Black, Ehsan Hajiramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M. Tseng, Sergey Levine, Tommaso Biancalani
2024	J	`jnl`	Feedback Efficient Online Fine-Tuning of Diffusion Models. CoRR Masatoshi Uehara, Yulai Zhao, Kevin Black, Ehsan Hajiramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M. Tseng, Sergey Levine, Tommaso Biancalani
2024	J	`jnl`	Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design. CoRR Chenyu Wang, Masatoshi Uehara, Yichun He, Amy Wang, Tommaso Biancalani, Avantika Lal, Tommi S. Jaakkola, Sergey Levine, Hanchen Wang, Aviv Regev
2024	J	`jnl`	Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Control. CoRR Masatoshi Uehara, Yulai Zhao, Kevin Black, Ehsan Hajiramezanali, Gabriele Scalia, Nathaniel Lee Diamant, Alex M. Tseng, Tommaso Biancalani, Sergey Levine
2024	A	`conf`	Functional Graphical Models: Structure Enables Offline Data-Driven Optimization. AISTATS Kuba Grudzien Kuba, Masatoshi Uehara, Sergey Levine, Pieter Abbeel
2024	J	`jnl`	Functional Graphical Models: Structure Enables Offline Data-Driven Optimization. CoRR Jakub Grudzien Kuba, Masatoshi Uehara, Pieter Abbeel, Sergey Levine
2024	J	`jnl`	Localized Debiased Machine Learning: Efficient Inference on Quantile Treatment Effects and Beyond. J. Mach. Learn. Res. Nathan Kallus, Xiaojie Mao, Masatoshi Uehara
2024	A*	`conf`	Provable Offline Preference-Based Reinforcement Learning. ICLR Wenhao Zhan, Masatoshi Uehara, Nathan Kallus, Jason D. Lee, Wen Sun
2024	A*	`conf`	Provable Reward-Agnostic Preference-Based Reinforcement Learning. ICLR Wenhao Zhan, Masatoshi Uehara, Wen Sun, Jason D. Lee
2024	J	`jnl`	Regularized DeepIV with Model Selection. CoRR Zihao Li, Hui Lan, Vasilis Syrgkanis, Mengdi Wang, Masatoshi Uehara
2024	J	`jnl`	Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Review. CoRR Masatoshi Uehara, Yulai Zhao, Tommaso Biancalani, Sergey Levine
2023	A*	`conf`	Computationally Efficient PAC RL in POMDPs with Latent Determinism and Conditional Embeddings. ICML Masatoshi Uehara, Ayush Sekhari, Jason D. Lee, Nathan Kallus, Wen Sun
2023	A*	`conf`	Distributional Offline Policy Evaluation with Predictive Error Guarantees. ICML Runzhe Wu, Masatoshi Uehara, Wen Sun
2023	J	`jnl`	Distributional Offline Policy Evaluation with Predictive Error Guarantees. CoRR Runzhe Wu, Masatoshi Uehara, Wen Sun
2023	A*	`conf`	Future-Dependent Value-Based Off-Policy Evaluation in POMDPs. NeurIPS Masatoshi Uehara, Haruka Kiyohara, Andrew Bennett, Victor Chernozhukov, Nan Jiang, Nathan Kallus, Chengchun Shi, Wen Sun
2023	J	`jnl`	How to Query Human Feedback Efficiently in RL? CoRR Wenhao Zhan, Masatoshi Uehara, Wen Sun, Jason D. Lee
2023	A*	`conf`	Inference on Strongly Identified Functionals of Weakly Identified Functions. COLT Andrew Bennett, Nathan Kallus, Xiaojie Mao, Whitney Newey, Vasilis Syrgkanis, Masatoshi Uehara
2023	J	`jnl`	Minimax Instrumental Variable Regression and $L_2$ Convergence Guarantees without Identification or Closedness. CoRR Andrew Bennett, Nathan Kallus, Xiaojie Mao, Whitney Newey, Vasilis Syrgkanis, Masatoshi Uehara
2023	A*	`conf`	Minimax Instrumental Variable Regression and L COLT Andrew Bennett, Nathan Kallus, Xiaojie Mao, Whitney Newey, Vasilis Syrgkanis, Masatoshi Uehara
2023	A*	`conf`	Off-Policy Evaluation of Ranking Policies under Diverse User Behavior. KDD Haruka Kiyohara, Masatoshi Uehara, Yusuke Narita, Nobuyuki Shimizu, Yasuo Yamamoto, Yuta Saito
2023	J	`jnl`	Off-Policy Evaluation of Ranking Policies under Diverse User Behavior. CoRR Haruka Kiyohara, Masatoshi Uehara, Yusuke Narita, Nobuyuki Shimizu, Yasuo Yamamoto, Yuta Saito
2023	A*	`conf`	Offline Minimax Soft-Q-learning Under Realizability and Partial Coverage. NeurIPS Masatoshi Uehara, Nathan Kallus, Jason D. Lee, Wen Sun
2023	A*	`conf`	PAC Reinforcement Learning for Predictive State Representations. ICLR Wenhao Zhan, Masatoshi Uehara, Wen Sun, Jason D. Lee
2023	J	`jnl`	Provable Offline Reinforcement Learning with Human Feedback. CoRR Wenhao Zhan, Masatoshi Uehara, Nathan Kallus, Jason D. Lee, Wen Sun
2023	J	`jnl`	Refined Value-Based Offline RL under Realizability and Partial Coverage. CoRR Masatoshi Uehara, Nathan Kallus, Jason D. Lee, Wen Sun
2023	J	`jnl`	Source Condition Double Robust Inference on Functionals of Inverse Problems. CoRR Andrew Bennett, Nathan Kallus, Xiaojie Mao, Whitney Newey, Vasilis Syrgkanis, Masatoshi Uehara
2022	A*	`conf`	A Minimax Learning Approach to Off-Policy Evaluation in Confounded Partially Observable Markov Decision Processes. ICML Chengchun Shi, Masatoshi Uehara, Jiawei Huang, Nan Jiang
2022	J	`jnl`	A Review of Off-Policy Evaluation in Reinforcement Learning. CoRR Masatoshi Uehara, Chengchun Shi, Nathan Kallus
2022	J	`jnl`	Computationally Efficient PAC RL in POMDPs with Latent Determinism and Conditional Embeddings. CoRR Masatoshi Uehara, Ayush Sekhari, Jason D. Lee, Nathan Kallus, Wen Sun
2022	J	`jnl`	Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning Approach. CoRR Xuezhou Zhang, Yuda Song, Masatoshi Uehara, Mengdi Wang, Alekh Agarwal, Wen Sun
2022	A*	`conf`	Efficient Reinforcement Learning in Block MDPs: A Model-free Representation Learning approach. ICML Xuezhou Zhang, Yuda Song, Masatoshi Uehara, Mengdi Wang, Alekh Agarwal, Wen Sun
2022	J	`jnl`	Efficiently Breaking the Curse of Horizon in Off-Policy Evaluation with Double Reinforcement Learning. Oper. Res. Nathan Kallus, Masatoshi Uehara
2022	J	`jnl`	Future-Dependent Value-Based Off-Policy Evaluation in POMDPs. CoRR Masatoshi Uehara, Haruka Kiyohara, Andrew Bennett, Victor Chernozhukov, Nan Jiang, Nathan Kallus, Chengchun Shi, Wen Sun
2022	J	`jnl`	Optimal Fixed-Budget Best Arm Identification using the Augmented Inverse Probability Weighting Estimator in Two-Armed Gaussian Bandits with Unknown Variances. CoRR Masahiro Kato, Kaito Ariu, Masaaki Imaizumi, Masatoshi Uehara, Masahiro Nomura, Chao Qin
2022	J	`jnl`	PAC Reinforcement Learning for Predictive State Representations. CoRR Wenhao Zhan, Masatoshi Uehara, Wen Sun, Jason D. Lee
2022	A*	`conf`	Pessimistic Model-based Offline Reinforcement Learning under Partial Coverage. ICLR Masatoshi Uehara, Wen Sun
2022	A*	`conf`	Provably Efficient Reinforcement Learning in Partially Observable Dynamical Systems. NeurIPS Masatoshi Uehara, Ayush Sekhari, Jason D. Lee, Nathan Kallus, Wen Sun
2022	J	`jnl`	Provably Efficient Reinforcement Learning in Partially Observable Dynamical Systems. CoRR Masatoshi Uehara, Ayush Sekhari, Jason D. Lee, Nathan Kallus, Wen Sun
2022	A*	`conf`	Representation Learning for Online and Offline RL in Low-rank MDPs. ICLR Masatoshi Uehara, Xuezhou Zhang, Wen Sun
2021	J	`jnl`	A Minimax Learning Approach to Off-Policy Evaluation in Partially Observable Markov Decision Processes. CoRR Chengchun Shi, Masatoshi Uehara, Nan Jiang
2021	J	`jnl`	Causal Inference Under Unmeasured Confounding With Negative Controls: A Minimax Learning Approach. CoRR Nathan Kallus, Xiaojie Mao, Masatoshi Uehara
2021	A*	`conf`	Fast Rates for the Regret of Offline Reinforcement Learning. COLT Yichun Hu, Nathan Kallus, Masatoshi Uehara
2021	J	`jnl`	Fast Rates for the Regret of Offline Reinforcement Learning. CoRR Yichun Hu, Nathan Kallus, Masatoshi Uehara
2021	J	`jnl`	Finite Sample Analysis of Minimax Offline Reinforcement Learning: Completeness, Fast Rates and First-Order Efficiency. CoRR Masatoshi Uehara, Masaaki Imaizumi, Nan Jiang, Nathan Kallus, Wen Sun, Tengyang Xie
2021	J	`jnl`	Information criteria for non-normalized models. J. Mach. Learn. Res. Takeru Matsuda, Masatoshi Uehara, Aapo Hyvärinen
2021	A*	`conf`	Mitigating Covariate Shift in Imitation Learning via Offline Data With Partial Coverage. NeurIPS Jonathan D. Chang, Masatoshi Uehara, Dhruv Sreenivas, Rahul Kidambi, Wen Sun
2021	J	`jnl`	Mitigating Covariate Shift in Imitation Learning via Offline Data Without Great Coverage. CoRR Jonathan D. Chang, Masatoshi Uehara, Dhruv Sreenivas, Rahul Kidambi, Wen Sun
2021	A*	`conf`	Optimal Off-Policy Evaluation from Multiple Logging Policies. ICML Nathan Kallus, Yuta Saito, Masatoshi Uehara
2021	J	`jnl`	Pessimistic Model-based Offline RL: PAC Bounds and Posterior Sampling under Partial Coverage. CoRR Masatoshi Uehara, Wen Sun
2021	J	`jnl`	Representation Learning for Online and Offline RL in Low-rank MDPs. CoRR Masatoshi Uehara, Xuezhou Zhang, Wen Sun
2020	A	`conf`	A Unified Statistically Efficient Estimation Framework for Unnormalized Models. AISTATS Masatoshi Uehara, Takafumi Kanamori, Takashi Takenouchi, Takeru Matsuda
2020	J	`jnl`	Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes. J. Mach. Learn. Res. Nathan Kallus, Masatoshi Uehara
2020	A*	`conf`	Double Reinforcement Learning for Efficient and Robust Off-Policy Evaluation. ICML Nathan Kallus, Masatoshi Uehara
2020	A*	`conf`	Doubly Robust Off-Policy Value and Gradient Estimation for Deterministic Policies. NeurIPS Nathan Kallus, Masatoshi Uehara
2020	J	`jnl`	Doubly Robust Off-Policy Value and Gradient Estimation for Deterministic Policies. CoRR Nathan Kallus, Masatoshi Uehara
2020	J	`jnl`	Efficient Evaluation of Natural Stochastic Policies in Offline Reinforcement Learning. CoRR Nathan Kallus, Masatoshi Uehara
2020	A	`conf`	Imputation estimators for unnormalized models with missing data. AISTATS Masatoshi Uehara, Takeru Matsuda, Jae Kwang Kim
2020	A*	`conf`	Minimax Weight and Q-Function Learning for Off-Policy Evaluation. ICML Masatoshi Uehara, Jiawei Huang, Nan Jiang
2020	A*	`conf`	Off-Policy Evaluation and Learning for External Validity under a Covariate Shift. NeurIPS Masatoshi Uehara, Masahiro Kato, Shota Yasui
2020	J	`jnl`	Off-Policy Evaluation and Learning for External Validity under a Covariate Shift. CoRR Masahiro Kato, Masatoshi Uehara, Shota Yasui
2020	J	`jnl`	Optimal Off-Policy Evaluation from Multiple Logging Policies. CoRR Nathan Kallus, Yuta Saito, Masatoshi Uehara
2020	A*	`conf`	Statistically Efficient Off-Policy Policy Gradients. ICML Nathan Kallus, Masatoshi Uehara
2020	J	`jnl`	Statistically Efficient Off-Policy Policy Gradients. CoRR Nathan Kallus, Masatoshi Uehara
2019	J	`jnl`	Double Reinforcement Learning for Efficient Off-Policy Evaluation in Markov Decision Processes. CoRR Nathan Kallus, Masatoshi Uehara
2019	J	`jnl`	Efficiently Breaking the Curse of Horizon: Double Reinforcement Learning in Infinite-Horizon Processes. CoRR Nathan Kallus, Masatoshi Uehara
2019	J	`jnl`	Imputation estimators for unnormalized models with missing data. CoRR Masatoshi Uehara, Takeru Matsuda, Jae Kwang Kim
2019	J	`jnl`	Information criteria for non-normalized models. CoRR Takeru Matsuda, Masatoshi Uehara, Aapo Hyvärinen
2019	A*	`conf`	Intrinsically Efficient, Stable, and Bounded Off-Policy Evaluation for Reinforcement Learning. NeurIPS Nathan Kallus, Masatoshi Uehara
2019	J	`jnl`	Intrinsically Efficient, Stable, and Bounded Off-Policy Evaluation for Reinforcement Learning. CoRR Nathan Kallus, Masatoshi Uehara
2019	J	`jnl`	Localized Debiased Machine Learning: Efficient Estimation of Quantile Treatment Effects, Conditional Value at Risk, and Beyond. CoRR Nathan Kallus, Xiaojie Mao, Masatoshi Uehara
2019	J	`jnl`	Minimax Weight and Q-Function Learning for Off-Policy Evaluation. CoRR Masatoshi Uehara, Nan Jiang
2019	J	`jnl`	Unified estimation framework for unnormalized models with statistical efficiency. CoRR Masatoshi Uehara, Takafumi Kanamori, Takashi Takenouchi, Takeru Matsuda
2018	J	`jnl`	Analysis of Noise Contrastive Estimation from the Perspective of Asymptotic Variance. CoRR Masatoshi Uehara, Takeru Matsuda, Fumiyasu Komaki