Toby Jia-Jun Li

171 papers A* 33A 7B 8C 1Journal 88Unranked 32
YearRankTypeTitle / Venue / Authors
2026 J jnl
Context-aware code summary generation.
J. Syst. Softw.
Chia-Yi Su, Aakash Bansal, Yu Huang, Toby Jia-Jun Li, Collin McMillan
2026 A* conf
Crepe: A Mobile Screen Data Collector Using Graph Query.
CHI
Yuwen Lu, Meng Chen, Qi Zhao, Victor V. Cox, Yang Yang, Meng Jiang, Jay Brockman, Tamara Kay, Toby Jia-Jun Li
2026 A* conf
Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight.
CHI
Jingyu Tang, Chaoran Chen, Jiawen Li, Zhiping Zhang, Bingcan Guo, Ibrahim Khalilov, Simret Araya Gebreegziabher, Bingsheng Yao, Dakuo Wang, Yanfang Ye, Tianshi Li, Ziang Xiao, Yaxing Yao, Toby Jia-Jun Li
2026 A* conf
Designing Staged Evaluation Workflows for LLMs: Integrating Domain Experts, Lay Users, and Model-Generated Evaluation Criteria.
CHI
Annalisa Szymanski, Simret Araya Gebreegziabher, Oghenemaro Anuyah, Ronald A. Metoyer, Toby Jia-Jun Li
2026 — conf
EvalAgent: Interactive Comparative Evaluation of Computer-Using GUI Agents.
IUI Companion
Yukun Yang, Simret Araya Gebreegziabher, Hojun Yoo, Charles Chiang, Chaoran Chen, Annalisa Szymanski, Hyo Jin Do, Zahra Ashktorab, Werner Geyer, Diego Gómez-Zará, Toby Jia-Jun Li
2026 — conf
From Human-Human Collaboration to Human-Agent Collaboration: A Vision, Design Philosophy, and an Empirical Framework for Achieving Successful Partnerships Between Humans and LLM Agents.
CHI Extended Abstracts
Bingsheng Yao, Chaoran Chen, April Yi Wang, Tongshuang Wu, Toby Jia-Jun Li, Dakuo Wang
2026 J jnl
From Human-Human Collaboration to Human-Agent Collaboration: A Vision, Design Philosophy, and an Empirical Framework for Achieving Successful Partnerships Between Humans and LLM Agents.
CoRR
Bingsheng Yao, Chaoran Chen, April Yi Wang, Sherry Tongshuang Wu, Toby Jia-Jun Li, Dakuo Wang
2026 — conf
From Verification Burden to Trusted Collaboration: Design Goals for LLM-Assisted Literature Reviews.
IUI Companion
Brenda Nogueira, Werner Geyer, Andrew A. Anderson, Toby Jia-Jun Li, Dongwhi Kim, Nuno Moniz, Nitesh V. Chawla
2026 J jnl
Investigating the Feasibility of Conducting Webcam-Based Eye-Tracking Studies in Code Comprehension.
IEEE Trans. Software Eng.
Zihan Fang, Robert Wallace, Zachary Karas, Toby Jia-Jun Li, Collin McMillan, Yu Huang
2026 A conf
Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study.
IUI
Annalisa Szymanski, Oghenemaro Anuyah, Toby Jia-Jun Li, Ronald A. Metoyer
2026 J jnl
Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study.
CoRR
Annalisa Szymanski, Oghenemaro Anuyah, Toby Jia-Jun Li, Ronald A. Metoyer
2026 J jnl
Lessons from Real-World Deployment of a Cognition-Preserving Writing Tool: Students Actively Engage with Critical Thinking and Planning Affordances.
CoRR
Yinuo Yang, Zheng Zhang, Ningzhi Tang, Xu Wang, Alex Ambrose, Nathaniel Myers, Patrick Clauss, Toby Jia-Jun Li
2026 — conf
MultEval: Collaboratively Creating Criteria for LLM-as-a-Judge Systems.
IUI Companion
Charles Chiang, Simret Araya Gebreegziabher, Annalisa Szymanski, Yukun Yang, Hyo Jin Do, Zahra Ashktorab, Werner Geyer, Toby Jia-Jun Li, Diego Gómez-Zará
2026 A* conf
My Favorite Streamer is an LLM: Discovering, Bonding, and Co-Creating in AI VTuber Fandom.
CHI
Jiayi Ye, Chaoran Chen, Yue Huang, Yanfang Ye, Toby Jia-Jun Li, Xiangliang Zhang
2026 J jnl
PriviSense: A Frida-Based Framework for Multi-Sensor Spoofing on Android.
CoRR
Ibrahim Khalilov, Chaoran Chen, Ziang Xiao, Tianshi Li, Toby Jia-Jun Li, Yaxing Yao
2026 — conf
TRUST-CUA: Trustworthy Computer-Using Generalist Agents for Intelligent User Interfaces (Workshop).
IUI Companion
Toby Jia-Jun Li, Segev Shlomov, Xiang Deng, Ronen I. Brafman, Avi Yaeli, Zora Zhiruo Wang
2026 A conf
The Behavioral Fabric of LLM-Powered GUI Agents: Human Values and Interaction Outcomes.
IUI
Simret Araya Gebreegziabher, Yukun Yang, Charles Chiang, Hojun Yoo, Chaoran Chen, Hyo Jin Do, Zahra Ashktorab, Werner Geyer, Diego Gómez-Zará, Toby Jia-Jun Li
2026 J jnl
The Behavioral Fabric of LLM-Powered GUI Agents: Human Values and Interaction Outcomes.
CoRR
Simret Araya Gebreegziabher, Yukun Yang, Charles Chiang, Hojun Yoo, Chaoran Chen, Hyo Jin Do, Zahra Ashktorab, Werner Geyer, Diego Gómez-Zará, Toby Jia-Jun Li
2026 A* conf
Through the Lens of Human-Human Collaboration: An Configurable Research Platform for Exploring Human-Agent Collaboration.
CHI
Bingsheng Yao, Jiaju Chen, Chaoran Chen, April Yi Wang, Toby Jia-Jun Li, Dakuo Wang
2026 A* conf
Vistoria: A Multimodal System to Support Fictional Story Writing through Instrumental Image-Text Co-Editing.
CHI
Kexue Fu, Jingfei Huang, Long Ling, Sumin Hong, Yihang Zuo, Ray LC, Toby Jia-Jun Li
2025 J jnl
'I'm Categorizing LLM as a Productivity Tool': Examining Ethics of LLM Use in HCI Research Practices.
Proc. ACM Hum. Comput. Interact.
Shivani Kapania, Ruiyi Wang, Toby Jia-Jun Li, Tianshi Li, Hong Shen
2025 J jnl
AI-Mediated Code Comment Improvement.
CoRR
Maria Dhakal, Chia-Yi Su, Robert Wallace, Chris Fakhimi, Aakash Bansal, Toby Jia-Jun Li, Yu Huang, Collin McMillan
2025 J jnl
ALLOY: Generating Reusable Agent Workflows from User Demonstration.
CoRR
Jiawen Li, Zheng Ning, Yuan Tian, Toby Jia-Jun Li
2025 J jnl
AROMA: Mixed-Initiative AI Assistance for Non-Visual Cooking by Grounding Multi-modal Information Between Reality and Videos.
CoRR
Zheng Ning, Leyang Li, Daniel Killough, Jooyoung Seo, Patrick Carrington, Yapeng Tian, Yuhang Zhao, Franklin Mingzhe Li, Toby Jia-Jun Li
2025 A* conf
AROMA: Mixed-Initiative AI Assistance for Non-Visual Cooking by Grounding Multimodal Information Between Reality and Videos.
UIST
Zheng Ning, Leyang Li, Daniel Killough, Jooyoung Seo, Patrick Carrington, Yapeng Tian, Yuhang Zhao, Franklin Mingzhe Li, Toby Jia-Jun Li
2025 B conf
AgentPbD: Interactive Agentic Workflow Generation from User Demonstration on Web Browsers.
VL/HCC
Jiawen Li, Zheng Ning, Yuan Tian, Toby Jia-Jun Li
2025 J jnl
Beyond Permissions: Investigating Mobile Personalization with Simulated Personas.
CoRR
Ibrahim Khalilov, Chaoran Chen, Ziang Xiao, Tianshi Li, Toby Jia-Jun Li, Yaxing Yao
2025 A conf
CLEAR: Towards Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation for Large Language Model Applications.
IUI
Chaoran Chen, Daodao Zhou, Yanfang Ye, Toby Jia-Jun Li, Yaxing Yao
2025 — conf
CSCW Contributions to Critical Futures of Work.
CSCW Companion
Alina Lushnikova, Michael Muller, Shaowen Bardzell, Toby Jia-Jun Li, Saiph Savage
2025 A* conf
Careful About What App Promotion Ads Recommend! Detecting and Explaining Malware Promotion via App Promotion Graph.
NDSS
Shang Ma, Chaoran Chen, Shao Yang, Shifu Hou, Toby Jia-Jun Li, Xusheng Xiao, Tao Xie, Yanfang Ye
2025 J jnl
Dark Patterns Meet GUI Agents: LLM Agent Susceptibility to Manipulative Interfaces and the Role of Human Oversight.
CoRR
Jingyu Tang, Chaoran Chen, Jiawen Li, Zhiping Zhang, Bingcan Guo, Ibrahim Khalilov, Simret Araya Gebreegziabher, Bingsheng Yao, Dakuo Wang, Yanfang Ye, Tianshi Li, Ziang Xiao, Yaxing Yao, Toby Jia-Jun Li
2025 B conf
Exploring Direct Instruction and Summary-Mediated Prompting in LLM-Assisted Code Modification.
VL/HCC
Ningzhi Tang, Emory Smith, Yu Huang, Collin McMillan, Toby Jia-Jun Li
2025 J jnl
Exploring Direct Instruction and Summary-Mediated Prompting in LLM-Assisted Code Modification.
CoRR
Ningzhi Tang, Emory Smith, Yu Huang, Collin McMillan, Toby Jia-Jun Li
2025 J jnl
EyeMulator: Improving Code Language Models by Mimicking Human Visual Attention.
CoRR
Yifan Zhang, Chen Huang, Yueke Zhang, Jiahao Zhang, Toby Jia-Jun Li, Collin McMillan, Kevin Leach, Yu Huang
2025 A* conf
From Operation to Cognition: Automatic Modeling Cognitive Dependencies from User Demonstrations for GUI Task Automation.
CHI
Yiwen Yin, Yu Mei, Chun Yu, Toby Jia-Jun Li, Aamir Khan Jadoon, Sixiang Cheng, Weinan Shi, Mohan Chen, Yuanchun Shi
2025 J jnl
From Verification Burden to Trusted Collaboration: Design Goals for LLM-Assisted Literature Reviews.
CoRR
Brenda Nogueira, Werner Geyer, Andrew A. Anderson, Toby Jia-Jun Li, Dongwhi Kim, Nuno Moniz, Nitesh V. Chawla
2025 A* conf
GLITTER: An AI-assisted Platform for Material-Grounded Asynchronous Discussion in Flipped Learning.
UIST
Weirui Peng, Yinuo Yang, Zheng Zhang, Toby Jia-Jun Li
2025 J jnl
GLITTER: An AI-assisted Platform for Material-Grounded Asynchronous Discussion in Flipped Learning.
CoRR
Weirui Peng, Yinuo Yang, Zheng Zhang, Toby Jia-Jun Li
2025 A* conf
HAIPS '25: First ACM CCS Workshop on Human-Centered AI Privacy and Security.
CCS
Tianshi Li, Toby Jia-Jun Li, Yaxing Yao, Sauvik Das
2025 A* conf
Hashtag Re-Appropriation for Audience Control on Recommendation-Driven Social Media Xiaohongshu (rednote).
CHI
Ruyuan Wan, Lingbo Tong, Tiffany Knearem, Toby Jia-Jun Li, Ting-Hao Kenneth Huang, Qunfang Wu
2025 J jnl
Hashtag Re-Appropriation for Audience Control on Recommendation-Driven Social Media Xiaohongshu (rednote).
CoRR
Ruyuan Wan, Lingbo Tong, Tiffany Knearem, Toby Jia-Jun Li, Ting-Hao (Kenneth) Huang, Qunfang Wu
2025 A* conf
LADICA: A Large Shared Display Interface for Generative AI Cognitive Assistance in Co-located Team Collaboration.
CHI
Zheng Zhang, Weirui Peng, Xinyue Chen, Luke Cao, Toby Jia-Jun Li
2025 — conf
Leveraging Variation Theory in Counterfactual Data Augmentation for Optimized Active Learning.
ACL (Findings)
Simret Araya Gebreegziabher, Kuangshi Ai, Zheng Zhang, Elena L. Glassman, Toby Jia-Jun Li
2025 A conf
Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks.
IUI
Annalisa Szymanski, Noah Ziems, Heather A. Eicher-Miller, Toby Jia-Jun Li, Meng Jiang, Ronald A. Metoyer
2025 J jnl
MORA: AI-Mediated Story-Based practice for Speech Sound Disorder from Clinic to Home.
CoRR
Sumin Hong, Xavier Briggs, Qingxiao Zheng, Yao Du, Jinjun Xiong, Toby Jia-Jun Li
2025 — conf
MetricMate: An Interactive Tool for Generating Evaluation Criteria for LLM-as-a-Judge Workflow.
CHIWORK
Simret Araya Gebreegziabher, Charles Chiang, Zichu Wang, Zahra Ashktorab, Michelle Brachman, Werner Geyer, Toby Jia-Jun Li, Diego Gómez-Zará
2025 — conf
MetricMate: An Interactive Tool for Generating Evaluation Criteria for LLM-as-a-Judge Workflow.
IUI Workshops
Simret Araya Gebreegziabher, Charles Chiang, Zichu Wang, Zahra Ashktorab, Michelle Brachman, Werner Geyer, Toby Jia-Jun Li, Diego Gómez-Zará
2025 J jnl
My Favorite Streamer is an LLM: Discovering, Bonding, and Co-Creating in AI VTuber Fandom.
CoRR
Jiayi Ye, Chaoran Chen, Yue Huang, Yanfang Ye, Toby Jia-Jun Li, Xiangliang Zhang
2025 J jnl
NaturalEdit: Code Modification through Direct Interaction with Adaptive Natural Language Representation.
CoRR
Ningzhi Tang, David Meininger, Gelei Xu, Yiyu Shi, Yu Huang, Collin McMillan, Toby Jia-Jun Li
2025 J jnl
OPeRA: A Dataset of Observation, Persona, Rationale, and Action for Evaluating LLMs on Human Online Shopping Behavior Simulation.
CoRR
Ziyi Wang, Yuxuan Lu, Wenbo Li, Amirali Amini, Bo Sun, Yakov Bart, Weimin Lyu, Jiri Gesi, Tian Wang, Jing Huang, Yu Su, Upol Ehsan, Malihe Alikhani, Toby Jia-Jun Li, Lydia B. Chilton, Dakuo Wang
2025 J jnl
PrivacyMotiv: Speculative Persona Journeys for Empathic and Motivating Privacy Reviews in UX Design.
CoRR
Zeya Chen, Jianing Wen, Ruth Schmidt, Yaxing Yao, Toby Jia-Jun Li, Tianshi Li
2025 J jnl
Programmer Visual Attention During Context-Aware Code Summarization.
IEEE Trans. Software Eng.
Robert Wallace, Aakash Bansal, Zachary Karas, Ningzhi Tang, Yu Huang, Toby Jia-Jun Li, Collin McMillan
2025 J jnl
Sketchar: Supporting Character Design and Illustration Prototyping Using Generative AI.
CoRR
Long Ling, Xinyi Chen, Ruoyu Wen, Toby Jia-Jun Li, Ray LC
2025 A* conf
Supporting Co-Adaptive Machine Teaching through Human Concept Learning and Cognitive Theories.
CHI
Simret Araya Gebreegziabher, Yukun Yang, Elena L. Glassman, Toby Jia-Jun Li
2025 J jnl
The Obvious Invisible Threat: LLM-Powered GUI Agents' Vulnerability to Fine-Print Injections.
CoRR
Chaoran Chen, Zhiping Zhang, Bingcan Guo, Shang Ma, Ibrahim Khalilov, Simret Araya Gebreegziabher, Yanfang Ye, Ziang Xiao, Yaxing Yao, Tianshi Li, Toby Jia-Jun Li
2025 J jnl
Through the Lens of Human-Human Collaboration: A Configurable Research Platform for Exploring Human-Agent Collaboration.
CoRR
Bingsheng Yao, Jiaju Chen, Chaoran Chen, April Yi Wang, Toby Jia-Jun Li, Dakuo Wang
2025 J jnl
Toward a Human-Centered Evaluation Framework for Trustworthy LLM-Powered GUI Agents.
CoRR
Chaoran Chen, Zhiping Zhang, Ibrahim Khalilov, Bingcan Guo, Simret Araya Gebreegziabher, Yanfang Ye, Ziang Xiao, Yaxing Yao, Tianshi Li, Toby Jia-Jun Li
2025 — conf
Towards a Design Guideline for RPA Evaluation: A Survey of Large Language Model-Based Role-Playing Agents.
ACL (Findings)
Chaoran Chen, Bingsheng Yao, Ruishi Zou, Wenyue Hua, Weimin Lyu, Toby Jia-Jun Li, Dakuo Wang
2025 J jnl
Towards a Design Guideline for RPA Evaluation: A Survey of Large Language Model-Based Role-Playing Agents.
CoRR
Chaoran Chen, Bingsheng Yao, Ruishi Zou, Wenyue Hua, Weimin Lyu, Toby Jia-Jun Li, Dakuo Wang
2025 J jnl
UXAgent: A System for Simulating Usability Testing of Web Design with LLM Agents.
CoRR
Yuxuan Lu, Bingsheng Yao, Hansu Gu, Jing Huang, Jessie Wang, Yang Li, Jiri Gesi, Qi He, Toby Jia-Jun Li, Dakuo Wang
2025 — conf
UXAgent: An LLM Agent-Based Usability Testing Framework for Web Design.
CHI Extended Abstracts
Yuxuan Lu, Bingsheng Yao, Hansu Gu, Jing Huang, Zheshen Jessie Wang, Yang Li, Jiri Gesi, Qi He, Toby Jia-Jun Li, Dakuo Wang
2025 J jnl
UXAgent: An LLM Agent-Based Usability Testing Framework for Web Design.
CoRR
Yuxuan Lu, Bingsheng Yao, Hansu Gu, Jing Huang, Zheshen Jessie Wang, Laurence Li, Jiri Gesi, Qi He, Toby Jia-Jun Li, Dakuo Wang
2025 J jnl
Uncovering Disparities in Rideshare Drivers Earning and Work Patterns: A Case Study of Chicago.
CoRR
Hy Dang, Yuwen Lu, Jason Spicer, Tamara Kay, Di Yang, Yang Yang, Jay Brockman, Meng Jiang, Toby Jia-Jun Li
2025 A conf
Unequal Opportunities: Examining the Bias in Geographical Recommendations by Large Language Models.
IUI
Shiran Dudy, Thulasi Tholeti, Resmi Ramachandranpillai, Muhammad Ali, Toby Jia-Jun Li, Ricardo Baeza-Yates
2025 J jnl
Unequal Opportunities: Examining the Bias in Geographical Recommendations by Large Language Models.
CoRR
Shiran Dudy, Thulasi Tholeti, Resmi Ramachandranpillai, Muhammad Ali, Toby Jia-Jun Li, Ricardo Baeza-Yates
2025 J jnl
Vistoria: A Multimodal System to Support Fictional Story Writing through Instrumental Text-Image Co-Editing.
CoRR
Kexue Fu, Jingfei Huang, Long Ling, Sumin Hong, Yihang Zuo, Ray LC, Toby Jia-Jun Li
2025 A* conf
Why am I seeing this: Democratizing End User Auditing for Online Content Recommendations.
UIST
Chaoran Chen, Leyang Li, Luke Cao, Yanfang Ye, Tianshi Li, Yaxing Yao, Toby Jia-Jun Li
2024 J jnl
"I'm categorizing LLM as a productivity tool": Examining ethics of LLM use in HCI research practices.
CoRR
Shivani Kapania, Ruiyi Wang, Toby Jia-Jun Li, Tianshi Li, Hong Shen
2024 J jnl
A Study on Developer Behaviors for Validating and Repairing LLM-Generated Code Using Eye Tracking and IDE Actions.
CoRR
Ningzhi Tang, Meng Chen, Zheng Ning, Aakash Bansal, Yu Huang, Collin McMillan, Toby Jia-Jun Li
2024 J jnl
A Tale of Two Comprehensions? Analyzing Student Programmer Attention during Code Summarization.
ACM Trans. Softw. Eng. Methodol.
Zachary Karas, Aakash Bansal, Yifan Zhang, Toby Jia-Jun Li, Collin McMillan, Yu Huang
2024 — conf
A Taxonomy for Human-LLM Interaction Modes: An Initial Exploration.
CHI Extended Abstracts
Jie Gao, Simret Araya Gebreegziabher, Kenny Tsu Wei Choo, Toby Jia-Jun Li, Simon Tangi Perrault, Thomas W. Malone
2024 J jnl
A Taxonomy for Human-LLM Interaction Modes: An Initial Exploration.
CoRR
Jie Gao, Simret Araya Gebreegziabher, Kenny Tsu Wei Choo, Toby Jia-Jun Li, Simon Tangi Perrault, Thomas W. Malone
2024 J jnl
AI Assistance for UX: A Literature Review Through Human-Centered AI.
CoRR
Yuwen Lu, Yuewen Yang, Qinyi Zhao, Chengzhi Zhang, Toby Jia-Jun Li
2024 A* conf
An Empathy-Based Sandbox Approach to Bridge the Privacy Gap among Attitudes, Goals, Knowledge, and Behaviors.
CHI
Chaoran Chen, Weijun Li, Wenxin Song, Yanfang Ye, Yaxing Yao, Toby Jia-Jun Li
2024 A* conf
AutoDroid: LLM-powered Task Automation in Android.
MobiCom
Hao Wen, Yuanchun Li, Guohong Liu, Shanhui Zhao, Tao Yu, Toby Jia-Jun Li, Shiqi Jiang, Yunhao Liu, Yaqin Zhang, Yunxin Liu
2024 J jnl
CLEAR: Towards Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation for Large Language Model Applications.
CoRR
Chaoran Chen, Daodao Zhou, Yanfang Ye, Toby Jia-Jun Li, Yaxing Yao
2024 J jnl
Careful About What App Promotion Ads Recommend! Detecting and Explaining Malware Promotion via App Promotion Graph.
CoRR
Shang Ma, Chaoran Chen, Shao Yang, Shifu Hou, Toby Jia-Jun Li, Xusheng Xiao, Tao Xie, Yanfang Ye
2024 — conf
Challenges and Opportunities of LLM-Based Synthetic Personae and Data in HCI.
CSCW Companion
Mirjana Prpa, Giovanni Maria Troiano, Bingsheng Yao, Toby Jia-Jun Li, Dakuo Wang, Hansu Gu
2024 J jnl
ChartifyText: Automated Chart Generation from Data-Involved Texts via LLM.
CoRR
Songheng Zhang, Lei Wang, Toby Jia-Jun Li, Qiaomu Shen, Yixin Cao, Yong Wang
2024 B conf
CoCo Matrix: Taxonomy of Cognitive Contributions in Co-writing with Intelligent Agents.
Creativity & Cognition
Ruyuan Wan, Simret Araya Gebreegziabher, Toby Jia-Jun Li, Karla Badillo-Urquiola
2024 J jnl
CoCo Matrix: Taxonomy of Cognitive Contributions in Co-writing with Intelligent Agents.
CoRR
Ruyuan Wan, Simret Araya Gebreegziabher, Toby Jia-Jun Li, Karla A. Badillo-Urquiola
2024 — conf
CodeGRITS: A Research Toolkit for Developer Behavior and Eye Tracking in IDE.
ICSE Companion
Ningzhi Tang, Junwen An, Meng Chen, Aakash Bansal, Yu Huang, Collin McMillan, Toby Jia-Jun Li
2024 A* conf
CollabCoder: A Lower-barrier, Rigorous Workflow for Inductive Collaborative Qualitative Analysis with Large Language Models.
CHI
Jie Gao, Yuchen Guo, Gionnieve Lim, Tianqin Zhang, Zheng Zhang, Toby Jia-Jun Li, Simon Tangi Perrault
2024 J jnl
Comparing Criteria Development Across Domain Experts, Lay Users, and Models in Large Language Model Evaluation.
CoRR
Annalisa Szymanski, Simret Araya Gebreegziabher, Oghenemaro Anuyah, Ronald A. Metoyer, Toby Jia-Jun Li
2024 — conf
Computational Methodologies for Understanding, Automating, and Evaluating User Interfaces.
CHI Extended Abstracts
Yue Jiang, Yuwen Lu, Tiffany Knearem, Clara E. Kliman-Silver, Christof Lutteroth, Toby Jia-Jun Li, Jeffrey Nichols, Wolfgang Stuerzlinger
2024 J jnl
Context-aware Code Summary Generation.
CoRR
Chia-Yi Su, Aakash Bansal, Yu Huang, Toby Jia-Jun Li, Collin McMillan
2024 J jnl
Crepe: A Mobile Screen Data Collector Using Graph Query.
CoRR
Yuwen Lu, Meng Chen, Qi Zhao, Victor V. Cox, Yang Yang, Meng Jiang, Jay Brockman, Tamara Kay, Toby Jia-Jun Li
2024 B conf
Developer Behaviors in Validating and Repairing LLM-Generated Code Using IDE and Eye Tracking.
VL/HCC
Ningzhi Tang, Meng Chen, Zheng Ning, Aakash Bansal, Yu Huang, Collin McMillan, Toby Jia-Jun Li
2024 J jnl
EyeTrans: Merging Human and Machine Attention for Neural Code Summarization.
CoRR
Yifan Zhang, Jiliang Li, Zachary Karas, Aakash Bansal, Toby Jia-Jun Li, Collin McMillan, Kevin Leach, Yu Huang
2024 J jnl
EyeTrans: Merging Human and Machine Attention for Neural Code Summarization.
Proc. ACM Softw. Eng.
Yifan Zhang, Jiliang Li, Zachary Karas, Aakash Bansal, Toby Jia-Jun Li, Collin McMillan, Kevin Leach, Yu Huang
2024 J jnl
Flowy: Supporting UX Design Decisions Through AI-Driven Pattern Annotation in Multi-Screen User Flows.
CoRR
Yuwen Lu, Ziang Tong, Qinyi Zhao, Yewon Oh, Bryan Wang, Toby Jia-Jun Li
2024 J jnl
From Awareness to Action: Exploring End-User Empowerment Interventions for Dark Patterns in UX.
Proc. ACM Hum. Comput. Interact.
Yuwen Lu, Chao Zhang, Yuewen Yang, Yaxing Yao, Toby Jia-Jun Li
2024 J jnl
Insights into Natural Language Database Query Errors: from Attention Misalignment to User Handling Strategies.
ACM Trans. Interact. Intell. Syst.
Zheng Ning, Yuan Tian, Zheng Zhang, Tianyi Zhang, Toby Jia-Jun Li
2024 J jnl
LADICA: A Large Shared Display Interface for Generative AI Cognitive Assistance in Co-Located Team Collaboration.
CoRR
Zheng Zhang, Weirui Peng, Xinyue Chen, Luke Cao, Toby Jia-Jun Li
2024 J jnl
Leveraging Variation Theory in Counterfactual Data Augmentation for Optimized Active Learning.
CoRR
Simret Araya Gebreegziabher, Kuangshi Ai, Zheng Zhang, Elena L. Glassman, Toby Jia-Jun Li
2024 J jnl
Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks.
CoRR
Annalisa Szymanski, Noah Ziems, Heather A. Eicher-Miller, Toby Jia-Jun Li, Meng Jiang, Ronald A. Metoyer
2024 A* conf
Luminate: Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation.
CHI
Sangho Suh, Meng Chen, Bryan Min, Toby Jia-Jun Li, Haijun Xia
2024 B conf
MIMOSA: Human-AI Co-Creation of Computational Spatial Audio Effects on Videos.
Creativity & Cognition
Zheng Ning, Zheng Zhang, Jerrick Ban, Kaiwen Jiang, Ruohong Gan, Yapeng Tian, Toby Jia-Jun Li
2024 J jnl
MIMOSA: Human-AI Co-Creation of Computational Spatial Audio Effects on Videos.
CoRR
Zheng Ning, Zheng Zhang, Jerrick Ban, Kaiwen Jiang, Ruohong Gan, Yapeng Tian, Toby Jia-Jun Li
2024 — conf
MOCHA: Model Optimization through Collaborative Human-AI Alignment.
UIST (Adjunct Volume)
Simret Araya Gebreegziabher, Elena L. Glassman, Toby Jia-Jun Li
2024 J jnl
Programmer Visual Attention During Context-Aware Code Summarization.
CoRR
Aakash Bansal, Robert Wallace, Zachary Karas, Ningzhi Tang, Yu Huang, Toby Jia-Jun Li, Collin McMillan
2024 A* conf
SPICA: Interactive Video Content Exploration through Augmented Audio Descriptions for Blind or Low-Vision Viewers.
CHI
Zheng Ning, Brianna L. Wimer, Kaiwen Jiang, Keyi Chen, Jerrick Ban, Yapeng Tian, Yuhang Zhao, Toby Jia-Jun Li
2024 J jnl
SPICA: Interactive Video Content Exploration through Augmented Audio Descriptions for Blind or Low-Vision Viewers.
CoRR
Zheng Ning, Brianna L. Wimer, Kaiwen Jiang, Keyi Chen, Jerrick Ban, Yapeng Tian, Yuhang Zhao, Toby Jia-Jun Li
2024 A* conf
SQLucid: Grounding Natural Language Database Queries with Interactive Explanations.
UIST
Yuan Tian, Jonathan K. Kummerfeld, Toby Jia-Jun Li, Tianyi Zhang
2024 J jnl
SQLucid: Grounding Natural Language Database Queries with Interactive Explanations.
CoRR
Yuan Tian, Jonathan K. Kummerfeld, Toby Jia-Jun Li, Tianyi Zhang
2024 J jnl
Sketchar: Supporting Character Design and Illustration Prototyping Using Generative AI.
Proc. ACM Hum. Comput. Interact.
Long Ling, Xinyi Chen, Ruoyu Wen, Toby Jia-Jun Li, Ray LC
2024 J jnl
Supporting Co-Adaptive Machine Teaching through Human Concept Learning and Cognitive Theories.
CoRR
Simret Araya Gebreegziabher, Yukun Yang, Elena L. Glassman, Toby Jia-Jun Li
2024 J jnl
Why am I seeing this: Democratizing End User Auditing for Online Content Recommendations.
CoRR
Chaoran Chen, Leyang Li, Luke Cao, Yanfang Ye, Tianshi Li, Yaxing Yao, Toby Jia-Jun Li
2023 J jnl
An Empathy-Based Sandbox Approach to Bridge Attitudes, Goals, Knowledge, and Behaviors in the Privacy Paradox.
CoRR
Chaoran Chen, Weijun Li, Wenxin Song, Yanfang Ye, Yaxing Yao, Toby Jia-Jun Li
2023 A conf
An Empirical Study of Model Errors and User Error Discovery and Repair Strategies in Natural Language Database Queries.
IUI
Zheng Ning, Zheng Zhang, Tianyi Sun, Yuan Tian, Tianyi Zhang, Toby Jia-Jun Li
2023 — conf
CollabCoder: A GPT-Powered WorkFlow for Collaborative Qualitative Analysis.
CSCW Companion
Jie Gao, Yuchen Guo, Toby Jia-Jun Li, Simon Tangi Perrault
2023 J jnl
CollabCoder: A GPT-Powered Workflow for Collaborative Qualitative Analysis.
CoRR
Jie Gao, Yuchen Guo, Gionnieve Lim, Tianqin Zhang, Zheng Zhang, Toby Jia-Jun Li, Simon Tangi Perrault
2023 — conf
Designing for AI-Powered Social Computing Systems.
CSCW Companion
Gionnieve Lim, Hyunwoo Kim, Yoonseo Choi, Toby Jia-Jun Li, Chinmay Kulkarni, Hariharan Subramonyam, Joseph Seering, Michael S. Bernstein, Amy X. Zhang, Elena L. Glassman, Simon T. Perrault, Juho Kim
2023 J jnl
Empowering LLM to use Smartphone for Intelligent Task Automation.
CoRR
Hao Wen, Yuanchun Li, Guohong Liu, Shanhui Zhao, Tao Yu, Toby Jia-Jun Li, Shiqi Jiang, Yunhao Liu, Yaqin Zhang, Yunxin Liu
2023 J jnl
From Awareness to Action: Exploring End-User Empowerment Interventions for Dark Patterns in UX.
CoRR
Yuwen Lu, Chao Zhang, Yuewen Yang, Yaxing Yao, Toby Jia-Jun Li
2023 J jnl
Human Still Wins over LLM: An Empirical Study of Active Learning on Domain-Specific Annotation Tasks.
CoRR
Yuxuan Lu, Bingsheng Yao, Shao Zhang, Yun Wang, Peng Zhang, Tun Lu, Toby Jia-Jun Li, Dakuo Wang
2023 J jnl
Impact of Human-AI Interaction on User Trust and Reliance in AI-Assisted Qualitative Coding.
CoRR
Jie Gao, Junming Cao, ShunYi Yeo, Kenny Tsu Wei Choo, Zheng Zhang, Toby Jia-Jun Li, Shengdong Zhao, Simon Tangi Perrault
2023 A* conf
Interactive Text-to-SQL Generation via Editable Step-by-Step Explanations.
EMNLP
Yuan Tian, Zheng Zhang, Zheng Ning, Toby Jia-Jun Li, Jonathan K. Kummerfeld, Tianyi Zhang
2023 J jnl
Interactive Text-to-SQL Generation via Editable Step-by-Step Explanations.
CoRR
Yuan Tian, Toby Jia-Jun Li, Jonathan K. Kummerfeld, Tianyi Zhang
2023 A* conf
Modeling Programmer Attention as Scanpath Prediction.
ASE
Aakash Bansal, Chia-Yi Su, Zachary Karas, Yifan Zhang, Yu Huang, Toby Jia-Jun Li, Collin McMillan
2023 J jnl
Modeling Programmer Attention as Scanpath Prediction.
CoRR
Aakash Bansal, Chia-Yi Su, Zachary Karas, Yifan Zhang, Yu Huang, Toby Jia-Jun Li, Collin McMillan
2023 A* conf
PEANUT: A Human-AI Collaborative Tool for Annotating Audio-Visual Data.
UIST
Zheng Zhang, Zheng Ning, Chenliang Xu, Yapeng Tian, Toby Jia-Jun Li
2023 J jnl
PEANUT: A Human-AI Collaborative Tool for Annotating Audio-Visual Data.
CoRR
Zheng Zhang, Zheng Ning, Chenliang Xu, Yapeng Tian, Toby Jia-Jun Li
2023 A* conf
PaTAT: Human-AI Collaborative Qualitative Coding with Explainable Interactive Rule Synthesis.
CHI
Simret Araya Gebreegziabher, Zheng Zhang, Xiaohang Tang, Yihao Meng, Elena L. Glassman, Toby Jia-Jun Li
2023 — conf
SHAI 2023: Workshop on Designing for Safety in Human-AI Interactions.
IUI Companion
Nitesh Goyal, Sungsoo Ray Hong, Regan L. Mandryk, Toby Jia-Jun Li, Kurt Luther, Dakuo Wang
2023 — conf
Shaping the Emerging Norms of Using Large Language Models in Social Computing Research.
CSCW Companion
Hong Shen, Tianshi Li, Toby Jia-Jun Li, Joon Sung Park, Diyi Yang
2023 J jnl
Shaping the Emerging Norms of Using Large Language Models in Social Computing Research.
CoRR
Hong Shen, Tianshi Li, Toby Jia-Jun Li, Joon Sung Park, Diyi Yang
2023 J jnl
Structured Generation and Exploration of Design Space with Large Language Models for Human-AI Co-Creation.
CoRR
Sangho Suh, Meng Chen, Bryan Min, Toby Jia-Jun Li, Haijun Xia
2023 — conf
The Future of Computational Approaches for Understanding and Adapting User Interfaces.
CHI Extended Abstracts
Yue Jiang, Yuwen Lu, Christof Lutteroth, Toby Jia-Jun Li, Jeffrey Nichols, Wolfgang Stuerzlinger
2023 J jnl
UI Layout Generation with LLMs Guided by UI Grammar.
CoRR
Yuwen Lu, Ziang Tong, Qinyi Zhao, Chengzhi Zhang, Toby Jia-Jun Li
2023 A* conf
VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping.
UIST
Zheng Zhang, Jie Gao, Ranjodh Singh Dhaliwal, Toby Jia-Jun Li
2023 J jnl
VISAR: A Human-AI Argumentative Writing Assistant with Visual Programming and Rapid Draft Prototyping.
CoRR
Zheng Zhang, Jie Gao, Ranjodh Singh Dhaliwal, Toby Jia-Jun Li
2022 — conf
A Bottom-Up End-User Intelligent Assistant Approach to Empower Gig Workers against AI Inequality.
CHIWORK
Toby Jia-Jun Li, Yuwen Lu, Jaylexia Clark, Meng Chen, Victor V. Cox, Meng Jiang, Yang Yang, Tamara Kay, Danielle M. Wood, Jay Brockman
2022 J jnl
A Bottom-Up End-User Intelligent Assistant Approach to Empower Gig Workers against AI Inequality.
CoRR
Toby Jia-Jun Li, Yuwen Lu, Jaylexia Clark, Meng Chen, Victor V. Cox, Meng Jiang, Yang Yang, Tamara Kay, Danielle M. Wood, Jay Brockman
2022 — conf
AI as an Active Writer: Interaction Strategies with Generated Text in Human-AI Collaborative Fiction Writing 56-65.
IUI Workshops
Daijin Yang, Yanpeng Zhou, Zhiyuan Zhang, Toby Jia-Jun Li, Ray LC
2022 — conf
Bridging the Gap Between UX Practitioners' Work Practices and AI-Enabled Design Support Tools.
CHI Extended Abstracts
Yuwen Lu, Chengzhi Zhang, Iris Zhang, Toby Jia-Jun Li
2022 — conf
Computational Approaches for Understanding, Generating, and Adapting User Interfaces.
CHI Extended Abstracts
Yue Jiang, Yuwen Lu, Jeffrey Nichols, Wolfgang Stuerzlinger, Chun Yu, Christof Lutteroth, Yang Li, Ranjitha Kumar, Toby Jia-Jun Li
2022 — conf
Fantastic Questions and Where to Find Them: FairytaleQA - An Authentic Dataset for Narrative Comprehension.
ACL (1)
Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Jia-Jun Li, Nora Bradford, Branda Sun, Tran Bao Hoang, Yisi Sang, Yufang Hou, Xiaojuan Ma, Diyi Yang, Nanyun Peng, Zhou Yu, Mark Warschauer
2022 J jnl
Fantastic Questions and Where to Find Them: FairytaleQA - An Authentic Dataset for Narrative Comprehension.
CoRR
Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Jia-Jun Li, Nora Bradford, Branda Sun, Tran Bao Hoang, Yisi Sang, Yufang Hou, Xiaojuan Ma, Diyi Yang, Nanyun Peng, Zhou Yu, Mark Warschauer
2022 — conf
It is AI's Turn to Ask Humans a Question: Question-Answer Pair Generation for Children's Story Books.
ACL (1)
Bingsheng Yao, Dakuo Wang, Tongshuang Wu, Zheng Zhang, Toby Jia-Jun Li, Mo Yu, Ying Xu
2022 J jnl
KnowledgeShovel: An AI-in-the-Loop Document Annotation System for Scientific Knowledge Base Construction.
CoRR
Shao Zhang, Yuting Jia, Hui Xu, Dakuo Wang, Toby Jia-Jun Li, Ying Wen, Xinbing Wang, Chenghu Zhou
2022 A* conf
StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvement.
CHI
Zheng Zhang, Ying Xu, Yanhao Wang, Bingsheng Yao, Daniel Ritchie, Tongshuang Wu, Mo Yu, Dakuo Wang, Toby Jia-Jun Li
2022 J jnl
StoryBuddy: A Human-AI Collaborative Chatbot for Parent-Child Interactive Storytelling with Flexible Parental Involvement.
CoRR
Zheng Zhang, Ying Xu, Yanhao Wang, Bingsheng Yao, Daniel Ritchie, Tongshuang Wu, Mo Yu, Dakuo Wang, Toby Jia-Jun Li
2021 J jnl
A Need-finding Study for Understanding Text Entry in SmartphoneApp Usage.
CoRR
Toby Jia-Jun Li, Brad A. Myers
2021 J jnl
It is AI's Turn to Ask Human a Question: Question and Answer Pair Generation for Children Storybooks in FairytaleQA Dataset.
CoRR
Bingsheng Yao, Dakuo Wang, Tongshuang Wu, Tran Bao Hoang, Branda Sun, Toby Jia-Jun Li, Mo Yu, Ying Xu
2021 A* conf
Screen2Vec: Semantic Embedding of GUI Screens and GUI Components.
CHI
Toby Jia-Jun Li, Lindsay Popowski, Tom M. Mitchell, Brad A. Myers
2021 J jnl
Screen2Vec: Semantic Embedding of GUI Screens and GUI Components.
CoRR
Toby Jia-Jun Li, Lindsay Popowski, Tom M. Mitchell, Brad A. Myers
2020 A* conf
Geno: A Developer Tool for Authoring Multimodal Interaction on Existing Web Applications.
UIST
Ritam Jyoti Sarmah, Yunpeng Ding, Di Wang, Cheuk Yin Phipson Lee, Toby Jia-Jun Li, Xiang 'Anthony' Chen
2020 J jnl
Geno: A Developer Tool for Authoring Multimodal Interaction on Existing Web Applications.
CoRR
Ritam Jyoti Sarmah, Yunpeng Ding, Di Wang, Cheuk Yin Phipson Lee, Toby Jia-Jun Li, Xiang 'Anthony' Chen
2020 — conf
Interactive Task Learning from GUI-Grounded Natural Language Instructions and Demonstrations.
ACL (demo)
Toby Jia-Jun Li, Tom M. Mitchell, Brad A. Myers
2020 — conf
Multi-Modal Interactive Task Learning from Demonstrations and Natural Language Instructions.
UIST (Adjunct Volume)
Toby Jia-Jun Li
2020 A* conf
Multi-Modal Repairs of Conversational Breakdowns in Task-Oriented Dialogs.
UIST
Toby Jia-Jun Li, Jingya Chen, Haijun Xia, Tom M. Mitchell, Brad A. Myers
2020 J jnl
Privacy-Preserving Script Sharing in GUI-based Programming-by-Demonstration Systems.
CoRR
Toby Jia-Jun Li, Jingya Chen, Brandon Canfield, Brad A. Myers
2020 J jnl
Privacy-Preserving Script Sharing in GUI-based Programming-by-Demonstration Systems.
Proc. ACM Hum. Comput. Interact.
Toby Jia-Jun Li, Jingya Chen, Brandon Canfield, Brad A. Myers
2020 J jnl
Towards Effective Human-AI Collaboration in GUI-Based Interactive Task Learning Agents.
CoRR
Toby Jia-Jun Li, Jingya Chen, Tom M. Mitchell, Brad A. Myers
2019 — conf
End user programing of intelligent agents using demonstrations and natural language instructions.
IUI Companion
Toby Jia-Jun Li
2019 J jnl
Not at Home on the Range: Peer Production and the Urban/Rural Divide.
CoRR
Isaac L. Johnson, Allen Yilun Lin, Toby Jia-Jun Li, Andrew Hall, Aaron Halfaker, Johannes Schöning, Brent J. Hecht
2019 A* conf
PUMICE: A Multi-Modal Agent that Learns Concepts and Conditionals from Natural Language and Demonstrations.
UIST
Toby Jia-Jun Li, Marissa Radensky, Justin Jia, Kirielle Singarajah, Tom M. Mitchell, Brad A. Myers
2019 J jnl
PUMICE: A Multi-Modal Agent that Learns Concepts and Conditionals from Natural Language and Demonstrations.
CoRR
Toby Jia-Jun Li, Marissa Radensky, Justin Jia, Kirielle Singarajah, Tom M. Mitchell, Brad A. Myers
2018 B conf
APPINITE: A Multi-Modal Interface for Specifying Data Descriptions in Programming by Demonstration Using Natural Language Instructions.
VL/HCC
Toby Jia-Jun Li, Igor Labutov, Xiaohan Nancy Li, Xiaoyi Zhang, Wenze Shi, Wanling Ding, Tom M. Mitchell, Brad A. Myers
2018 B conf
How End Users Express Conditionals in Programming by Demonstration for Mobile Apps.
VL/HCC
Marissa Radensky, Toby Jia-Jun Li, Brad A. Myers
2018 A conf
Kite: Building Conversational Bots from Mobile Apps.
MobiSys
Toby Jia-Jun Li, Oriana Riva
2018 — ch.
Teaching Agents When They Fail: End User Development in Goal-Oriented Conversational Agents.
Studies in Conversational UX Design
Toby Jia-Jun Li, Igor Labutov, Brad A. Myers, Amos Azaria, Alexander I. Rudnicky, Tom M. Mitchell
2017 B conf
End user mobile task automation using multimodal programming by demonstration.
VL/HCC
Toby Jia-Jun Li
2017 — ch.
Making End User Development More Natural.
New Perspectives in End-User Development
Brad A. Myers, Amy J. Ko, Chris Scaffidi, Stephen Oney, YoungSeok Yoon, Kerry Shih-Ping Chang, Mary Beth Kery, Toby Jia-Jun Li
2017 J jnl
PrivacyStreams: Enabling Transparency in Personal Data Processing for Mobile Apps.
Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.
Yuanchun Li, Fanglin Chen, Toby Jia-Jun Li, Yao Guo, Gang Huang, Matthew Fredrikson, Yuvraj Agarwal, Jason I. Hong
2017 — conf
Programming IoT Devices by Demonstration Using Mobile Apps.
IS-EUD
Toby Jia-Jun Li, Yuanchun Li, Fanglin Chen, Brad A. Myers
2017 A* conf
SUGILITE: Creating Multimodal Smartphone Automation by Demonstration.
CHI
Toby Jia-Jun Li, Amos Azaria, Brad A. Myers
2016 A* conf
Not at Home on the Range: Peer Production and the Urban/Rural Divide.
CHI
Isaac L. Johnson, Yilun Lin, Toby Jia-Jun Li, Andrew Hall, Aaron Halfaker, Johannes Schöning, Brent J. Hecht
2014 — conf
Leveraging advances in natural language processing to better understand Tobler's first law of geography.
SIGSPATIAL/GIS
Toby Jia-Jun Li, Shilad Sen, Brent J. Hecht
2014 C conf
WikiBrain: Democratizing computation on Wikipedia.
OpenSym
Shilad Sen, Toby Jia-Jun Li, WikiBrain Team, Brent J. Hecht