Chris Welty

70 papers A* 2B 5Journal 27Unranked 27
YearRankTypeTitle / Venue / Authors
2026 A* conf
Forest vs Tree: The (N, K) Trade-off in Reproducible ML Evaluation.
AAAI
Deepak Pandita, Flip Korn, Chris Welty, Christopher M. Homan
2026 — conf
How Many Ratings per Item are Necessary for Reliable Significance Testing?
EACL (Findings)
Christopher M. Homan, Flip Korn, Deepak Pandita, Chris Welty
2025 J jnl
Forest vs Tree: The (N, K) Trade-off in Reproducible ML Evaluation.
CoRR
Deepak Pandita, Flip Korn, Chris Welty, Christopher M. Homan
2024 J jnl
How Many Ratings per Item are Necessary for Reliable Significance Testing?
CoRR
Christopher Homan, Flip Korn, Chris Welty
2024 J jnl
Introducing v0.5 of the AI Safety Benchmark from MLCommons.
CoRR
Bertie Vidgen, Adarsh Agrawal, Ahmed M. Ahmed, Victor Akinwande, Namir Al-Nuaimi, Najla Alfaraj, Elie Alhajjar, Lora Aroyo, Trupti Bavalatti, Borhane Blili-Hamelin, Kurt Bollacker, Rishi Bomassani, Marisa Ferrara Boston, Siméon Campos, Kal Chakra, Canyu Chen, Cody Coleman, Zacharie Delpierre Coudert, Leon Derczynski, Debojyoti Dutta, Ian Eisenberg, James Ezick, Heather Frase, Brian Fuller, Ram Gandikota, Agasthya Gangavarapu, Ananya Gangavarapu, James Gealy, Rajat Ghosh, James Goel, Usman Gohar, Subhra S. Goswami, Scott A. Hale, Wiebke Hutiri, Joseph Marvin Imperial, Surgan Jandial, Nick Judd, Felix Juefei-Xu, Foutse Khomh, Bhavya Kailkhura, Hannah Rose Kirk, Kevin Klyman, Chris Knotz, Michael Kuchnik, Shachi H. Kumar, Chris Lengerich, Bo Li, Zeyi Liao, Eileen Peters Long, Victor Lu, Yifan Mai, Priyanka Mary Mammen, Kelvin N. Manyeki, Sean McGregor, Virendra Mehta, Shafee Mohammed, Emanuel Moss, Lama Nachman, Dinesh Jinenhally Naganna, Amin Nikanjam, Besmira Nushi, Luis Oala, Iftach Orr, Alicia Parrish, Cigdem Patlak, William Pietri, Forough Poursabzi-Sangdeh, Eleonora Presani, Fabrizio Puletti, Paul Röttger, Saurav Sahay, Tim Santos, Nino Scherrer, Alice Schoenauer Sebag, Patrick Schramowski, Abolfazl Shahbazi, Vin Sharma, Xudong Shen, Vamsi Sistla, Leonard Tang, Davide Testuggine, Vithursan Thangarasa, Elizabeth Anne Watkins, Rebecca Weiss, Chris Welty, Tyler Wilbers, Adina Williams, Carole-Jean Wu, Poonam Yadav, Xianjun Yang, Yi Zeng, Wenhui Zhang, Fedor Zhdanov, Jiacheng Zhu, Percy Liang, Peter Mattson, Joaquin Vanschoren
2023 — conf
Follow the leader(board) with confidence: Estimating p-values from a single test set with item and response variance.
ACL (Findings)
Shira Wein, Christopher Homan, Lora Aroyo, Chris Welty
2022 J jnl
Addressing Label Sparsity With Class-Level Common Sense for Google Maps.
Frontiers Artif. Intell.
Chris Welty, Lora Aroyo, Flip Korn, Sara M. McCarthy, Shubin Zhao
2022 — conf
Annotator Response Distributions as a Sampling Frame.
NLPerspectives@LREC
Christopher Homan, Tharindu Cyril Weerasooriya, Lora Aroyo, Chris Welty
2022 A* conf
Vexation-Aware Active Learning for On-Menu Restaurant Dish Availability.
KDD
Jean-François Kagy, Flip Korn, Afshin Rostamizadeh, Chris Welty
2021 J jnl
Betting on Bets.
AI Mag.
Chris Welty, Praveen K. Paritosh, Kurt Bollacker
2021 J jnl
Empirical methodology for crowdsourcing ground truth.
Semantic Web
Anca Dumitrache, Oana Inel, Benjamin Timmermans, Carlos Martinez-Ortiz, Robert-Jan Sips, Lora Aroyo, Chris Welty
2021 B conf
Rapid Instance-Level Knowledge Acquisition for Google Maps from Class-Level Common Sense.
HCOMP
Chris Welty, Lora Aroyo, Flip Korn, Sara M. McCarthy, Shubin Zhao
2020 B conf
Embedding Semantic Taxonomies.
COLING
Alyssa Lees, Chris Welty, Shubin Zhao, Jacek Korycki, Sara Mc Carthy
2020 J jnl
The AI Bookie - Place Your Bets: Will Machine Learning Outgrow Human Labeling?
AI Mag.
Mike Schaekermann, Christopher Homan, Lora Aroyo, Praveen K. Paritosh, Kurt Bollacker, Chris Welty
2019 — conf
A Crowdsourced Frame Disambiguation Corpus with Ambiguity.
NAACL-HLT (1)
Anca Dumitrache, Lora Aroyo, Chris Welty
2019 J jnl
A Crowdsourced Frame Disambiguation Corpus with Ambiguity.
CoRR
Anca Dumitrache, Lora Aroyo, Chris Welty
2019 — conf
Crowdsourcing Inclusivity: Dealing with Diversity of Opinions, Perspectives and Ambiguity in Annotated Data.
WWW (Companion Volume)
Lora Aroyo, Anca Dumitrache, Oana Inel, Zoltán Szlávik, Benjamin Timmermans, Chris Welty
2019 — conf
Discovering User Bias in Ordinal Voting Systems.
WWW (Companion Volume)
Alyssa Lees, Chris Welty
2019 J jnl
Metrology for AI: From Benchmarks to Instruments.
CoRR
Chris Welty, Praveen K. Paritosh, Lora Aroyo
2019 — conf
Taxonomy Embeddings on PubMed Article Subject Headings.
SEPDA@ISWC
Alyssa Lees, Jacek Korycki, Chris Welty, Shubin Zhao
2019 J jnl
The AI Bookie The Future of Voice Interfaces to Intelligent Assistants.
AI Mag.
Chris Welty, Lora Aroyo, Eric Horvitz
2019 J jnl
What is Fair? Exploring Pareto-Efficiency for Fairness Constrained Classifiers.
CoRR
Ananth Balashankar, Alyssa Lees, Chris Welty, Lakshminarayanan Subramanian
2018 — conf
Augmenting Intelligence with Humans-in-the-Loop (HumL@WWW2018) Chairs' Welcome & Organization.
WWW (Companion Volume)
Lora Aroyo, Gianluca Demartini, Anna Lisa Gentile, Chris Welty
2018 B conf
Capturing Ambiguity in Crowdsourcing Frame Disambiguation.
HCOMP
Anca Dumitrache, Lora Aroyo, Chris Welty
2018 J jnl
Capturing Ambiguity in Crowdsourcing Frame Disambiguation.
CoRR
Anca Dumitrache, Lora Aroyo, Chris Welty
2018 — conf
CrowdTruth 2.0: Quality Metrics for Crowdsourcing with Disagreement (short paper).
SAD/CrowdBias@HCOMP
Anca Dumitrache, Oana Inel, Lora Aroyo, Benjamin Timmermans, Chris Welty
2018 J jnl
CrowdTruth 2.0: Quality Metrics for Crowdsourcing with Disagreement.
CoRR
Anca Dumitrache, Oana Inel, Lora Aroyo, Benjamin Timmermans, Chris Welty
2018 J jnl
Crowdsourcing Ground Truth for Medical Relation Extraction.
ACM Trans. Interact. Intell. Syst.
Anca Dumitrache, Lora Aroyo, Chris Welty
2018 — conf
Crowdsourcing Semantic Label Propagation in Relation Classification.
FEVER@EMNLP
Anca Dumitrache, Lora Aroyo, Chris Welty
2018 J jnl
Crowdsourcing Semantic Label Propagation in Relation Classification.
CoRR
Anca Dumitrache, Lora Aroyo, Chris Welty
2018 J jnl
Empirical Methodology for Crowdsourcing Ground Truth.
CoRR
Anca Dumitrache, Oana Inel, Benjamin Timmermans, Carlos Martinez-Ortiz, Robert-Jan Sips, Lora Aroyo, Chris Welty
2018 — ed.
Proceedings of the 1st Workshop on Subjectivity, Ambiguity and Disagreement in Crowdsourcing, and Short Paper Proceedings of the 1st Workshop on Disentangling the Relation Between Crowdsourcing and Bias Management (SAD 2018 and CrowdBias 2018) co-located the 6th AAAI Conference on Human Computation and Crowdsourcing (HCOMP 2018), Zürich, Switzerland, July 5, 2018.
SAD/CrowdBias@HCOMP
Lora Aroyo, Anca Dumitrache, Praveen K. Paritosh, Alexander J. Quinn, Chris Welty, Alessandro Checco, Gianluca Demartini, Ujwal Gadiraju, Cristina Sarasua
2018 — ed.
Proceedings of the 2nd International Workshop on Augmenting Intelligence with Humans­-in-­the-­Loop co-located with 17th International Semantic Web Conference (ISWC 2018), Monterey, California, October 9th, 2018.
HumL@ISWC
Anna Lisa Gentile, Lora Aroyo, Gianluca Demartini, Chris Welty
2018 J jnl
The AI Bookie Place Your Bets: Adversarial Collaboration for Scientific Advancement.
AI Mag.
Kurt Bollacker, Praveen K. Paritosh, Chris Welty
2018 — conf
The Quantum Collective.
WWW (Companion Volume)
Lora Aroyo, Chris Welty
2017 J jnl
Crowdsourcing Ground Truth for Medical Relation Extraction.
CoRR
Anca Dumitrache, Lora Aroyo, Chris Welty
2017 — conf
False Positive and Cross-relation Signals in Distant Supervision Data.
AKBC@NIPS
Anca Dumitrache, Lora Aroyo, Chris Welty
2017 J jnl
False Positive and Cross-relation Signals in Distant Supervision Data.
CoRR
Anca Dumitrache, Lora Aroyo, Chris Welty
2015 — conf
Achieving Expert-Level Annotation Quality with CrowdTruth: The Case of Medical Relation Extraction.
BDM2I@ISWC
Anca Dumitrache, Lora Aroyo, Chris Welty
2015 — conf
CrowdTruth Measures for Language Ambiguity: The Case of Medical Relation Extraction.
LD4IE@ISWC
Anca Dumitrache, Lora Aroyo, Chris Welty
2015 — conf
Crowdsourcing ground truth for Question Answering using CrowdTruth.
WebSci
Benjamin Timmermans, Lora Aroyo, Chris Welty
2015 — conf
Medical Concept Resolution.
ISWC (Posters & Demos)
Nitish Aggarwal, Ken Barker, Chris Welty
2015 J jnl
Truth Is a Lie: Crowd Truth and the Seven Myths of Human Annotation.
AI Mag.
Lora Aroyo, Chris Welty
2014 — ed.
The Semantic Web - ISWC 2014 - 13th International Semantic Web Conference, Riva del Garda, Italy, October 19-23, 2014. Proceedings, Part I
ISWC (1)
Peter Mika, Tania Tudorache, Abraham Bernstein, Chris Welty, Craig A. Knoblock, Denny Vrandecic, Paul Groth, Natasha F. Noy, Krzysztof Janowicz, Carole A. Goble
2014 — ed.
The Semantic Web - ISWC 2014 - 13th International Semantic Web Conference, Riva del Garda, Italy, October 19-23, 2014. Proceedings, Part II
ISWC (2)
Peter Mika, Tania Tudorache, Abraham Bernstein, Chris Welty, Craig A. Knoblock, Denny Vrandecic, Paul Groth, Natasha F. Noy, Krzysztof Janowicz, Carole A. Goble
2014 J jnl
The Three Sides of CrowdTruth.
Hum. Comput.
Lora Aroyo, Chris Welty
2013 — conf
"Dr. Detective": combining gamification techniques and crowdsourcing to create a gold standard in medical text.
CrowdSem
Anca Dumitrache, Lora Aroyo, Chris Welty, Robert-Jan Sips, Anthony Levas
2013 — conf
Content and Behaviour Based Metrics for Crowd Truth.
CrowdSem
Guillermo Soberon, Lora Aroyo, Chris Welty, Oana Inel, Manfred Overmeen, Hui Lin
2013 — conf
Domain-Independent Quality Measures for Crowd Truth Disagreement.
DeRiVE@ISWC
Oana Inel, Lora Aroyo, Chris Welty, Robert-Jan Sips
2013 B conf
Long-Distance Time-Event Relation Extraction.
IJCNLP
Alessandro Moschitti, Siddharth Patwardhan, Chris Welty
2013 — conf
Measuring Crowd Truth for Medical Relation Extraction.
AAAI Fall Symposia
Lora Aroyo, Chris Welty
2013 — conf
Semantic Web and Best Practice in Watson.
WaSABi@ISWC
Chris Welty
2013 — ed.
The Semantic Web - ISWC 2013 - 12th International Semantic Web Conference, Sydney, NSW, Australia, October 21-25, 2013, Proceedings, Part I
ISWC (1)
Harith Alani, Lalana Kagal, Achille Fokoue, Paul Groth, Chris Biemann, Josiane Xavier Parreira, Lora Aroyo, Natasha F. Noy, Chris Welty, Krzysztof Janowicz
2013 — ed.
The Semantic Web - ISWC 2013 - 12th International Semantic Web Conference, Sydney, NSW, Australia, October 21-25, 2013, Proceedings, Part II
ISWC (2)
Harith Alani, Lalana Kagal, Achille Fokoue, Paul Groth, Chris Biemann, Josiane Xavier Parreira, Lora Aroyo, Natasha F. Noy, Chris Welty, Krzysztof Janowicz
2012 — conf
A Comparison of Hard Filters and Soft Evidence for Answer Typing in Watson.
ISWC (2)
Chris Welty, J. William Murdock, Aditya Kalyanpur, James Fan
2012 J jnl
A framework for merging and ranking of answers in DeepQA.
IBM J. Res. Dev.
David Gondek, Adam Lally, Aditya Kalyanpur, J. William Murdock, Pablo Ariel Duboue, Lei Zhang, Yue Pan, Zhaoming Qiu, Chris Welty
2012 J jnl
Finding needles in the haystack: Search and candidate generation.
IBM J. Res. Dev.
Jennifer Chu-Carroll, James Fan, Branimir Boguraev, David Carmel, Dafna Sheinwald, Chris Welty
2012 — conf
Harnessing Disagreement for Event Semantics.
DeRiVE@ISWC
Chris Welty, Lora Aroyo
2012 — conf
Query Driven Hypothesis Generation for Answering Queries over NLP Graphs.
ISWC (2)
Chris Welty, Ken Barker, Lora Aroyo, Shilpa Arora
2012 J jnl
Structured data and inference in DeepQA.
IBM J. Res. Dev.
Aditya Kalyanpur, Branimir Boguraev, Siddharth Patwardhan, J. William Murdock, Adam Lally, Chris Welty, John M. Prager, Bonaventura Coppola, Achille Fokoue-Nkoutche, Lei Zhang, Yue Pan, Zhaoming Qiu
2012 J jnl
Typing candidate answers using type coercion.
IBM J. Res. Dev.
J. William Murdock, Aditya Kalyanpur, Chris Welty, James Fan, David A. Ferrucci, David Gondek, Lei Zhang, Hiroshi Kanayama
2011 — ed.
The Semantic Web - ISWC 2011 - 10th International Semantic Web Conference, Bonn, Germany, October 23-27, 2011, Proceedings, Part I
ISWC (1)
Lora Aroyo, Chris Welty, Harith Alani, Jamie Taylor, Abraham Bernstein, Lalana Kagal, Natasha Fridman Noy, Eva Blomqvist
2011 — ed.
The Semantic Web - ISWC 2011 - 10th International Semantic Web Conference, Bonn, Germany, October 23-27, 2011, Proceedings, Part II
ISWC (2)
Lora Aroyo, Chris Welty, Harith Alani, Jamie Taylor, Abraham Bernstein, Lalana Kagal, Natasha Fridman Noy, Eva Blomqvist
2010 — conf
Archeology for Ontology Patterns.
WOP
Chris Welty
2010 — conf
Context Slices: Representing Contexts in OWL.
WOP
Chris Welty
2010 J jnl
Introduction to the Special Issue on Question Answering.
AI Mag.
David Gunning, Vinay K. Chaudhri, Chris Welty
2010 B conf
Learning to Predict Readability using Diverse Linguistic Features.
COLING
Rohit J. Kate, Xiaoqiang Luo, Siddharth Patwardhan, Martin Franz, Radu Florian, Raymond J. Mooney, Salim Roukos, Chris Welty
2007 — conf
COALA: A Tool for Inter-document Coreference Resolution Evaluation.
AAAI Spring Symposium: Machine Reading
Bradley M. Andrews, James Fan, J. William Murdock, Chris Welty
2006 — ch.
The Object Management Group Ontology Definition Metamodel.
Ontologies for Software Engineering and Software Technology
Robert M. Colomb, Kerry Raymond, Lewis Hart, Patrick Emery, Chris Welty, Guo Tong Xie, Elisa F. Kendall
1994 J jnl
The Eighth Annual Knowledge-Based Software Engineering Conference.
Autom. Softw. Eng.
Chris Welty