Dmitry Ustalov

75 papers A* 2A 5B 4C 2Misc 4Journal 28Unranked 25
YearRankTypeTitle / Venue / Authors
2025 — conf
Challenge on Optimization of Context Collection for Code Completion.
ASEW
Dmitry Ustalov, Egor Bogomolov, Alexander Bezzubov, Yaroslav Golubev, Evgeniy Glukhov, Georgii Levtsov, Vladimir Kovalenko
2025 J jnl
Challenge on Optimization of Context Collection for Code Completion.
CoRR
Dmitry Ustalov, Egor Bogomolov, Alexander Bezzubov, Yaroslav Golubev, Evgeniy Glukhov, Georgii Levtsov, Vladimir Kovalenko
2025 — conf
Confidence and Stability of Global and Pairwise Scores in NLP Evaluation.
ACL (4)
Georgii Levtsov, Dmitry Ustalov
2025 J jnl
Confidence and Stability of Global and Pairwise Scores in NLP Evaluation.
CoRR
Georgii Levtsov, Dmitry Ustalov
2025 J jnl
Mellum: Production-Grade in-IDE Contextual Code Completion with Multi-File Project Understanding.
CoRR
Nikita Pavlichenko, Iurii Nazarov, Ivan Dolgov, Ekaterina Garanina, Dmitry Ustalov, Ivan Bondyrev, Kseniia Lysaniuk, Evgeniia Vu, Kirill Chekmenev, Joseph Shtok, Yaroslav Golubev, Anton Semenkin, Uladzislau Sazanovich
2025 — conf
Reliable, Reproducible, and Really Fast Leaderboards with Evalica.
COLING (Demonstrations)
Dmitry Ustalov
2024 J jnl
Learning from Crowds with Crowd-Kit.
J. Open Source Softw.
Dmitry Ustalov, Nikita Pavlichenko, Boris Tseitlin
2024 — conf
Overview of PAN 2024: Multi-author Writing Style Analysis, Multilingual Text Detoxification, Oppositional Thinking Analysis, and Generative AI Authorship Verification - Extended Abstract.
ECIR (6)
Janek Bevendorff, Xavier Bonet Casals, Berta Chulvi, Daryna Dementieva, Ashraf Elnagar, Dayne Freitag, Maik Fröbe, Damir Korencic, Maximilian Mayerl, Animesh Mukherjee, Alexander Panchenko, Martin Potthast, Francisco Rangel, Paolo Rosso, Alisa Smirnova, Efstathios Stamatatos, Benno Stein, Mariona Taulé, Dmitry Ustalov, Matti Wiegmann, Eva Zangerle
2024 — conf
Overview of PAN 2024: Multi-author Writing Style Analysis, Multilingual Text Detoxification, Oppositional Thinking Analysis, and Generative AI Authorship Verification Condensed Lab Overview.
CLEF (2)
Abinew Ali Ayele, Nikolay Babakov, Janek Bevendorff, Xavier Bonet Casals, Berta Chulvi, Daryna Dementieva, Ashaf Elnagar, Dayne Freitag, Maik Fröbe, Damir Korencic, Maximilian Mayerl, Daniil Moskovskiy, Animesh Mukherjee, Alexander Panchenko, Martin Potthast, Francisco Rangel, Naquee Rizwan, Paolo Rosso, Florian Schneider, Alisa Smirnova, Efstathios Stamatatos, Elisei Stakovskii, Benno Stein, Mariona Taulé, Dmitry Ustalov, Xintong Wang, Matti Wiegmann, Seid Muhie Yimam, Eva Zangerle
2024 — conf
Overview of the Multilingual Text Detoxification Task at PAN 2024.
CLEF (Working Notes)
Daryna Dementieva, Daniil Moskovskiy, Nikolay Babakov, Abinew Ali Ayele, Naquee Rizwan, Florian Schneider, Xintong Wang, Seid Muhie Yimam, Dmitry Ustalov, Elisei Stakovskii, Alisa Smirnova, Ashraf Elnagar, Animesh Mukherjee, Alexander Panchenko
2024 J jnl
Reliable, Reproducible, and Really Fast Leaderboards with Evalica.
CoRR
Dmitry Ustalov
2023 A conf
4th Crowd Science Workshop - CANDLE: Collaboration of Humans and Learning Algorithms for Data Labeling.
WSDM
Dmitry Ustalov, Saiph Savage, Niels van Berkel, Yang Liu
2023 A* conf
Best Prompts for Text-to-Image Models and How to Find Them.
SIGIR
Nikita Pavlichenko, Dmitry Ustalov
2023 — conf
Clustering Without Knowing How To: Application and Evaluation.
ECIR (3)
Daniil Likhobaba, Daniil Fedulov, Dmitry Ustalov
2023 — conf
Crowdsourcing for Information Retrieval.
ECIR (3)
Dmitry Ustalov, Alisa Smirnova, Natalia Fedorova, Nikita Pavlichenko
2023 A* conf
Data Labeling for Machine Learning Engineers: Project-Based Curriculum and Data-Centric Competitions.
AAAI
Anastasia Zhdanovskaya, Daria Baidakova, Dmitry Ustalov
2023 — ed.
Proceedings of the 4th Crowd Science Workshop on Collaboration of Humans and Learning Algorithms for Data Labeling co-located with ACM International WSDM Conference (WSDM 2023), Singapore, March 3, 2023.
CSW@WSDM
Dmitry Ustalov, Saiph Savage, Niels van Berkel, Yang Liu, Alisa Smirnova
2023 J jnl
Toloka Visual Question Answering Benchmark.
CoRR
Dmitry Ustalov, Nikita Pavlichenko, Sergey Koshelev, Daniil Likhobaba, Alisa Smirnova
2023 — conf
WSDM Cup 2023 Challenge on Visual Question Answering.
CSW@WSDM
Dmitry Ustalov, Nikita Pavlichenko, Daniil Likhobaba, Alisa Smirnova
2022 J jnl
Best Prompts for Text-to-Image Models and How to Find Them.
CoRR
Nikita Pavlichenko, Dmitry Ustalov
2022 A conf
Challenges in Data Production for AI with Human-in-the-Loop.
WSDM
Dmitry Ustalov
2022 J jnl
Clustering Without Knowing How To: Application and Evaluation.
CoRR
Daniil Likhobaba, Daniil Fedulov, Dmitry Ustalov
2022 A conf
Improving Recommender Systems with Human-in-the-Loop.
RecSys
Dmitry Ustalov, Natalia Fedorova, Nikita Pavlichenko
2022 — conf
REGROW: Reimagining Global Crowdsourcing for Better Human-AI Collaboration.
CHI Extended Abstracts
Andy Alorwu, Saiph Savage, Niels van Berkel, Dmitry Ustalov, Alexey Drutsa, Jonas Oppenlaender, Oliver Bates, Danula Hettiachchi, Ujwal Gadiraju, Jorge Gonçalves, Simo Hosio
2022 B conf
Web Engineering with Human-in-the-Loop.
ICWE
Dmitry Ustalov, Nikita Pavlichenko, Boris Tseytlin, Daria Baidakova, Alexey Drutsa
2021 J jnl
A General-Purpose Crowdsourcing Computational Quality Control Toolkit for Python.
CoRR
Dmitry Ustalov, Nikita Pavlichenko, Vladimir Losev, Evgeny Tulin, Iulian Giliazev
2021 — conf
CrowdSpeech and Vox DIY: Benchmark Dataset for Crowdsourced Audio Transcription.
NeurIPS Datasets and Benchmarks
Nikita Pavlichenko, Ivan Stelmakh, Dmitry Ustalov
2021 J jnl
IMDB-WIKI-SbS: An Evaluation Dataset for Crowdsourced Pairwise Comparisons.
CoRR
Nikita Pavlichenko, Dmitry Ustalov
2021 — ed.
Proceedings of the 2nd Crowd Science Workshop: Trust, Ethics, and Excellence in Crowdsourced Data Management at Scale co-located with 47th International Conference on Very Large Data Bases (VLDB 2021), Copenhagen, Denmark, August 20, 2021.
CSW@VLDB
Dmitry Ustalov, Fabio Casati, Alexey Drutsa, Ivan Stelmakh, Nikita Pavlichenko, Daria Baidakova
2021 — conf
VLDB 2021 Crowd Science Challenge on Aggregating Crowdsourced Audio Transcriptions.
CSW@VLDB
Dmitry Ustalov, Nikita Pavlichenko, Ivan Stelmakh, Dmitriy Kuznetsov
2021 J jnl
Vox Populi, Vox DIY: Benchmark Dataset for Crowdsourced Audio Transcription.
CoRR
Nikita Pavlichenko, Ivan Stelmakh, Dmitry Ustalov
2020 — conf
Crowdsourcing Practice for Efficient Data Labeling: Aggregation, Incremental Relabeling, and Pricing.
SIGMOD Conference
Alexey Drutsa, Valentina Fedorova, Dmitry Ustalov, Olga Megorskaya, Evfrosiniya Zerminova, Daria Baidakova
2020 A conf
Practice of Efficient Data Collection via Crowdsourcing: Aggregation, Incremental Relabelling, and Pricing.
WSDM
Alexey Drutsa, Valentina Fedorova, Dmitry Ustalov, Olga Megorskaya, Evfrosiniya Zerminova, Daria Baidakova
2020 — ed.
Proceedings of the Crowd Science Workshop: Remoteness, Fairness, and Mechanisms as Challenges of Data Supply by Humans for Automation co-located with 34th Conference on Neural Information Processing Systems, CSW@NeurIPS 2020, Vancouver, BC, Canada / Online Event, December 11, 2020.
CSW@NeurIPS
Dmitry Ustalov, Fabio Casati, Alexey Drutsa, Daria Baidakova
2020 B conf
Word Sense Disambiguation for 158 Languages using Word Embeddings Only.
LREC
Varvara Logacheva, Denis Teslenko, Artem Shelmanov, Steffen Remus, Dmitry Ustalov, Andrey Kutuzov, Ekaterina Artemova, Chris Biemann, Simone Paolo Ponzetto, Alexander Panchenko
2020 J jnl
Word Sense Disambiguation for 158 Languages using Word Embeddings Only.
CoRR
Varvara Logacheva, Denis Teslenko, Artem Shelmanov, Steffen Remus, Dmitry Ustalov, Andrey Kutuzov, Ekaterina Artemova, Chris Biemann, Simone Paolo Ponzetto, Alexander Panchenko
2019 — conf
HHMM at SemEval-2019 Task 2: Unsupervised Frame Induction using Contextualized Word Embeddings.
SemEval@NAACL-HLT
Saba Anwar, Dmitry Ustalov, Nikolay Arefyev, Simone Paolo Ponzetto, Chris Biemann, Alexander Panchenko
2019 J jnl
HHMM at SemEval-2019 Task 2: Unsupervised Frame Induction using Contextualized Word Embeddings.
CoRR
Saba Anwar, Dmitry Ustalov, Nikolay Arefyev, Simone Paolo Ponzetto, Chris Biemann, Alexander Panchenko
2019 — ed.
Proceedings of the Thirteenth Workshop on Graph-Based Methods for Natural Language Processing, TextGraphs@EMNLP 2019, Hong Kong, November 4, 2019
TextGraphs@EMNLP
Dmitry Ustalov, Swapna Somasundaran, Peter Jansen, Goran Glavas, Martin Riedl, Mihai Surdeanu, Michalis Vazirgiannis
2019 — conf
TextGraphs 2019 Shared Task on Multi-Hop Inference for Explanation Regeneration.
TextGraphs@EMNLP
Peter Jansen, Dmitry Ustalov
2019 — conf
The Role of Student Projects in Teaching Machine Learning and High Performance Computing.
RuSCDays
Andrey Sozykin, Anton Koshelev, Dmitry Ustalov
2019 J jnl
Watset.
Comput. Linguistics
Dmitry Ustalov, Alexander Panchenko, Chris Biemann, Simone Paolo Ponzetto
2018 B conf
An Unsupervised Word Sense Disambiguation System for Under-Resourced Languages.
LREC
Dmitry Ustalov, Denis Teslenko, Alexander Panchenko, Mikhail Chernoskutov, Chris Biemann, Simone Paolo Ponzetto
2018 J jnl
An Unsupervised Word Sense Disambiguation System for Under-Resourced Languages.
CoRR
Dmitry Ustalov, Denis Teslenko, Alexander Panchenko, Mikhail Chernoskutov, Chris Biemann, Simone Paolo Ponzetto
2018 — conf
Equidistant Nodes Clustering: a Soft Clustering Algorithm Applied for Synset Induction.
DAMDID/RCDL
Mikhail Chernoskutov, Dmitry Ustalov
2018 B conf
Improving Hypernymy Extraction with Distributional Semantic Classes.
LREC
Alexander Panchenko, Dmitry Ustalov, Stefano Faralli, Simone Paolo Ponzetto, Chris Biemann
2018 J jnl
Local-Global Graph Clustering with Applications in Sense and Frame Induction.
CoRR
Dmitry Ustalov, Alexander Panchenko, Chris Biemann, Simone Paolo Ponzetto
2018 J jnl
RUSSE'2018: A Shared Task on Word Sense Induction for the Russian Language.
CoRR
Alexander Panchenko, Anastasiya Lopukhina, Dmitry Ustalov, Konstantin Lopukhin, Nikolay Arefyev, Alexey Leontyev, Natalia V. Loukachevitch
2018 J jnl
RUSSE: The First Workshop on Russian Semantic Similarity.
CoRR
Alexander Panchenko, Natalia V. Loukachevitch, Dmitry Ustalov, Denis Paperno, Christian Meyer, Natalia Konstantinova
2018 — conf
Unsupervised Semantic Frame Induction using Triclustering.
ACL (2)
Dmitry Ustalov, Alexander Panchenko, Andrei Kutuzov, Chris Biemann, Simone Paolo Ponzetto
2018 J jnl
Unsupervised Semantic Frame Induction using Triclustering.
CoRR
Dmitry Ustalov, Alexander Panchenko, Andrei Kutuzov, Chris Biemann, Simone Paolo Ponzetto
2018 — conf
Unsupervised Sense-Aware Hypernymy Extraction.
KONVENS
Dmitry Ustalov, Alexander Panchenko, Chris Biemann, Simone Paolo Ponzetto
2018 J jnl
Unsupervised Sense-Aware Hypernymy Extraction.
CoRR
Dmitry Ustalov, Alexander Panchenko, Chris Biemann, Simone Paolo Ponzetto
2017 — conf
Automatic Induction of Synsets from a Graph of Synonyms.
ACL (1)
Dmitry Ustalov, Alexander Panchenko, Chris Biemann
2017 Misc conf
Fighting with the Sparsity of Synonymy Dictionaries for Automatic Synset Induction.
AIST
Dmitry Ustalov, Mikhail Chernoskutov, Chris Biemann, Alexander Panchenko
2017 J jnl
Fighting with the Sparsity of Synonymy Dictionaries.
CoRR
Dmitry Ustalov, Mikhail Chernoskutov, Chris Biemann, Alexander Panchenko
2017 J jnl
Human and Machine Judgements for Russian Semantic Relatedness.
CoRR
Alexander Panchenko, Dmitry Ustalov, Nikolay Arefyev, Denis Paperno, Natalia Konstantinova, Natalia V. Loukachevitch, Chris Biemann
2017 J jnl
Improving Hypernymy Extraction with Distributional Semantic Classes.
CoRR
Alexander Panchenko, Dmitry Ustalov, Stefano Faralli, Simone Paolo Ponzetto, Chris Biemann
2017 — conf
Negative Sampling Improves Hypernymy Extraction Based on Projection Learning.
EACL (2)
Dmitry Ustalov, Nikolay Arefyev, Chris Biemann, Alexander Panchenko
2017 J jnl
Negative Sampling Improves Hypernymy Extraction Based on Projection Learning.
CoRR
Dmitry Ustalov, Nikolay Arefyev, Chris Biemann, Alexander Panchenko
2017 — conf
Unsupervised, Knowledge-Free, and Interpretable Word Sense Disambiguation.
EMNLP (System Demonstrations)
Alexander Panchenko, Fide Marten, Eugen Ruppert, Stefano Faralli, Dmitry Ustalov, Simone Paolo Ponzetto, Chris Biemann
2017 J jnl
Unsupervised, Knowledge-Free, and Interpretable Word Sense Disambiguation.
CoRR
Alexander Panchenko, Fide Marten, Eugen Ruppert, Stefano Faralli, Dmitry Ustalov, Simone Paolo Ponzetto, Chris Biemann
2017 J jnl
Watset: Automatic Induction of Synsets from a Graph of Synonyms.
CoRR
Dmitry Ustalov, Alexander Panchenko, Chris Biemann
2016 C conf
Eliminating Fuzzy Duplicates in Crowdsourced Lexical Resources.
GWC
Yuri Kiselev, Dmitry Ustalov, Sergey Porshnev
2016 Misc conf
Human and Machine Judgements for Russian Semantic Relatedness.
AIST
Alexander Panchenko, Dmitry Ustalov, Nikolay Arefyev, Denis Paperno, Natalia Konstantinova, Natalia V. Loukachevitch, Chris Biemann
2016 C conf
YARN: Spinning-in-Progress.
GWC
Pavel Braslavski, Dmitry Ustalov, Mikhail Mukhin, Yuri Kiselev
2015 Misc conf
TagBag: Annotating a Foreign Language Lexical Resource with Pictures.
AIST
Dmitry Ustalov
2014 A conf
A Spinning Wheel for YARN: User Interface for a Crowdsourced Thesaurus.
EACL
Pavel Braslavski, Dmitry Ustalov, Mikhail Mukhin
2014 Misc conf
Enhancing Russian Wordnets Using the Force of the Crowd.
AIST
Dmitry Ustalov
2014 — conf
NLPub: каталог и сообщество русских лингвистических ресурсов (NLPub: a Catalogue and a Community for Russian Linguistic Resources).
RCDL
Dmitry Ustalov
2014 — ed.
Supplementary Proceedings of the 3rd International Conference on Analysis of Images, Social Networks and Texts (AIST'2014), Yekaterinburg, Russia, April 10-12, 2014.
AIST (Supplement)
Dmitry I. Ignatov, Mikhail Yu. Khachay, Alexander Panchenko, Natalia Konstantinova, Rostislav E. Yavorsky, Dmitry Ustalov
2014 — conf
Towards Crowdsourcing and Cooperation in Linguistic Resources.
RuSSIR
Dmitry Ustalov
2014 J jnl
Towards crowdsourcing and cooperation in linguistic resources.
CoRR
Dmitry Ustalov
2014 — conf
Words Worth Attention: Predicting Words of the Week on the Russian Wiktionary.
KESW
Dmitry Ustalov
2013 J jnl
Orchestrating the Natural Language Processing Software in the Cloud Computing Environment.
J. Digit. Inf. Manag.
Dmitry Ustalov, Mikhail Goldshtein