Tom Kocmi - RankMe

86 papers A 1B 1C 3Misc 1Journal 33Unranked 43

Year	Rank	Type	Title / Venue / Authors
2026	J	`jnl`	Pearmut: Human Evaluation of Translation Made Trivial. CoRR Vilém Zouhar, Tom Kocmi
2026	J	`jnl`	Tiny Aya: Bridging Scale and Multilingual Depth. CoRR Alejandro Salamanca, Diana Abagyan, Daniel D'souza, Ammar Khairi, David Mora, Saurabh Dash, Viraat Aryabumi, Sara Rajaee, Mehrnaz Mofakhami, Ananya Sahu, Thomas Euyang, Brittawnya Prince, Madeline Smith, Hangyu Lin, Acyr Locatelli, Sara Hooker, Tom Kocmi, Aidan N. Gomez, Ivan Zhang, Phil Blunsom, Nick Frosst, Joelle Pineau, Beyza Ermis, Ahmet Üstün, Julia Kreutzer, Marzieh Fadaee
2026	J	`jnl`	Unlocking Reasoning Capability on Machine Translation in Large Language Models. CoRR Sara Rajaee, Sebastian Vincent, Alexandre Berard, Marzieh Fadaee, Kelly Marchisio, Tom Kocmi
2025	—	`conf`	AI-Assisted Human Evaluation of Machine Translation. NAACL (Long Papers) Vilém Zouhar, Tom Kocmi, Mrinmaya Sachan
2025	J	`jnl`	Déjà Vu: Multilingual LLM Evaluation through the Lens of Machine Translation Evaluation. CoRR Julia Kreutzer, Eleftheria Briakou, Sweta Agrawal, Marzieh Fadaee, Tom Kocmi
2025	—	`conf`	Estimating Machine Translation Difficulty. EMNLP (Findings) Lorenzo Proietti, Stefano Perrella, Vilém Zouhar, Roberto Navigli, Tom Kocmi
2025	J	`jnl`	Estimating Machine Translation Difficulty. CoRR Lorenzo Proietti, Stefano Perrella, Vilém Zouhar, Roberto Navigli, Tom Kocmi
2025	J	`jnl`	Machine Translation Meta Evaluation through Translation Accuracy Challenge Sets. Comput. Linguistics Nikita Moghe, Arnisa Fazla, Chantal Amrhein, Tom Kocmi, Mark Steedman, Alexandra Birch, Rico Sennrich, Liane Guillou
2025	J	`jnl`	Preliminary Ranking of WMT25 General Machine Translation Systems. CoRR Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ondrej Bojar, Konstantin Dranch, Anton Dvorkovich, Sergey Dukanov, Natalia Fedorova, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Marzena Karpinska, Philipp Koehn, Howard Lakougna, Jessica Lundin, Kenton Murray, Masaaki Nagata, Stefano Perrella, Lorenzo Proietti, Martin Popel, Maja Popovic, Parker Riley, Mariya Shmatova, Steinþór Steingrímsson, Lisa Yankovskaya, Vilém Zouhar
2024	J	`jnl`	AI-Assisted Human Evaluation of Machine Translation. CoRR Vilém Zouhar, Tom Kocmi, Mrinmaya Sachan
2024	—	`conf`	Are LLMs Breaking MT Metrics? Results of the WMT24 Metrics Shared Task. WMT Markus Freitag, Nitika Mathur, Daniel Deutsch, Chi-kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Frédéric Blain, Tom Kocmi, Jiayi Wang, David Ifeoluwa Adelani, Marianna Buchicchio, Chrysoula Zerva, Alon Lavie
2024	J	`jnl`	Aya Expanse: Combining Research Breakthroughs for a New Multilingual Frontier. CoRR John Dang, Shivalika Singh, Daniel D'souza, Arash Ahmadian, Alejandro Salamanca, Madeline Smith, Aidan Peppin, Sungjin Hong, Manoj Govindassamy, Terrence Zhao, Sandra Kublik, Meor Amer, Viraat Aryabumi, Jon Ander Campos, Yi Chern Tan, Tom Kocmi, Florian Strub, Nathan Grinsztajn, Yannis Flet-Berliac, Acyr Locatelli, Hangyu Lin, Dwarak Talupuru, Bharat Venkitesh, David Cairuz, Bowen Yang, Tim Chung, Wei-Yin Ko, Sylvie Shang Shi, Amir Shukayev, Sammie Bae, Aleksandra Piktus, Roman Castagné, Felipe Cruz-Salinas, Eddie Kim, Lucas Crawhall-Stein, Adrien Morisot, Sudip Roy, Phil Blunsom, Ivan Zhang, Aidan N. Gomez, Nick Frosst, Marzieh Fadaee, Beyza Ermis, Ahmet Üstün, Sara Hooker
2024	—	`conf`	Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Models. ACL (Findings) Qingyu Lu, Baopu Qiu, Liang Ding, Kanjian Zhang, Tom Kocmi, Dacheng Tao
2024	—	`conf`	Error Span Annotation: A Balanced Approach for Human Evaluation of Machine Translation. WMT Tom Kocmi, Vilém Zouhar, Eleftherios Avramidis, Roman Grundkiewicz, Marzena Karpinska, Maja Popovic, Mrinmaya Sachan, Mariya Shmatova
2024	J	`jnl`	Error Span Annotation: A Balanced Approach for Human Evaluation of Machine Translation. CoRR Tom Kocmi, Vilém Zouhar, Eleftherios Avramidis, Roman Grundkiewicz, Marzena Karpinska, Maja Popovic, Mrinmaya Sachan, Mariya Shmatova
2024	—	`conf`	Findings of the WMT24 General Machine Translation Shared Task: The LLM Era Is Here but MT Is Not Solved Yet. WMT Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ondrej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Marzena Karpinska, Philipp Koehn, Benjamin Marie, Christof Monz, Kenton Murray, Masaaki Nagata, Martin Popel, Maja Popovic, Mariya Shmatova, Steinthór Steingrímsson, Vilém Zouhar
2024	J	`jnl`	Machine Translation Meta Evaluation through Translation Accuracy Challenge Sets. CoRR Nikita Moghe, Arnisa Fazla, Chantal Amrhein, Tom Kocmi, Mark Steedman, Alexandra Birch, Rico Sennrich, Liane Guillou
2024	—	`conf`	Navigating the Metrics Maze: Reconciling Score Magnitudes and Accuracies. ACL (1) Tom Kocmi, Vilém Zouhar, Christian Federmann, Matt Post
2024	J	`jnl`	Navigating the Metrics Maze: Reconciling Score Magnitudes and Accuracies. CoRR Tom Kocmi, Vilém Zouhar, Christian Federmann, Matt Post
2024	—	`conf`	Not All Metrics Are Guilty: Improving NLG Evaluation by Diversifying References. NAACL-HLT Tianyi Tang, Hongyuan Lu, Yuchen Jiang, Haoyang Huang, Dongdong Zhang, Wayne Xin Zhao, Tom Kocmi, Furu Wei
2024	J	`jnl`	Preliminary WMT24 Ranking of General MT Systems and LLMs. CoRR Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ondrej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Marzena Karpinska, Philipp Koehn, Benjamin Marie, Kenton Murray, Masaaki Nagata, Martin Popel, Maja Popovic, Mariya Shmatova, Steinþór Steingrímsson, Vilém Zouhar
2024	—	`ed.`	Proceedings of the Ninth Conference on Machine Translation, WMT 2024, Miami, FL, USA, November 15-16, 2024 WMT Barry Haddow, Tom Kocmi, Philipp Koehn, Christof Monz
2024	—	`conf`	SLIDE: Reference-free Evaluation for Machine Translation using a Sliding Document Window. NAACL (Short Papers) Vikas Raunak, Tom Kocmi, Matt Post
2023	—	`conf`	Cometoid: Distilling Strong Reference-based Machine Translation Metrics into Even Stronger Quality Estimation Metrics. WMT Thamme Gowda, Tom Kocmi, Marcin Junczys-Dowmunt
2023	—	`conf`	Evaluating Metrics for Document-context Evaluation in Machine Translation. WMT Vikas Raunak, Tom Kocmi, Matt Post
2023	—	`conf`	Findings of the 2023 Conference on Machine Translation (WMT23): LLMs Are Here but Not Quite There Yet. WMT Tom Kocmi, Eleftherios Avramidis, Rachel Bawden, Ondrej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Markus Freitag, Thamme Gowda, Roman Grundkiewicz, Barry Haddow, Philipp Koehn, Benjamin Marie, Christof Monz, Makoto Morishita, Kenton Murray, Makoto Nagata, Toshiaki Nakazawa, Martin Popel, Maja Popovic, Mariya Shmatova
2023	—	`conf`	Findings of the WMT 2023 Shared Task on Machine Translation with Terminologies. WMT Kirill Semenov, Vilém Zouhar, Tom Kocmi, Dongdong Zhang, Wangchunshu Zhou, Yuchen Eleanor Jiang
2023	—	`conf`	GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4. WMT Tom Kocmi, Christian Federmann
2023	J	`jnl`	GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4. CoRR Tom Kocmi, Christian Federmann
2023	C	`conf`	Large Language Models Are State-of-the-Art Evaluators of Translation Quality. EAMT Tom Kocmi, Christian Federmann
2023	J	`jnl`	Large Language Models Are State-of-the-Art Evaluators of Translation Quality. CoRR Tom Kocmi, Christian Federmann
2023	—	`conf`	Overview of the Second Shared Task on Automatic Minuting (AutoMin) at INLG 2023. INLG (Generation Challenges) Tirthankar Ghosal, Ondrej Bojar, Marie Hledíková, Tom Kocmi, Anna Nedoluzhko
2023	A	`conf`	Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference. EACL Vilém Zouhar, Shehzaad Dhuliawala, Wangchunshu Zhou, Nico Daheim, Tom Kocmi, Yuchen Eleanor Jiang, Mrinmaya Sachan
2023	J	`jnl`	Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference. CoRR Vilém Zouhar, Shehzaad Dhuliawala, Wangchunshu Zhou, Nico Daheim, Tom Kocmi, Yuchen Eleanor Jiang, Mrinmaya Sachan
2023	—	`ed.`	Proceedings of the Eighth Conference on Machine Translation, WMT 2023, Singapore, December 6-7, 2023 WMT Philipp Koehn, Barry Haddon, Tom Kocmi, Christof Monz
2023	—	`conf`	Results of WMT23 Metrics Shared Task: Metrics Might Be Guilty but References Are Not Innocent. WMT Markus Freitag, Nitika Mathur, Chi-kiu Lo, Eleftherios Avramidis, Ricardo Rei, Brian Thompson, Tom Kocmi, Frédéric Blain, Daniel Deutsch, Craig Stewart, Chrysoula Zerva, Sheila Castilho, Alon Lavie, George F. Foster
2023	J	`jnl`	SLIDE: Reference-free Evaluation for Machine Translation using a Sliding Document Window. CoRR Vikas Raunak, Tom Kocmi, Matt Post
2023	—	`conf`	eBLEU: Unexpectedly Good Machine Translation Evaluation Using Simple Word Embeddings. WMT Muhammad N. ElNokrashy, Tom Kocmi
2022	—	`conf`	Findings of the 2022 Conference on Machine Translation (WMT22). WMT Tom Kocmi, Rachel Bawden, Ondrej Bojar, Anton Dvorkovich, Christian Federmann, Mark Fishel, Thamme Gowda, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Rebecca Knowles, Philipp Koehn, Christof Monz, Makoto Morishita, Masaaki Nagata, Toshiaki Nakazawa, Michal Novák, Martin Popel, Maja Popovic
2022	—	`conf`	MS-COMET: More and Better Human Judgements Improve Metric Performance. WMT Tom Kocmi, Hitokazu Matsushita, Christian Federmann
2022	—	`ed.`	Proceedings of the Seventh Conference on Machine Translation, WMT 2022, Abu Dhabi, United Arab Emirates (Hybrid), December 7-8, 2022 WMT Philipp Koehn, Loïc Barrault, Ondrej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno-Yepes, Tom Kocmi, André F. T. Martins, Makoto Morishita, Christof Monz, Masaaki Nagata, Toshiaki Nakazawa, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Marco Turchi, Marcos Zampieri
2022	—	`conf`	Results of WMT22 Metrics Shared Task: Stop Using BLEU - Neural Metrics Are Better and More Robust. WMT Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, Eleftherios Avramidis, Tom Kocmi, George F. Foster, Alon Lavie, André F. T. Martins
2022	—	`conf`	Searching for a Higher Power in the Human Evaluation of MT. WMT Johnny Tian-Zheng Wei, Tom Kocmi, Christian Federmann
2022	J	`jnl`	Searching for a higher power in the human evaluation of MT. CoRR Johnny Tian-Zheng Wei, Tom Kocmi, Christian Federmann
2022	J	`jnl`	The Reality of Multi-Lingual Machine Translation. CoRR Tom Kocmi, Dominik Machácek, Ondrej Bojar
2021	—	`conf`	Findings of the 2021 Conference on Machine Translation (WMT21). WMT@EMNLP Farhad Akhbardeh, Arkady Arkhangorodsky, Magdalena Biesialska, Ondrej Bojar, Rajen Chatterjee, Vishrav Chaudhary, Marta R. Costa-jussà, Cristina España-Bonet, Angela Fan, Christian Federmann, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Leonie Harter, Kenneth Heafield, Christopher Homan, Matthias Huck, Kwabena Amponsah-Kaakyire, Jungo Kasai, Daniel Khashabi, Kevin Knight, Tom Kocmi, Philipp Koehn, Nicholas Lourie, Christof Monz, Makoto Morishita, Masaaki Nagata, Ajay Nagesh, Toshiaki Nakazawa, Matteo Negri, Santanu Pal, Allahsera Auguste Tapo, Marco Turchi, Valentin Vydrin, Marcos Zampieri
2021	J	`jnl`	On User Interfaces for Large-Scale Document-Level Human Evaluation of Machine Translation Outputs. CoRR Roman Grundkiewicz, Marcin Junczys-Dowmunt, Christian Federmann, Tom Kocmi
2021	—	`ed.`	Proceedings of the Sixth Conference on Machine Translation, WMT@EMNLP 2021, Online Event, November 10-11, 2021 WMT@EMNLP Loïc Barrault, Ondrej Bojar, Fethi Bougares, Rajen Chatterjee, Marta R. Costa-jussà, Christian Federmann, Mark Fishel, Alexander Fraser, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Paco Guzman, Barry Haddow, Matthias Huck, Antonio Jimeno-Yepes, Philipp Koehn, Tom Kocmi, André F. T. Martins, Makoto Morishita, Christof Monz
2021	—	`conf`	THEaiTRE 1.0: Interactive Generation of Theatre Play Scripts. Text2Story@ECIR Rudolf Rosa, Tomás Musil, Ondrej Dusek, Dominik Jurko, Patrícia Schmidtová, David Marecek, Ondrej Bojar, Tom Kocmi, Daniel Hrbek, David Kosták, Martina Kinská, Marie Nováková, Josef Dolezal, Klára Vosecká, Tomás Studeník, Petr Zabka
2021	J	`jnl`	THEaiTRE 1.0: Interactive generation of theatre play scripts. CoRR Rudolf Rosa, Tomás Musil, Ondrej Dusek, Dominik Jurko, Patrícia Schmidtová, David Marecek, Ondrej Bojar, Tom Kocmi, Daniel Hrbek, David Kosták, Martina Kinská, Marie Nováková, Josef Dolezal, Klára Vosecká, Tomás Studeník, Petr Zabka
2021	—	`conf`	To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation. WMT@EMNLP Tom Kocmi, Christian Federmann, Roman Grundkiewicz, Marcin Junczys-Dowmunt, Hitokazu Matsushita, Arul Menezes
2021	J	`jnl`	To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation. CoRR Tom Kocmi, Christian Federmann, Roman Grundkiewicz, Marcin Junczys-Dowmunt, Hitokazu Matsushita, Arul Menezes
2021	C	`conf`	When a Robot Writes a Play: Automatically Generating a Theatre Play Script. ALIFE Rudolf Rosa, Tomás Musil, Ondrej Dusek, Dominik Jurko, Patrícia Schmidtová, David Marecek, Ondrej Bojar, Tom Kocmi, Daniel Hrbek, David Kosták, Martina Kinská, Marie Nováková, Josef Dolezal, Klára Vosecká, Tomás Studeník, Petr Zabka
2020	J	`jnl`	Announcing CzEng 2.0 Parallel Corpus with over 2 Gigawords. CoRR Tom Kocmi, Martin Popel, Ondrej Bojar
2020	—	`conf`	CUNI Submission for the Inuktitut Language in WMT News 2020. WMT@EMNLP Tom Kocmi
2020	—	`conf`	CUNI Systems for the Unsupervised and Very Low Resource Translation Task in WMT20. WMT@EMNLP Ivana Kvapilíková, Tom Kocmi, Ondrej Bojar
2020	J	`jnl`	CUNI Systems for the Unsupervised and Very Low Resource Translation Task in WMT20. CoRR Ivana Kvapilíková, Tom Kocmi, Ondrej Bojar
2020	C	`conf`	Efficiently Reusing Old Models Across Languages via Transfer Learning. EAMT Tom Kocmi, Ondrej Bojar
2020	J	`jnl`	Exploring Benefits of Transfer Learning in Neural Machine Translation. CoRR Tom Kocmi
2020	—	`conf`	Findings of the 2020 Conference on Machine Translation (WMT20). WMT@EMNLP Loïc Barrault, Magdalena Biesialska, Ondrej Bojar, Marta R. Costa-jussà, Christian Federmann, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Matthias Huck, Eric Joanis, Tom Kocmi, Philipp Koehn, Chi-kiu Lo, Nikola Ljubesic, Christof Monz, Makoto Morishita, Masaaki Nagata, Toshiaki Nakazawa, Santanu Pal, Matt Post, Marcos Zampieri
2020	—	`conf`	Gender Coreference and Bias Evaluation at WMT 2020. WMT@EMNLP Tom Kocmi, Tomasz Limisiewicz, Gabriel Stanovsky
2020	J	`jnl`	Gender Coreference and Bias Evaluation at WMT 2020. CoRR Tom Kocmi, Tomasz Limisiewicz, Gabriel Stanovsky
2020	—	`conf`	THEaiTRE: Artificial Intelligence to Write a Theatre Play. AI4Narratives@IJCAI Rudolf Rosa, Ondrej Dusek, Tom Kocmi, David Marecek, Tomás Musil, Patrícia Schmidtová, Dominik Jurko, Ondrej Bojar, Daniel Hrbek, David Kosták, Martina Kinská, Josef Dolezal, Klára Vosecká
2020	J	`jnl`	THEaiTRE: Artificial Intelligence to Write a Theatre Play. CoRR Rudolf Rosa, Ondrej Dusek, Tom Kocmi, David Marecek, Tomás Musil, Patrícia Schmidtová, Dominik Jurko, Ondrej Bojar, Daniel Hrbek, David Kosták, Martina Kinská, Josef Dolezal, Klára Vosecká
2019	—	`conf`	CUNI Submission for Low-Resource Languages in WMT News 2019. WMT (2) Tom Kocmi, Ondrej Bojar
2019	J	`jnl`	Transfer Learning across Languages from Someone Else's NMT Model. CoRR Tom Kocmi, Ondrej Bojar
2018	—	`conf`	CUNI Basque-to-English Submission in IWSLT18. IWSLT Tom Kocmi, Dusan Varis, Ondrej Bojar
2018	—	`conf`	CUNI NMT System for WAT 2018 Translation Tasks. WAT@PACLIC Tom Kocmi, Shantipriya Parida, Ondrej Bojar
2018	—	`conf`	CUNI Submissions in WMT18. WMT (shared task) Tom Kocmi, Roman Sudarikov, Ondrej Bojar
2018	—	`conf`	Neural Monkey: The Current State and Beyond. AMTA (1) Jindrich Helcl, Jindrich Libovický, Tom Kocmi, Tomás Musil, Ondrej Cífka, Dusan Varis, Ondrej Bojar
2018	J	`jnl`	SubGram: Extending Skip-gram Word Representation with Substrings. CoRR Tom Kocmi, Ondrej Bojar
2018	B	`conf`	SumeCzech: Large Czech News-Based Summarization Dataset. LREC Milan Straka, Nikita Mediankin, Tom Kocmi, Zdenek Zabokrtský, Vojtech Hudecek, Jan Hajic
2018	—	`conf`	Trivial Transfer Learning for Low-Resource Neural Machine Translation. WMT Tom Kocmi, Ondrej Bojar
2018	J	`jnl`	Trivial Transfer Learning for Low-Resource Neural Machine Translation. CoRR Tom Kocmi, Ondrej Bojar
2017	—	`conf`	An Exploration of Word Embedding Initialization in Deep-Learning Tasks. ICON Tom Kocmi, Ondrej Bojar
2017	J	`jnl`	An Exploration of Word Embedding Initialization in Deep-Learning Tasks. CoRR Tom Kocmi, Ondrej Bojar
2017	—	`conf`	CUNI NMT System for WAT 2017 Translation Tasks. WAT@IJCNLP Tom Kocmi, Dusan Varis, Ondrej Bojar
2017	—	`conf`	CUNI submission in WMT17: Chimera goes neural. WMT Roman Sudarikov, David Marecek, Tom Kocmi, Dusan Varis, Ondrej Bojar
2017	Misc	`conf`	Curriculum Learning and Minibatch Bucketing in Neural Machine Translation. RANLP Tom Kocmi, Ondrej Bojar
2017	J	`jnl`	Curriculum Learning and Minibatch Bucketing in Neural Machine Translation. CoRR Tom Kocmi, Ondrej Bojar
2017	J	`jnl`	LanideNN: Multilingual Language Identification on Character Window. CoRR Tom Kocmi, Ondrej Bojar
2017	—	`conf`	LanideNN: Multilingual Language Identification on Text Stream. EACL (1) Tom Kocmi, Ondrej Bojar
2017	—	`conf`	Results of the WMT17 Neural MT Training Task. WMT Ondrej Bojar, Jindrich Helcl, Tom Kocmi, Jindrich Libovický, Tomás Musil
2016	—	`conf`	CzEng 1.6: Enlarged Czech-English Parallel Corpus with Processing Tools Dockered. TSD Ondrej Bojar, Ondrej Dusek, Tom Kocmi, Jindrich Libovický, Michal Novák, Martin Popel, Roman Sudarikov, Dusan Varis
2016	—	`conf`	SubGram: Extending Skip-Gram Word Representation with Substrings. TSD Tom Kocmi, Ondrej Bojar
2016	—	`conf`	UFAL Submissions to the IWSLT 2016 MT Track. IWSLT Ondrej Bojar, Ondrej Cífka, Jindrich Helcl, Tom Kocmi, Roman Sudarikov