Varun Gumma

27 papers A* 1C 2Journal 18Unranked 6
YearRankTypeTitle / Venue / Authors
2025 J jnl
OffTopicEval: When Large Language Models Enter the Wrong Chat, Almost Always!
CoRR
Jingdi Lei, Varun Gumma, Rishabh Bhardwaj, Seok Min Lim, Chuan Li, Amir Zadeh, Soujanya Poria
2025 J jnl
The role of synthetic data in Multilingual, Multi-cultural AI systems: Lessons from Indic Languages.
CoRR
Pranjal A. Chitale, Varun Gumma, Sanchit Ahuja, Prashant Kodali, Manan Uppadhyay, Deepthi Sudharsan, Sunayana Sitaram
2025 — conf
Towards Inducing Long-Context Abilities in Multilingual Neural Machine Translation Models.
NAACL (Long Papers)
Varun Gumma, Pranjal A. Chitale, Kalika Bali
2024 — conf
Akal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technology.
FAccT
Rishav Hada, Safiya Husain, Varun Gumma, Harshita Diddee, Aditya Yadavalli, Agrima Seth, Nidhi Kulkarni, Ujwal Gadiraju, Aditya Vashistha, Vivek Seshadri, Kalika Bali
2024 J jnl
Akal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technology.
CoRR
Rishav Hada, Safiya Husain, Varun Gumma, Harshita Diddee, Aditya Yadavalli, Agrima Seth, Nidhi Kulkarni, Ujwal Gadiraju, Aditya Vashistha, Vivek Seshadri, Kalika Bali
2024 J jnl
An Empirical Analysis of In-context Learning Abilities of LLMs for MT.
CoRR
Pranjal A. Chitale, Jay P. Gala, Varun Gumma, Mitesh M. Khapra, Raj Dabre
2024 — conf
Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?
EACL (Findings)
Rishav Hada, Varun Gumma, Adrian de Wynter, Harshita Diddee, Mohamed Ahmed, Monojit Choudhury, Kalika Bali, Sunayana Sitaram
2024 J jnl
Beyond Metrics: Evaluating LLMs' Effectiveness in Culturally Nuanced, Low-Resource Real-World Scenarios.
CoRR
Millicent Ochieng, Varun Gumma, Sunayana Sitaram, Jindong Wang, Vishrav Chaudhary, Keshet Ronen, Kalika Bali, Jacki O'Neill
2024 J jnl
Contamination Report for Multilingual Benchmarks.
CoRR
Sanchit Ahuja, Varun Gumma, Sunayana Sitaram
2024 J jnl
HEALTH-PARIKSHA: Assessing RAG Models for Health Chatbots in Real-World Multilingual Settings.
CoRR
Varun Gumma, Anandhita Raghunath, Mohit Jain, Sunayana Sitaram
2024 J jnl
MAFIA: Multi-Adapter Fused Inclusive LanguAge Models.
CoRR
Prachi Jain, Ashutosh Sathe, Varun Gumma, Kabir Ahuja, Sunayana Sitaram
2024 — conf
MAFIA: Multi-Adapter Fused Inclusive Language Models.
EACL (1)
Prachi Jain, Ashutosh Sathe, Varun Gumma, Kabir Ahuja, Sunayana Sitaram
2024 — conf
MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks.
NAACL-HLT
Sanchit Ahuja, Divyanshu Aggarwal, Varun Gumma, Ishaan Watts, Ashutosh Sathe, Millicent Ochieng, Rishav Hada, Prachi Jain, Mohamed Ahmed, Kalika Bali, Sunayana Sitaram
2024 — conf
METAL: Towards Multilingual Meta-Evaluation.
NAACL-HLT (Findings)
Rishav Hada, Varun Gumma, Mohamed Ahmed, Kalika Bali, Sunayana Sitaram
2024 J jnl
METAL: Towards Multilingual Meta-Evaluation.
CoRR
Rishav Hada, Varun Gumma, Mohamed Ahmed, Kalika Bali, Sunayana Sitaram
2024 J jnl
MunTTS: A Text-to-Speech System for Mundari.
CoRR
Varun Gumma, Rishav Hada, Aditya Yadavalli, Pamir Gogoi, Ishani Mondal, Vivek Seshadri, Kalika Bali
2024 J jnl
On the Interchangeability of Positional Embeddings in Multilingual Neural Machine Translation Models.
CoRR
Varun Gumma, Pranjal A. Chitale, Kalika Bali
2024 J jnl
PARIKSHA : A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data.
CoRR
Ishaan Watts, Varun Gumma, Aditya Yadavalli, Vivek Seshadri, Manohar Swaminathan, Sunayana Sitaram
2024 A* conf
PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data.
EMNLP
Ishaan Watts, Varun Gumma, Aditya Yadavalli, Vivek Seshadri, Manohar Swaminathan, Sunayana Sitaram
2023 C conf
An Empirical Study of Leveraging Knowledge Distillation for Compressing Multilingual Neural Machine Translation Models.
EAMT
Varun Gumma, Raj Dabre, Pratyush Kumar
2023 J jnl
An Empirical Study of Leveraging Knowledge Distillation for Compressing Multilingual Neural Machine Translation Models.
CoRR
Varun Gumma, Raj Dabre, Pratyush Kumar
2023 J jnl
Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?
CoRR
Rishav Hada, Varun Gumma, Adrian de Wynter, Harshita Diddee, Mohamed Ahmed, Monojit Choudhury, Kalika Bali, Sunayana Sitaram
2023 J jnl
IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages.
CoRR
Jay P. Gala, Pranjal A. Chitale, Raghavan AK, Sumanth Doddapaneni, Varun Gumma, Aswanth Kumar, Janki Nawale, Anupama Sujatha, Ratish Puduppully, Vivek Raghavan, Pratyush Kumar, Mitesh M. Khapra, Raj Dabre, Anoop Kunchukuttan
2023 J jnl
IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languages.
Trans. Mach. Learn. Res.
Jay P. Gala, Pranjal A. Chitale, Raghavan AK, Varun Gumma, Sumanth Doddapaneni, Aswanth Kumar M., Janki Atul Nawale, Anupama Sujatha, Ratish Puduppully, Vivek Raghavan, Pratyush Kumar, Mitesh M. Khapra, Raj Dabre, Anoop Kunchukuttan
2023 J jnl
MEGAVERSE: Benchmarking Large Language Models Across Languages, Modalities, Models and Tasks.
CoRR
Sanchit Ahuja, Divyanshu Aggarwal, Varun Gumma, Ishaan Watts, Ashutosh Sathe, Millicent Ochieng, Rishav Hada, Prachi Jain, Maxamed Axmed, Kalika Bali, Sunayana Sitaram
2022 C conf
PAMMELA: Policy Administration Methodology using Machine Learning.
SECRYPT
Varun Gumma, Barsha Mitra, Soumyadeep Dey, Pratik Shashikantbhai Patel, Sourabh Suman, Saptarshi Das, Jaideep Vaidya
2021 J jnl
PAMMELA: Policy Administration Methodology using Machine Learning.
CoRR
Varun Gumma, Barsha Mitra, Soumyadeep Dey, Pratik Shashikantbhai Patel, Sourabh Suman, Saptarshi Das