publications

2024

1. Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs

   Sumanth Doddapaneni, Mohammed Safi Ur Rahman Khan, Dilip Venkatesh, and 3 more authors

   2024
2. BITS Pilani at SemEval-2024 Task 10: Fine-tuning BERT and Llama 2 for Emotion Recognition in Conversation

   Dilip Venkatesh, Pasunti Prasanjith, and Yashvardhan Sharma

   In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), Jun 2024

   Abs DOI

   Emotion Recognition in Conversation (ERC)aims to assign an emotion to a dialogue in aconversation between people. The first subtaskof EDiReF shared task aims to assign an emo-tions to a Hindi-English code mixed conversa-tion. For this, our team proposes a system toidentify the emotion based on fine-tuning largelanguage models on the MaSaC dataset. Forour study we have fine tuned 2 LLMs BERTand Llama 2 to perform sequence classificationto identify the emotion of the text.

3. BITS Pilani at SemEval-2024 Task 9: Prompt Engineering with GPT-4 for Solving Brainteasers

   Dilip Venkatesh, and Yashvardhan Sharma

   In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), Jun 2024

   Abs DOI

   Solving brainteasers is a task that requires complex reasoning prowess. The increase of research in natural language processing has leadto the development of massive large languagemodels with billions (or trillions) of parameters that are able to solve difficult questionsdue to their advanced reasoning capabilities.The SemEval BRAINTEASER shared tasks consists of sentence and word puzzles along withoptions containing the answer for the puzzle.Our team uses OpenAI’s GPT-4 model alongwith prompt engineering to solve these brainteasers.

4. BITS Pilani at SemEval-2024 Task 1: Using text-embedding-3-large and LaBSE embeddings for Semantic Textual Relatedness

   Dilip Venkatesh, and Sundaresan Raman

   In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024), Jun 2024

   Abs DOI

   Semantic Relatedness of a pair of text (sentences or words) is the degree to which theirmeanings are close. The Track A of the Semantic Textual Relatedness shared task aimsto find the semantic relatedness for the English language along with multiple other lowresource languages with the use of pretrainedlanguage models. We proposes a system tofind the Spearman coefficient of a textual pairusing pretrained embedding models like textembedding-3-large and LaBSE.