MSc Thesis Defense: Sine Mete, A COMPARATIVE ANALYSIS OF GENDER BIAS IN AI CHATBOTS ACROSS ENGLISH AND TURKISH, Date & Time: 21 July, 2026 – 1:00 PM, Place: FENS L058

A COMPARATIVE ANALYSIS OF GENDER BIAS IN AI CHATBOTS ACROSS ENGLISH AND TURKISH

Sine Mete
Computer Science & Engineering, MSc Thesis, 2026

Thesis Jury

     Asst. Prof. Altuğ Tanaltay (Thesis Advisor)

  Asst. Prof. Dilara Keküllüoğlu

  Prof. Selcen Öztürkcan

Date & Time: 21st of July, 2026 – 1.00 PM

Place: FENS L058

Keywords : large language models, conversational AI, LLM-as-a-judge, AI ethics

Abstract

As AI chatbots become increasingly integrated into daily life, understanding how they reproduce gender bias is essential for fairer AI chatbot design. This thesis presents a comparative analysis of gender bias in three state-of-the-art AI chatbots: Google Gemini, Mistral Vibe, and DeepSeek using four distinct evaluation methods: sentence completion, persona generation, the Multilingual Bias Benchmark for Question Answering (MBBQ), and word association by ranking. While most existing studies focus on English, limited research has investigated gender bias in Turkish. A bilingual dataset was constructed in order to evaluate gender bias of AI chatbots across English and Turkish. For the sentence completion method, an LLM-as-a-Judge model was fine-tuned which was employed to score the gender bias of generated responses, while the remaining tasks were assessed using method-specific metrics. The results demonstrate that gender bias varies across evaluation methods and models as the evaluated models exhibited different bias patterns across tasks and languages. Across the four evaluation methods, prompts presented in Turkish elicited stronger gender-biased responses from the evaluated AI chatbots than their English counterparts in the sentence completion, persona generation, and word association tasks, whereas MBBQ produced largely consistent results across languages. These findings highlight the importance of employing multiple evaluation methods and multilingual assessments to better understand and mitigate gender bias in conversational AI systems.