MSc Thesis Defense: Pelin Karadal, INVESTIGATING IMPLICIT PERSONALIZATION IN LARGE LANGUAGE MODELS THROUGH INFERRED POLITICAL ORIENTATION, Date & Time: 17 July, 2026 – 10:30 AM, Place: FENS L061

INVESTIGATING IMPLICIT PERSONALIZATION IN LARGE LANGUAGE MODELS THROUGH INFERRED POLITICAL ORIENTATION

Pelin Karadal
Computer Science and Engineering, MSc Thesis, 2026

Thesis Jury

     Asst. Prof. Dilara Keküllüoğlu (Thesis Advisor)

  Asst. Prof. Onur Varol

  Prof. Kürşat Çağıltay

Date & Time: 17th July, 2026 – 10:30 AM

Place: FENS L061

Keywords : Large Language Models, Implicit Personalization, Political Bias, Semantic Similarity, Natural Language Processing

Abstract

Large language models (LLMs) are increasingly used for everyday information-seeking tasks, raising concerns about implicit personalization, political bias, and the influence of inferred user characteristics on generated responses. This thesis investigates whether LLMs personalize responses based on inferred political orientation and examines the effects of such personalization across different models. The research consists of two complementary studies. Study 1 focuses on ChatGPT and investigates how its memory and custom instruction features influence responses when political orientation is implied through user statements rather than explicitly stated. A qualitative analysis is complemented by a quantitative extension using sentence embeddings and cosine similarity to measure semantic differences between persona-conditioned responses. Study 2 extends this investigation to multiple state-of-the-art LLMs, including DeepSeek, LLaMA, Mistral, and Qwen, using both open-ended and fill-in-the-blank questions to evaluate whether persona-conditioning effects generalize across model families. The findings indicate that responses remain largely similar in semantic content across conditions, while systematic differences emerge in framing, emphasis, and justification. In ChatGPT, responses generated under a neutral persona show greater semantic similarity to U.S. Democrat-aligned responses than to U.S. Republican-aligned responses. Across models, persona conditioning produces subtle but consistent variations, suggesting that implicit personalization operates at a fine-grained level rather than altering core semantic content. Survey findings further indicate that personalization features such as memory and custom instructions are actively used by ChatGPT users, highlighting the practical relevance of these findings. Overall, this thesis provides evidence that LLMs may adapt responses to inferred user characteristics in subtle ways, with implications for fairness, transparency, and information diversity.