The field of natural language processing is moving towards developing more culturally aware and inclusive language models. Recent research has highlighted the importance of evaluating language models in realistic, multicultural conversational settings and has introduced new frameworks and benchmarks to assess their cultural adaptation. The focus is on developing models that can understand and respond to diverse cultural contexts, values, and beliefs. Noteworthy papers in this area include: Culturally-Aware Conversations: A Framework & Benchmark for LLMs, which introduces a framework and benchmark for evaluating LLMs in multicultural conversational settings. I Am Aligned, But With Whom? MENA Values Benchmark for Evaluating Cultural Alignment and Multilingual Bias in LLMs, which presents a novel benchmark for evaluating cultural alignment and multilingual biases in LLMs with respect to the Middle East and North Africa region.
Cultural Awareness in Large Language Models
Sources
Culturally-Aware Conversations: A Framework & Benchmark for LLMs
Discrepancy Detection at the Data Level: Toward Consistent Multilingual Question Answering
Hey, wait a minute: on at-issue sensitivity in Language Models
The Curious Case of Curiosity across Human Cultures and LLMs
Developing and Validating the Arabic Version of the Attitudes Toward Large Language Models Scale
I Am Aligned, But With Whom? MENA Values Benchmark for Evaluating Cultural Alignment and Multilingual Bias in LLMs
Are Proverbs the New Pythian Oracles? Exploring Sentiment in Greek Sayings
CRaFT: An Explanation-Based Framework for Evaluating Cultural Reasoning in Multilingual Language Models
Building a Macedonian Recipe Dataset: Collection, Parsing, and Comparative Analysis
Assessing Socio-Cultural Alignment and Technical Safety of Sovereign LLMs