Mitigating Biases in Language Models

The field of Natural Language Processing is moving towards a greater emphasis on fairness and transparency, with a particular focus on mitigating biases in language models. Recent studies have highlighted the importance of evaluating and addressing biases in language models, including gender bias, nationality bias, and social biases. The development of benchmark datasets and evaluation frameworks tailored to specific languages and contexts is crucial for advancing this research area. Noteworthy papers in this regard include the introduction of the Dutch CrowS-Pairs dataset for measuring social biases in Dutch language models, and the Obscured but Not Erased study, which explores nationality bias in large language models via name-based bias benchmarks, showing that small models exhibit more bias and are less accurate than larger counterparts.

Sources

GG-BBQ: German Gender Bias Benchmark for Question Answering

Dutch CrowS-Pairs: Adapting a Challenge Dataset for Measuring Social Biases in Language Models for Dutch

Introducing Quality Estimation to Machine Translation Post-editing Workflow: An Empirical Study on Its Usefulness

Exploring Gender Bias in Large Language Models: An In-depth Dive into the German Language

Obscured but Not Erased: Evaluating Nationality Bias in LLMs via Name-Based Bias Benchmarks

Uncertainty Quantification for Evaluating Machine Translation Bias

Built with on top of