Large Language Models in Social Science and Gaming

The field of large language models (LLMs) is rapidly advancing, with a focus on improving their ability to simulate human personality, behavior, and decision-making. Recent research has explored the use of LLMs in social science experiments, evaluating their capacity to emulate human personality in virtual persona role-playing. Another area of focus is on developing more robust and consistent LLMs that can provide factual and reliable information, regardless of user context or personalization.

Noteworthy papers in this area include: Scaling Law in LLM Simulated Personality, which proposes a systematic framework for evaluating LLM virtual personality and identifies a scaling law in LLM personality simulation. ConsistencyAI introduces a benchmark for measuring the factual consistency of LLMs for different personas, offering impartial evaluation and accountability. Beyond One World benchmark evaluates character-grounded roleplay of LLMs across multiversal contexts, exposing critical gaps in multiversal consistency and reasoning alignment.

Sources

Scaling Law in LLM Simulated Personality: More Detailed and Realistic Persona Profile Is All You Need

Who's Asking? Evaluating LLM Robustness to Inquiry Personas in Factual Question Answering

Deflanderization for Game Dialogue: Balancing Character Authenticity with Task Execution in LLM-based NPCs

ConsistencyAI: A Benchmark to Assess LLMs' Factual Consistency When Responding to Different Demographic Groups

Combining Reinforcement Learning and Behavior Trees for NPCs in Video Games with AMD Schola

DPRF: A Generalizable Dynamic Persona Refinement Framework for Optimizing Behavior Alignment Between Personalized LLM Role-Playing Agents and Humans

Beyond One World: Benchmarking Super Heros in Role-Playing Across Multiversal Contexts

Built with on top of