Multilingual Advances in Large Language Models

The field of natural language processing is witnessing significant developments in the area of multilingual large language models. Recent research has focused on improving the performance and reliability of these models across languages, with a particular emphasis on low-resource languages. One of the key directions is the exploration of novel architectures and training methods that can better capture cross-lingual relationships and alignments. Another important area of research is the evaluation and mitigation of biases in machine translation and other NLP tasks, including gender bias and language-specific biases. Noteworthy papers in this area include XTRA, which proposes a novel framework for cross-lingual topic modeling, and Beyond the Final Layer, which introduces a suite of training-free methods for improving multilingual calibration in large language models. Additionally, TRepLiNa presents a low-cost approach to improving low-resource machine translation by aligning mid-level layers using Centered Kernel Alignment and REPINA. GAMBIT+ introduces a large-scale challenge set for evaluating gender bias in machine translation quality estimation metrics, and Language Lives in Sparse Dimensions proposes a simple, training-free method to identify and manipulate sparse dimensions for multilingual control.

Sources

XTRA: Cross-Lingual Topic Modeling with Topic and Representation Alignments

Topic Modeling as Long-Form Generation: Can Long-Context LLMs revolutionize NTM via Zero-Shot Prompting?

Beyond the Final Layer: Intermediate Representations for Better Multilingual Calibration in Large Language Models

Multilingual Routing in Mixture-of-Experts

TRepLiNa: Layer-wise CKA+REPINA Alignment Improves Low-Resource Machine Translation in Aya-23 8B

GAMBIT+: A Challenge Set for Evaluating Gender Bias in Machine Translation Quality Estimation Metrics

Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models

Revisiting Metric Reliability for Fine-grained Evaluation of Machine Translation and Summarization in Indian Languages

Language Lives in Sparse Dimensions: Toward Interpretable and Efficient Multilingual Control for Large Language Models

Built with on top of