Advances in Multilingual Language Models and Speech Translation

The field of natural language processing is moving towards more advanced and nuanced models for speech translation and multilingual language understanding. Recent work has focused on improving the performance of large language models (LLMs) in low-resource languages and developing more effective methods for cross-lingual transfer. One notable direction is the integration of spatial perception into speech translation, allowing for more immersive and interactive experiences. Another area of research is the development of more efficient and scalable methods for domain-adaptive continual pretraining, which can improve the performance of LLMs in specific domains such as law and the process industry. Noteworthy papers in this area include those that propose novel approaches to calibrating translation decoding, enhancing LLM language adaptation through cross-lingual in-context pretraining, and improving retrieval-augmented neural machine translation with monolingual data. For example, Calibrating Translation Decoding with Quality Estimation on LLMs presents a method for enhancing the effectiveness of translation decoding, while Enhancing LLM Language Adaption through Cross-lingual In-Context Pre-training proposes a simple and scalable approach to improving cross-lingual transfer. Improving Retrieval-Augmented Neural Machine Translation with Monolingual Data demonstrates the benefits of using monolingual data to improve translation performance.

Sources

Spatial Speech Translation: Translating Across Space With Binaural Hearables

SynLexLM: Scaling Legal LLMs with Synthetic Data and Curriculum Learning

LawFlow : Collecting and Simulating Lawyers' Thought Processes

Calibrating Translation Decoding with Quality Estimation on LLMs

Efficient Domain-adaptive Continual Pretraining for the Process Industry in the German Language

What Causes Knowledge Loss in Multilingual Language Models?

Enhancing LLM Language Adaption through Cross-lingual In-Context Pre-training

Enhancing Non-Core Language Instruction-Following in Speech LLMs via Semi-Implicit Cross-Lingual CoT Reasoning

HYPEROFA: Expanding LLM Vocabulary to New Languages via Hypernetwork-Based Embedding Initialization

Automatic Legal Writing Evaluation of LLMs

Retrieval-Enhanced Few-Shot Prompting for Speech Event Extraction

20min-XD: A Comparable Corpus of Swiss News Articles

Investigating the Effect of Parallel Data in the Cross-Lingual Transfer for Vision-Language Encoders

Improving Retrieval-Augmented Neural Machine Translation with Monolingual Data

Aleph-Alpha-GermanWeb: Improving German-language LLM pre-training with model-based data curation and synthetic data generation

Built with on top of