The field of speech processing and language models is moving towards greater emphasis on privacy and flexibility in data usage. Researchers are exploring ways to improve speech quality assessment, speech tokenization, and language model adherence to user-defined privacy preferences. A key direction is the development of mixture-of-experts (MoE) architectures, which enable more efficient and specialized processing of speech and language data. Another trend is the creation of novel speech tokenizers that can preserve prosodic and emotional content, leading to more accurate and effective speech representation. Furthermore, there is a growing interest in flexible language models that can be trained and used with closed datasets, allowing for greater control over data access and usage. Notable papers in this area include:

• Omni-Router, which introduces a shared router across different MoE layers for improved speech recognition.
• FlexOlmo, which proposes a new class of language models supporting distributed training and flexible data use.
• Speech Tokenizer is Key to Consistent Representation, which presents a novel speech tokenizer with broad applicability across downstream tasks.