The field of AI safety is moving towards a deeper understanding of the complexity and evolution of jailbreaking strategies, with a focus on the development of practical bounds on attack sophistication. Researchers are investigating the limits of human ingenuity in exploiting AI vulnerabilities and the potential for defensive measures to advance. A key area of research is the development of methods to detect and prevent jailbreaking, including the use of probes to anticipate arithmetic errors and monitor misaligned reasoning models.

Noteworthy papers in this area include:

• Mass-Scale Analysis of In-the-Wild Conversations Reveals Complexity Bounds on LLM Jailbreaking, which challenges the prevailing narrative of an escalating arms race between attackers and defenders.
• Jailbreak-Tuning: Models Efficiently Learn Jailbreak Susceptibility, which demonstrates the vulnerability of fine-tunable models to jailbreaking attacks.
• Probing for Arithmetic Errors in Language Models, which shows that internal activations in language models can be used to detect arithmetic errors.
• Can We Predict Alignment Before Models Finish Thinking, which investigates the use of chain-of-thought traces to predict final response misalignment.