The field of autonomous systems is experiencing rapid growth, driven by advancements in artificial intelligence (AI) and its integration with various technologies. Recent developments have shown significant potential in improving the safety, efficiency, and reliability of autonomous vehicles and drones. Notably, the use of large language models (LLMs) and vision-language models (VLMs) has enabled more accurate and robust perception, planning, and decision-making capabilities. These models have been successfully applied to tasks such as image segmentation, object detection, and autonomous driving, demonstrating improved performance and adaptability in complex scenarios. Furthermore, innovative approaches have been proposed to address the challenges of domain shift, explainability, and reliability in autonomous systems, paving the way for more widespread adoption and deployment. Noteworthy papers include Poutine, which presents a 3B-parameter vision-language model for end-to-end autonomous driving, achieving state-of-the-art performance on the Waymo test set. Another significant work is ADRD, which introduces a novel framework leveraging LLMs to generate executable, rule-based decision systems for autonomous driving, showcasing superior performance and interpretability. Additionally, NetRoller proposes an adapter for integrating general and specialized models in autonomous driving tasks, enabling seamless interaction and improved performance.
Autonomous Systems Advancements through AI Integration
Sources
Poutine: Vision-Language-Trajectory Pre-Training and Reinforcement Learning Post-Training Enable Robust End-to-End Autonomous Driving
On the Natural Robustness of Vision-Language Models Against Visual Perception Attacks in Autonomous Driving
Image Segmentation with Large Language Models: A Survey with Perspectives for Intelligent Transportation Systems