Integrating Physics and Language Models for Enhanced Automation and Realism

The field of engineering construction and automation is witnessing a significant shift towards integrating physics and language models to achieve more realistic and physically viable structures. Researchers are exploring the potential of large language models (LLMs) in various applications, including construction automation, anomaly detection, and image editing. A key focus area is the development of physics-informed LLMs that can operate within the constraints of physical principles, enabling more accurate and reliable results. Another area of research is the creation of benchmarks and evaluation protocols to assess the capabilities of LLMs in these domains. Noteworthy papers in this area include: BuildArena, which introduces a physics-aligned interactive benchmark for language-driven engineering construction. PILLM, which presents a physics-informed LLM framework for anomaly detection in HVAC systems. PICABench, which systematically evaluates physical realism in image editing and proposes effective solutions by learning physics from videos.

Sources

BuildArena: A Physics-Aligned Interactive Benchmark of LLMs for Engineering Construction

Physics-Informed Large Language Models for HVAC Anomaly Detection with Autonomous Rule Generation

Real-Time World Crafting: Generating Structured Game Behaviors from Natural Language with Large Language Models

PICABench: How Far Are We from Physically Realistic Image Editing?

Built with on top of