Physics-Informed Video Generation and Prediction

The field of video generation and prediction is moving towards incorporating physics-informed models to improve the realism and accuracy of generated videos. Researchers are exploring various approaches, including integrating physics simulators with video diffusion models, using Bayesian intention inference, and leveraging vision-language frameworks to predict trajectories and generate physically plausible motion. These innovative methods have shown significant improvements over traditional approaches, enabling more realistic and controllable video generation. Noteworthy papers include: ControlHair, which introduces a physics-informed video diffusion framework for controllable dynamic hair rendering. Generating Stable Placements via Physics-guided Diffusion Models, which integrates stability directly into the sampling process of a diffusion model to generate stable placements. Enhancing Physical Plausibility in Video Generation by Reasoning the Implausibility, which improves physical plausibility at inference time by explicitly reasoning about implausibility and guiding the generation away from it.

Sources

ControlHair: Physically-based Video Diffusion for Controllable Dynamic Hair Rendering

What Happens Next? Anticipating Future Motion by Generating Point Trajectories

Generating Stable Placements via Physics-guided Diffusion Models

Enhancing Physical Plausibility in Video Generation by Reasoning the Implausibility

Trajectory Prediction via Bayesian Intention Inference under Unknown Goals and Kinematics

From Seeing to Predicting: A Vision-Language Framework for Trajectory Forecasting and Controlled Video Generation

Learning to Generate Object Interactions with Physics-Guided Video Diffusion

Inferring Dynamic Physical Properties from Video Foundation Models

Built with on top of