Dynamic Scene Reconstruction and Physics-Informed Modeling

The field of computer vision is moving towards more sophisticated and physics-informed methods for dynamic scene reconstruction and modeling. Recent developments have focused on incorporating physical principles and constraints into neural network architectures to improve the accuracy and plausibility of reconstructions and predictions. This includes the use of 3D Gaussian representations, motion trajectory fields, and physics-informed losses to model complex motion patterns and dynamic scenes. Notable papers in this area include:

  • A paper that introduces a novel approach combining 3D Gaussian Splatting with a motion trajectory field for dynamic scene reconstruction, achieving state-of-the-art results in novel-view synthesis and motion trajectory recovery.
  • A paper that proposes a framework named TRACE to model the motion physics of complex dynamic 3D scenes from multi-view videos, directly learning a translation rotation dynamics system for each particle and explicitly estimating physical parameters to govern motion over time.

Sources

3D Gaussian Representations with Motion Trajectory Field for Dynamic Scene Reconstruction

Forecasting Continuous Non-Conservative Dynamical Systems in SO(3)

Mem4D: Decoupling Static and Dynamic Memory for Dynamic Scene Reconstruction

NeeCo: Image Synthesis of Novel Instrument States Based on Dynamic and Deformable 3D Gaussian Reconstruction

Learning an Implicit Physics Model for Image-based Fluid Simulation

Hybrid Long and Short Range Flows for Point Cloud Filtering

TRACE: Learning 3D Gaussian Physical Dynamics from Multi-view Videos

Built with on top of