Advances in Video Restoration and Generation

The field of video restoration and generation is rapidly advancing with a focus on developing more efficient and effective methods. Recent research has explored the use of implicit neural representations, hybrid temporal modeling, and adaptive frequency-aware skipping to improve video quality and reduce computational costs. These innovative approaches have shown promising results in terms of sharpness, detail preservation, and denoising efficacy. Notably, some methods have achieved state-of-the-art results while requiring significantly fewer computational resources. Overall, the field is moving towards more flexible and scalable solutions that can handle arbitrary scales and degradations. Noteworthy papers include:

  • VR-INR, which introduces a novel video restoration approach based on Implicit Neural Representations that generalizes effectively to arbitrary super-resolution scales.
  • LiftVSR, which proposes an efficient VSR framework that leverages image-wise diffusion prior and achieves state-of-the-art results using only 4xRTX 4090 GPUs.
  • SkipVAR, which presents a sample-adaptive framework that leverages frequency information to dynamically select the most suitable acceleration strategy for each instance.
  • MagCache, which introduces a Magnitude-aware Cache that adaptively skips unimportant timesteps using an error modeling mechanism and adaptive caching strategy.

Sources

Implicit Neural Representation for Video Restoration

LiftVSR: Lifting Image Diffusion to Video Super-Resolution via Hybrid Temporal Modeling with Only 4$\times$RTX 4090s

SkipVAR: Accelerating Visual Autoregressive Modeling via Adaptive Frequency-Aware Skipping

MagCache: Fast Video Generation with Magnitude-Aware Cache

Built with on top of