Advances in Image Generative Models and Memorization

The field of image generative models is moving towards a better understanding of memorization and its implications for privacy. Recent research has focused on localizing and mitigating memorization in image autoregressive models, exploring the internal behavior of these models and developing practical strategies for reducing privacy risks. Additionally, there is a growing interest in understanding the relationship between model-internal features and image memorability, with studies investigating the correlates of image memorability in pretrained vision encoders. Furthermore, research on predictive coding models of the neocortex has shed light on the episodic capabilities of these models and their relation to semantic memory. Noteworthy papers include:

  • A study on localizing and mitigating memorization in image autoregressive models, which reveals that memorization patterns differ across various architectures and can be reduced by intervening on the most memorizing components.
  • A paper on traces of image memorability in vision encoders, which finds that latent activations, attention distributions, and autoencoder losses correlate with memorability to some extent.
  • A study on semantic and episodic memories in a predictive coding model of the neocortex, which shows that the model can recall individual examples but only if trained on a small number of examples.

Sources

Localizing and Mitigating Memorization in Image Autoregressive Models

Traces of Image Memorability in Vision Encoders: Activations, Attention Distributions and Autoencoder Losses

Semantic and episodic memories in a predictive coding model of the neocortex

\emph{FoQuS}: A Forgetting-Quality Coreset Selection Framework for Automatic Modulation Recognition

Built with on top of