Advances in Text-Guided Image Editing and Concept Erasure

The field of text-to-image diffusion models is rapidly advancing, with a focus on developing innovative methods for text-guided image editing and concept erasure. Researchers are working to improve the efficiency, precision, and safety of these models, with a particular emphasis on addressing the challenges of preserving identity, maintaining geometric consistency, and preventing the generation of harmful or undesirable content.

Recent innovations have led to the development of novel architectures and techniques, such as identity-preserving architectures, geometry-aware distillation losses, and collaborative concept erasing frameworks. These advancements have significantly enhanced the editing workflow, improved rendering and geometric quality, and increased the robustness of models against unsafe prompts.

Noteworthy papers in this area include: NeuSEditor, which introduces a novel method for text-guided editing of neural implicit surfaces derived from multi-view images, outperforming recent state-of-the-art methods. One Image is Worth a Thousand Words, which proposes a text-image Collaborative Concept Erasing framework that effectively bypasses the knowledge gap between text and image, significantly enhancing erasure efficacy. Instructing Text-to-Image Diffusion Models via Classifier-Guided Semantic Optimization, which optimizes semantic embeddings guided by attribute classifiers to steer text-to-image models toward desired edits without relying on text prompts. Responsible Diffusion Models via Constraining Text Embeddings within Safe Regions, which proposes a novel self-discovery approach to identifying a semantic direction vector in the embedding space to restrict text embedding within a safe region. Comprehensive Evaluation and Analysis for NSFW Concept Erasure in Text-to-Image Diffusion Models, which introduces a full-pipeline toolkit specifically designed for concept erasure and conducts a systematic study of NSFW concept erasure methods. Erased or Dormant, which systematically evaluates the robustness and reversibility of concept erasure methods and finds that erased concepts often reemerge with substantial visual fidelity after minimal adaptation. When Are Concepts Erased From Diffusion Models, which proposes two conceptual models for the erasure mechanism in diffusion models and introduces a suite of independent evaluations to assess the thoroughness of concept erasure.

Sources

NeuSEditor: From Multi-View Images to Text-Guided Neural Surface Edits

One Image is Worth a Thousand Words: A Usability Preservable Text-Image Collaborative Erasing Framework

Instructing Text-to-Image Diffusion Models via Classifier-Guided Semantic Optimization

Responsible Diffusion Models via Constraining Text Embeddings within Safe Regions

Comprehensive Evaluation and Analysis for NSFW Concept Erasure in Text-to-Image Diffusion Models

Erased or Dormant? Rethinking Concept Erasure Through Reversibility

When Are Concepts Erased From Diffusion Models?

Built with on top of