Recently, large-scale generative models have demonstrated outstanding text-to-image generation
capabilities. However, generating high-fidelity personalized images with specific subjects still
presents challenges, especially in cases involving multiple subjects. In this paper, we propose
AnyStory, a unified approach for personalized subject generation. AnyStory not only achieves
high-fidelity personalization for single subjects, but also for multiple subjects, without
sacrificing subject fidelity. Specifically, AnyStory models the subject personalization problem
in an "encode-then-route" manner. In the encoding step, AnyStory utilizes a universal and
powerful image encoder, i.e., ReferenceNet, in conjunction with CLIP vision encoder to achieve
high-fidelity encoding of subject features. In the routing step, AnyStory utilizes a decoupled
instance-aware subject router to accurately perceive and predict the potential location of the
corresponding subject in the latent space, and guide the injection of subject conditions.
Detailed experimental results demonstrate the excellent performance of our method in retaining
subject details, aligning text descriptions, and personalizing for multiple subjects.
AnyStory follows the "encode-then-route" conditional generation paradigm. It first utilizes a simplified ReferenceNet combined with a CLIP vision encoder to encode the subject, and then employs a decoupled instance-aware subject router to guide the subject condition injection. The training process is divided into two stages: the subject encoder training stage and the router training stage. For brevity, we omit the text conditional branch here.
Scroll for more examples.
All subject images referenced in this paper are sourced from Pixabay and Unsplash. We extend our gratitude to the owners of these images for sharing their valuable assets.