ELITE with Control

ELITE has the ability to create images of a specific object without the need for training. To further enhance control over the generated images, I integrated the use of Ip-Adapter.

Ip-Adapter

Ip-Adapter was originally impelemented with huggingface diffusors >=0.19.3. And the diffusors used in ELITE is 0.11.1 which is lack of many indispensibale features compared to 0.19.3 for implementing Ip-Adapter. I implemented Ip-Adapter here without using diffusors because it is eaiser than upgrade the diffusors.

Getting Started

Environment Setup

git clone https://github.com/Stevada/ELITE_plus_Control.git
cd ELITE_pluc_Control
conda create -n elite python=3.9
conda activate elite
pip install -r requirements.txt

Inference

Play around it by the inference script. If you want to test your customized dataset, you should align the image to ensure the object is at the center of image, and also provide the corresponding object mask. The object mask can be obtained by image-matting-app, or other image matting methods.

sh inferece_local_control.sh

TODO

ControlNet is on the way!!

Acknowledgements

This repo is a fork of ELITE which is built on diffusers version of Stable Diffusion. And reference to Ip-Adpater. I sincerely appreciate the authors for sharing the codes.