Instructions to use InstantX/SD3-Controlnet-Canny with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Diffusers
How to use InstantX/SD3-Controlnet-Canny with Diffusers:
pip install -U diffusers transformers accelerate
import torch from diffusers import DiffusionPipeline # switch to "mps" for apple devices pipe = DiffusionPipeline.from_pretrained("InstantX/SD3-Controlnet-Canny", dtype=torch.bfloat16, device_map="cuda") prompt = "Astronaut in a jungle, cold color palette, muted colors, detailed, 8k" image = pipe(prompt).images[0] - Notebooks
- Google Colab
- Kaggle
Unable to run on free tier Google Colab
#9
by sudhir2016 - opened
Runs out of VRAM.
HI, you can use any one of the memory optimization option given here: https://huggingface.co/docs/diffusers/main/en/api/pipelines/stable_diffusion/stable_diffusion_3#memory-optimisations-for-sd3
eg: pipe = StableDiffusion3ControlNetPipeline.from_pretrained(
"stabilityai/stable-diffusion-3-medium-diffusers", controlnet=controlnet, torch_dtype=torch.float16,
text_encoder_3=None, tokenizer_3=None ) will drop the T5 text encoder making it possible to run on colab or try other methods given in the link.