MoD ControlNet Tile Upscaler for SDXL🤗

This project implements the 📜 MoD (Mixture-of-Diffusers) tiled diffusion technique and combines it with SDXL's ControlNet Tile process. 💻 GitHub Code 🚀 Controlnet Union Power! Check out the model: Controlnet Union 🎨 RealVisXL V5.0 for Stunning Visuals! Explore it here: RealVisXL
Users are getting the 'ZeroGPU worker error' because the execution time on Zero GPU for non-Pro users is 120s. I reduced the number of steps to 18 in the LCM examples to avoid this error, however this will result in low quality in the final result. It is better to run this application locally or duplicate the environment for a paid account. If you are not a Pro account, run the LCM sampler examples on the RealVisXL_V5.0_Lightning model. For best results use the UniPC sampler and RealVisXL_V5.0 model examples.

Method

This project proposes an enhanced image upscaling method that leverages ControlNet Tile and Mixture-of-Diffusers techniques, integrating tile diffusion directly into the denoising process within the latent space.

Let's compare our method with conventional ControlNet Tile upscaling:

Conventional ControlNet Tile:

  • Processes tiles in pixel space, potentially leading to edge artifacts during fusion.
  • Processes each tile sequentially, increasing overall execution time (e.g., 16 tiles x 3 min = 48 min).
  • Pixel space fusion using masks (e.g., Gaussian) can result in visible seams.
  • Fixed or adaptively sized tiles and overlap can vary, causing inconsistencies.

Proposed Method (MoD ControlNet Tile Upscaler):

  • Processes tiles in latent space, enabling smoother fusion and mitigating edge artifacts.
  • Processes all tiles in parallel during denoising, drastically reducing execution time.
  • Latent space fusion with dynamically calculated weights ensures seamless transitions between tiles.
  • Tile size and overlap are dynamically adjusted based on the upscaling scale. For scales below 4x, fixed overlap maintains consistency.

General parameters
Model
Tile Weighting Method
Max. Tile Size
128 8192
2 100
1 20
0.1 1
0.1 2
0 1
Sampler
Examples
Input Image Model Prompt Negative Prompt (Optional) Resolution HDR Effect Inference Steps Denoising Strength ControlNet Strength Gaussian Sigma Sampler Guidance Scale Max. Tile Size Tile Weighting Method

📧 Contact
If you have any questions or suggestions, feel free to send your question to contact@devaiexp.com.