Purpose

StreamDiffusion transforms incoming images according to a prompt and a selected StreamDiffusionTD configuration. That configuration determines the model, resolution and support for ControlNet or IP-Adapter inputs.

Use

Place StreamDiffusion under Generate in the Image view. It requires a compatible StreamDiffusionTD installation, its Python environment, model assets and a working configuration.

On Voro’s Runtimes page, set StreamDiffusionTD Root and StreamDiffusion Python for that installation. Select the configuration in Config, connect a picture to in, and enter a Prompt.

Begin with a fixed Seed and feedback disabled so you can judge changes to the prompt.

Inputs and outputs

• in is the image to transform.
• control supplies a ControlNet image when the configuration supports it and Control Image is enabled.
• style supplies an IP-Adapter image when configured and Style Image is on.
• out is the generated picture.

Controls

• Config selects the pipeline configuration.
• Prompt and Prompt Blend steer the subject. Negative Prompt requires a pipeline using full classifier-free guidance.
• Seed Mode selects Fixed, Random or Blend. Seed Blend and Seed Interp control blended seeds.
• T-Index List selects diffusion timesteps within the configured pipeline.
• Feedback Loop and Feedback Strength feed generated images back into processing. Zoom, pan, rotation and Feedback Grade modify that loop.
• Engine Builder exposes model, resolution and optional-input choices. Build TRT Engine starts the build for those choices.

Notes

Build settings must match the model and optional inputs you intend to use. Building an engine can take time; it is separate from changing a live prompt. The builder uses the StreamDiffusionTD installation and model assets already configured on the machine.

Ports

Inputs

• in image required
  RGBA
• control image
  RGBA
• style image
  RGBA

Outputs

• out image
  RGBA

Parameters

Model

Config (Configpath) Str op('stream_diffusion').par.Configpath

Default:

td_config.yaml

Builder

Engine Builder (Builder) Toggle op('stream_diffusion').par.Builder

Default:

Off

Options:

Off, On

Build TRT Engine (Build) Pulse op('stream_diffusion').par.Build

Default:

None

Base Model (Buildmodel) Str op('stream_diffusion').par.Buildmodel

Default:

stabilityai/sdxl-turbo

Resolution (H x W) (Buildresolution) Str op('stream_diffusion').par.Buildresolution

Default:

512x512

Shape Profile (Buildshapemode) Menu op('stream_diffusion').par.Buildshapemode

Default:

static

Options:

static, dynamic

T-Index List (Buildtindex) Str op('stream_diffusion').par.Buildtindex

Default:

"" (Empty String)

ControlNets (Buildcontrolnets) Str op('stream_diffusion').par.Buildcontrolnets

Default:

"" (Empty String)

IP-Adapter (Buildipadapter) Menu op('stream_diffusion').par.Buildipadapter

Default:

none

Options:

none, standard, faceid

IP-Adapter Model (Buildipamodel) Str op('stream_diffusion').par.Buildipamodel

Default:

"" (Empty String)

IP-Adapter Tokens (Buildipatokens) Int op('stream_diffusion').par.Buildipatokens

Default:

4

Range:

1 to 64

IP-Adapter Image Encoder (Buildipaencoder) Str op('stream_diffusion').par.Buildipaencoder

Default:

"" (Empty String)

Frame Batch (Buildbatch) Int op('stream_diffusion').par.Buildbatch

Default:

1

Range:

1 to 4

Config Output Folder (Buildfolder) Str op('stream_diffusion').par.Buildfolder

Default:

"" (Empty String)

Prompt

Prompt (Prompt) Str op('stream_diffusion').par.Prompt

Default:

"" (Empty String)

Prompt Blend (Promptblend) Str op('stream_diffusion').par.Promptblend

Default:

"" (Empty String)

Prompt Interp (Promptinterp) Menu op('stream_diffusion').par.Promptinterp

Default:

slerp

Options:

slerp, linear, cosine_weighted

Negative Prompt (needs full CFG) (Negprompt) Str op('stream_diffusion').par.Negprompt

Default:

"" (Empty String)

Generate

T-Index List (Tindex) Str op('stream_diffusion').par.Tindex

Default:

"" (Empty String)

Delta (Delta) Float op('stream_diffusion').par.Delta

Default:

1

Range:

0 to 2

Seed Mode (Seedmode) Menu op('stream_diffusion').par.Seedmode

Default:

fixed

Options:

fixed, random, blend

Seed (Seed) Int op('stream_diffusion').par.Seed

Default:

-1

Range:

-1 to 2147483600

Seed Blend (Seedblend) Str op('stream_diffusion').par.Seedblend

Default:

"" (Empty String)

Seed Interp (Seedinterp) Menu op('stream_diffusion').par.Seedinterp

Default:

linear

Options:

linear, slerp

Reset (Reset) Pulse op('stream_diffusion').par.Reset

Default:

None

Feedback

Feedback Loop (Feedback) Toggle op('stream_diffusion').par.Feedback

Default:

Off

Options:

Off, On

Feedback Strength (Feedbackstrength) Float op('stream_diffusion').par.Feedbackstrength

Default:

0

Range:

0 to 1

Zoom (Zoom) Float op('stream_diffusion').par.Zoom

Default:

1

Range:

0.75 to 1.5

Pan X (Panx) Float op('stream_diffusion').par.Panx

Default:

0

Range:

-0.15 to 0.15

Pan Y (Pany) Float op('stream_diffusion').par.Pany

Default:

0

Range:

-0.15 to 0.15

Rotation (Rotation) Float op('stream_diffusion').par.Rotation

Default:

0

Range:

-10 to 10

Border Mode (Bordermode) Menu op('stream_diffusion').par.Bordermode

Default:

zeros

Options:

zeros, border, reflection

Feedback Grade (Grade) Toggle op('stream_diffusion').par.Grade

Default:

Off

Options:

Off, On

Grade Strength (Gradestrength) Float op('stream_diffusion').par.Gradestrength

Default:

0

Range:

0 to 1

Brightness (Gradebrightness) Float op('stream_diffusion').par.Gradebrightness

Default:

0

Range:

-1 to 1

Contrast (Gradecontrast) Float op('stream_diffusion').par.Gradecontrast

Default:

1

Range:

0 to 2

Saturation (Gradesaturation) Float op('stream_diffusion').par.Gradesaturation

Default:

1

Range:

0 to 2

Gamma (Gradegamma) Float op('stream_diffusion').par.Gradegamma

Default:

1

Range:

0 to 2

Black Level (Gradeblacklevel) Float op('stream_diffusion').par.Gradeblacklevel

Default:

0

Range:

0 to 1

Hue (Gradehue) Float op('stream_diffusion').par.Gradehue

Default:

0

Range:

-180 to 180

Temperature (Gradetemperature) Float op('stream_diffusion').par.Gradetemperature

Default:

0

Range:

-1 to 1

Invert (Gradeinvert) Float op('stream_diffusion').par.Gradeinvert

Default:

0

Range:

0 to 1

Control

Control Image (Usecontrol) Menu op('stream_diffusion').par.Usecontrol

Default:

off

Options:

off, control-port

CN 1 Weight (Cnweight1) Float op('stream_diffusion').par.Cnweight1

Default:

-1

Range:

-1 to 2

CN 2 Weight (Cnweight2) Float op('stream_diffusion').par.Cnweight2

Default:

-1

Range:

-1 to 2

CN 3 Weight (Cnweight3) Float op('stream_diffusion').par.Cnweight3

Default:

-1

Range:

-1 to 2

Style Image (Usestyle) Toggle op('stream_diffusion').par.Usestyle

Default:

Off

Options:

Off, On

IP-Adapter Scale (Ipascale) Float op('stream_diffusion').par.Ipascale

Default:

-1

Range:

-1 to 3