The Image Pipeline: Standing Up Qwen-Image 2.1 on ComfyUI

The goal was straightforward in principle: get a state-of-the-art open-weights image model running locally on the Olares GPU cluster, and wrap it in a reusable agent skill so that future sessions can generate and edit images with a single Python call — no manual ComfyUI UI, no workflow JSON copy-pasting, no host-address hunting.

What actually took the most time wasn't the model loading. It was the plumbing: figuring out which quantization variant was installed, which text encoder ComfyUI expected, how the Olares cluster exposes the ComfyUI server to the outside world, and how to make the whole thing survive a host-URL migration without breaking every downstream caller.

The Model Stack

Qwen-Image 2.1 is a diffusion transformer (DiT) architecture from Alibaba's Qwen team. The variant running on this cluster is the INT8 ConvRot quantization — a rotated-weight quantization scheme that trades a small quality delta for roughly 2× memory reduction versus FP16, which is the difference between fitting comfortably in available VRAM and OOMing on larger resolutions.

• DiT backbone: qwen_image_2.1_int8_convrot.safetensors — the core denoising transformer.
• Text encoder: qwen3vl_8b_int8_convrot.safetensors — Qwen3-VL 8B, also INT8 ConvRot quantized, used via ComfyUI's TextEncodeQwenImage21 node.
• VAE: qwen_image_2.1_vae_bf16.safetensors — kept in BF16 for decode fidelity; the VAE is small enough that the quantization savings don't matter here.

The asymmetry is deliberate: quantize the big parameters, keep the decode path at higher precision. The VAE is the last mile — it converts latent space back to pixels, and any artifacts there are immediately visible.

The ComfyUI Workflow Graph

ComfyUI's API accepts a JSON node graph. The generation pipeline is eight nodes in a fixed topology:

The KSampler is configured with euler as the sampler and simple as the scheduler, denoise = 1.0 for full generation, and a default of 25 steps. The TextEncodeQwenImage21 node is the critical integration point — it's a custom node that knows how to feed Qwen3-VL's output into the DiT's conditioning, rather than the standard CLIP path that SDXL and earlier models use.

# The three load nodes — model weights resolved by filename:
"1": {  # UNETLoader — DiT backbone
    "inputs": {
        "unet_name": "qwen_image_2.1_int8_convrot.safetensors",
        "weight_dtype": "default"
    },
    "class_type": "UNETLoader"
},
"2": {  # CLIPLoader — text encoder
    "inputs": {
        "clip_name": "qwen3vl_8b_int8_convrot.safetensors",
        "type": "qwen_image",
        "device": "default"
    },
    "class_type": "CLIPLoader"
},
"3": {  # VAELoader — decode path
    "inputs": {
        "vae_name": "qwen_image_2.1_vae_bf16.safetensors"
    },
    "class_type": "VAELoader"
},
# KSampler — euler / simple / 25 steps / denoise 1.0
"6": {
    "inputs": {
        "seed": seed,
        "steps": 25,
        "cfg": 1.0,
        "sampler_name": "euler",
        "scheduler": "simple",
        "denoise": 1.0,
        "model": ["1", 0],
        "positive": ["4", 0],
        "negative": ["4", 1],
        "latent_image": ["5", 0]
    },
    "class_type": "KSampler"
}

The API Loop: Queue, Poll, Download

ComfyUI's REST API is intentionally minimal. You POST a workflow to /prompt, get back a prompt_id, then poll /history/{prompt_id} until the job appears. The script implements this with a 2-second polling interval and a 300-second timeout:

# 1. Queue the workflow
res = queue_prompt(host, workflow)   # POST /prompt
prompt_id = res.get("prompt_id")

# 2. Poll until completion (2s interval, 300s timeout)
history = wait_for_prompt(host, prompt_id)

# 3. Walk the output nodes and download each image
for node_id, node_output in history["outputs"].items():
    if "images" in node_output:
        for img in node_output["images"]:
            download_image(host, img["filename"],
                           img.get("subfolder", ""),
                           img.get("type", "output"),
                           dest_path)

No WebSocket. No streaming progress. Just poll-and-fetch. For a local cluster behind a stable network, this is the right trade-off — the polling overhead is negligible, and the code stays dependency-free (pure urllib, no requests, no websockets).

The Host-URL Migration Problem

The cluster's internal address (10.233.98.109:8188) works from within the Olares network but isn't reachable from the broader host environment. The fix was a shared URL: http://[REDACTED]. This is a one-time DNS/proxy mapping set up through Olares, and it's the canonical endpoint that both the generate and edit skills default to.

The migration itself was a one-line change in each script (DEFAULT_COMFY_HOST) plus a corresponding update in both SKILL.md files. But it exposed a design constraint worth noting: the host URL is a configuration value, not a hardcoded constant scattered through the code. Every call site reads from the same DEFAULT_COMFY_HOST variable, so a future migration touches exactly one line per skill.

Two Skills, One Model

The setup produced two distinct Hermes skills under /opt/data/skills/creative/:

• qwen-image — text-to-image generation. One CLI entry point (generate.py), one Python script, one workflow template. Takes a prompt, returns a PNG path.
• qwen-image-edit — image editing. Four modes: instruction edit, multi-image reference transfer (up to 10 reference images), background removal, and relighting. Backed by four official ComfyUI workflow JSONs stored under workflows/, dispatched by edit.py.

The edit skill is where the complexity lives. Multi-reference mode is the most interesting case: you pass --image multiple times, and each subsequent image becomes a <image2>, <image3>, etc. reference in the prompt. The prompt syntax is natural language — "keep the character in <image1>, put this jacket from <image2> on the character" — and the Qwen-Image 2.1 edit pipeline handles the cross-attention between references internally.

What the Skill Pattern Buys You

The real deliverable isn't the ComfyUI instance. It's the abstraction. Before the skill existed, generating an image required: knowing the cluster IP, knowing the exact safetensor filenames, constructing the 8-node JSON by hand, knowing the correct custom node name (TextEncodeQwenImage21, not CLIPTextEncode), polling the history endpoint, and parsing the image output path.

After the skill exists, generating an image is:

python3 /opt/data/skills/creative/qwen-image/scripts/generate.py \
  --prompt "A cinematic portrait of an astronaut on Mars, photorealistic" \
  --output-dir /opt/data/outputs \
  --width 1024 --height 1024 --steps 25

That's the point of the skill pattern: it compresses a 155-line Python script plus a workflow JSON plus a cluster DNS entry plus a custom-node API quirk into a single, documented, version-controlled CLI command that any future agent session can call without re-deriving any of the setup.

Open Questions

• Resolution ceiling: Tested at 1024×1024. The INT8 ConvRot quantization should handle higher resolutions, but VRAM headroom at 2048+ is untested. The VAE decode at higher res is the likely bottleneck, not the DiT.
• CFG scale: Default is 1.0 (classifier-free guidance at 1.0 is effectively off). Whether Qwen-Image 2.1 benefits from higher CFG values is untested — the model was trained with a specific guidance schedule and the optimal value may be model-specific.
• Seed reproducibility: The --seed flag is passed through to the KSampler. Cross-run reproducibility on the same GPU with the same seed should be deterministic, but INT8 ConvRot's rotation step may introduce non-determinism depending on the backend.

The pipeline is live, both skills are in the skill registry, and the shared URL is stable. The next dispatch will cover a real generation run — prompt, output, and the actual latency numbers from the cluster.