What you actually need to run FLUX, Wan, and SDXL locally

The number in the model's name is one of four models you have to load

FLUX.1-dev is described everywhere as a 12B model. The HuggingFace repo declares exactly 11,901,408,320 parameters. But that count covers the transformer directory alone, and you cannot generate an image with only the transformer. The files you actually need total 33.74 GB across four components:

(Sizes read from the HuggingFace blobs API on 2026-07-28.)

Cloning the whole repo actually pulls 57.89 GB, because the root also carries flux1-dev.safetensors (23,802,932,552 bytes) — the same transformer again in Black Forest Labs' original single-file layout — plus ae.safetensors, a 335,304,388-byte fp32 copy of the VAE. You need one layout or the other, never both. If you downloaded 58 GB, delete the half you aren't using.

The real parameter count of the runnable pipeline is about 16.87B, not 11.9B. Every site that multiplies 12B by 2 bytes and tells you FLUX needs 24 GB has skipped 40% of the model.

Put the text encoder on the CPU. This is the whole game.

T5-XXL is 9.52 GB of the 33.74 GB, and it runs once per prompt, not once per denoising step. A 20-step generation calls the denoiser 20 times and T5 exactly once. Keeping 9.5 GB of weights resident on the GPU for one forward pass at the start of the job is the worst trade in the stack.

In ComfyUI, CLIPLoader and DualCLIPLoader both take an advanced device input with the options default and cpu. Setting it to cpu sets load_device and offload_device to CPU (nodes.py, load_clip), so the encoder never touches VRAM. The --lowvram launch flag does the same thing globally — its help text is literally "makes the text encoders run on the CPU" — though note that as of current master it is a no-op when DynamicVRAM is active, and DynamicVRAM is on by default unless you pass --disable-dynamic-vram, --highvram, --gpu-only, --novram, or --cpu.

This matters even more for video. Wan-AI/Wan2.1-T2V-1.3B ships a 1.42B denoiser (5,676,070,424 bytes, stored in fp32, so 2.84 GB at bf16) alongside models_t5_umt5-xxl-enc-bf16.pth at 11,361,920,418 bytes. The text encoder is four times the size of the model you came for. Nobody's "1.3B video model runs on 8GB" post survives contact with that file unless the encoder is on the CPU or quantized.

GGUF quantizes the denoiser, and only the denoiser

city96's repos are the de facto standard here. city96/FLUX.1-dev-gguf contains only the transformer:

Bits per weight are computed against the declared 11,901,408,320 parameters, and they are all above nominal for the same reason they are in LLM quants: normalization and modulation tensors stay in F32, so a "4-bit" file is 4.58 bits in practice.

The text encoder needs its own separate quant. city96/t5-v1_1-xxl-encoder-gguf has Q8_0 at 5,061,584,064 bytes and Q4_K_M at 2,896,123,072 bytes, against 9,526,060,224 for f16. For Wan, city96/umt5-xxl-encoder-gguf drops the 11,368,687,456-byte F16 to 3,655,145,312 at Q4_K_M. If you loaded a Q4 denoiser and are still OOMing, check whether you left the encoder at full precision — that is the usual answer.

SDXL cannot be quantized this way at all. The ComfyUI-GGUF README states plainly that "quantization wasn't feasible for regular UNET models (conv2d)"; the format works on DiT-style transformers, not convolutional UNets. SDXL doesn't need it anyway. Its entire fp16 pipeline is 6.94 GB: UNet 5,135,149,760, OpenCLIP-bigG 1,389,382,176, CLIP-L 246,144,152, VAE 167,335,342. If you have an 8 GB card and want images today, this is still the model that fits without arguing with you.

What actually moves peak VRAM: sequence length, not parameter count

FLUX's VAE downsamples 8x spatially into 16 channels, then the transformer packs each 2x2 latent patch into one token. So 1024x1024 becomes a 128x128 latent, then 64x64 = 4096 image tokens, joined with a fixed 512-token T5 sequence (diffusers raises if you ask for more than 512). Total sequence: 4608 across 19 dual-stream and 38 single-stream blocks, 24 heads of dim 128.

Double the resolution and the sequence quadruples. 2048x2048 gives 16384 image tokens, 16896 total. If your attention backend materializes the score matrix, that array alone is 16896² x 24 heads x 2 bytes = 12.76 GiB, against 0.95 GiB at 1024x1024. Four times the pixels, thirteen times the attention memory. With PyTorch SDPA, xformers, or flash-attention this term becomes linear in sequence length and mostly disappears — which is why "same resolution, same model, wildly different VRAM" reports are usually an attention-backend difference and nothing else. Check that first.

Video multiplies the same axis by frames. Wan 2.1's VAE compresses 8x spatially and 4x temporally into 16 channels, and its DiT uses patch size (1,2,2). An 832x480 clip at 81 frames becomes a 21x60x104 latent, patchified to 32,760 tokens — eight times a 1024x1024 FLUX image, so roughly sixty-four times the attention work. Frame count is not a linear cost. Wan 2.2's 5B TI2V model changes the geometry instead: 16x spatial and 4x temporal compression into 48 channels, which is how it targets a smaller budget than the 14B line.

Then there's the decode. ComfyUI estimates VAE decode memory as 2178 * H_latent * W_latent * 64 * dtype_size for AutoencoderKL models, which at 1024x1024 is 4.25 GiB and at 2048x2048 is 17.0 GiB. For Wan 2.1 it is 7000 * H_latent * W_latent * 64 * dtype_size once you exceed 4 latent frames — 5.21 GiB for that 832x480x81 clip. These are ComfyUI's own upper-bound heuristics, not measurements (and it multiplies them by 2.73 on AMD), but the shape is right: the decode is frequently the peak, not the denoising. ComfyUI catches the OOM and retries with tiled decoding automatically, logging "Ran out of memory when regular VAE decoding, retrying with tiled VAE decoding." If you see that line, put a VAEDecodeTiled node in the graph deliberately — default tile_size 512, overlap 64, temporal_size 64.

Because ComfyUI unloads models to CPU RAM after use by default, your peak is roughly the max of the encode, denoise, and decode stages rather than their sum. That is what makes a Q4_K_S FLUX (6.34 GiB) plus VAE plausible on a 12 GB card with the encoder on CPU. We have not measured it on a 12 GB card, so treat that as arithmetic, not a benchmark.

Why there are no it/s numbers on this page

Per-GPU throughput and measured peak VRAM for these models are essentially unpublished. There is no llama.cpp-bench equivalent for diffusion, and the blog posts that do exist disagree by three to five times on identical hardware — different attention backends, different schedulers, different step counts, sage/flash on or off, and almost never a stated methodology.

We can tell you exactly how many bytes of weights must be resident, because we read the file sizes. We can tell you how the sequence length scales, because it falls out of the VAE compression ratio and the patch size. We cannot tell you how many seconds a step takes on your 3090, and we are not going to invent it. If you want throughput, run your own A/B with one variable changed at a time and post the config alongside the number.

Wan 2.2's A14B is the one place where capacity planning genuinely breaks the usual rules: it ships two 14B expert denoisers (high-noise and low-noise), each 57,154,077,376 bytes in fp32, and a single generation uses both — early steps on one, late steps on the other. Total download is 126 GB. Resident-at-once is one expert, if your runner swaps them. That gap between "what you must store" and "what must be in VRAM simultaneously" is the same distinction that matters for MoE language models, and it is the number most guides get wrong.