Another world model, another High VRAM requirement. Requires at least 24GB VRAM. Really wish people tried their best to make them run on consumer machines.
https://x.com/TencentHunyuan/status/2113676827114053892?s=20
Yes 24GB is better than supercomputers but still not enough. Even a 16GB VRAM support would be great. The README doesn’t give an exact VRAM number. Here’s what it does say:
- Minimum (single GPU): one RTX 4090 (24 GB), using the low-memory config (single_gpu_low_mem_inference.yaml). That setup uses the INT8 DiT weights and KV cache, the FP8 PixelUMM decoder, a sliding-window history and CPU offload of the text encoder, DiT and VAE.
- The offloading needs plenty of system RAM to hold the models, and it slows generation down.
- Peak VRAM still changes with resolution and segment length.
- Full-precision inference and the interactive demo: these use the FP32 distilled DiT with sequence parallelism. That points to a multi-GPU setup, but the README doesn’t list an exact requirement.
- LoRA training: the reference config runs on 128 GPUs (DP16 × SP8), so it’s datacenter-scale.
In practice, 24 GB is the minimum, and only with the INT8/FP8 checkpoints and offloading. Anything below a 4090 isn’t officially supported.
Maybe there will be a gguf port, but no guarantee that it will be usable yet, and my exact point is, the people who release these models should try their best to optimize BEFORE releasing, instead of waiting for community port.
https://github.com/Tencent-Hunyuan/Hy-Nomas
Another world model, another High VRAM requirement. Requires at least 24GB VRAM. Really wish people tried their best to make them run on consumer machines.
https://x.com/TencentHunyuan/status/2113676827114053892?s=20
Yes 24GB is better than supercomputers but still not enough. Even a 16GB VRAM support would be great. The README doesn’t give an exact VRAM number. Here’s what it does say:
In practice, 24 GB is the minimum, and only with the INT8/FP8 checkpoints and offloading. Anything below a 4090 isn’t officially supported.
Maybe there will be a gguf port, but no guarantee that it will be usable yet, and my exact point is, the people who release these models should try their best to optimize BEFORE releasing, instead of waiting for community port.
https://github.com/Tencent-Hunyuan/Hy-Nomas