Error upon finishing denoising

@jdldposted Wed, 09 Sep 2026 17:54:18 GMT·0 replies

What happened?
Error upon finishing denoising.

Your system (OS / GPU / RAM / VRAM / etc.)
Win 11/RTX5080laptop/48G/16G

Logs / full error output

Paste the full log output here

[GGUF][llama.cpp CUDA] kernels available.
[Quanto][INT8] Injected int8 kernels ACTIVE (backend=triton).
Loaded plugin: Motion Designer (from motion_designer)

Running on local URL: http://127.0.0.1:42003
To create a public link, set share=True in launch().
Loading Model 'ckpts\MiniMax-H3-Ref2VA_int8_convrot.safetensors' ...
Qwen3-VL-32B-Instruct/qwen3vl_32b_minima(…): 100%|████████████████████████████████████████████████████████████████████████████████████████| 15.7G/15.7G [03:39<00:00, 71.5MB/s]
Loading Text Encoder 'ckpts\Qwen3-VL-32B-Instruct\qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors' ...
NVFP4: kernels available (lightx2v); optimized path will be used when compatible.
Loading MiniMax H3 Video VAE 'ckpts\MiniMax-H3-video_vae_fp16.safetensors'...
************ Memory Management for the GPU Poor (mmgp 3.7.14) by DeepBeepMeep ************
Switching to partial pinning since full requirements for pinned models is 32446.9 MB while estimated available reservable RAM is 19525.3 MB. You may increase the value of parameter 'perc_reserved_mem_max' to a value higher than 0.40 to force full pinnning.
Partial pinning of data of 'transformer' to reserved RAM
Found 200 tied weights for a total of 0.00 MB, last : blocks.49.attn.q_proj.output_scale <-> blocks.49.attn.v_proj.output_scale
The model was partially pinned to reserved RAM: 96 large blocks spread across 19465.77 MB
Hooked to model 'transformer' (MiniMaxH3Model)
Async loading plan for model 'transformer' : base size of 173.37 MB will be preloaded with a 616.07 MB async circular shuttle
Hooked to model 'vae' (ModuleDict)
Hooked to model 'video_encoder' (ModuleDict)
Hooked to model 'audio_vae' (MiniMaxH3AudioVAE)
Hooked to model 'text_encoder' (Qwen3VLTextModel)
Async loading plan for model 'text_encoder' : base size of 742.17 MB will be preloaded with a 261.65 MB async circular shuttle
Hooked to model 'vision_encoder' (Qwen3VLVisionModel)
Hooked to model 'latent_upscaler' (MiniMaxH3LatentUpscaler)
NVFP4: using lightx2v kernel
H3 denoising: 0%| | 0/20 [00:00<?, ?steps/s][MiniMax H3] Sol-Attn enabled with Triton on SM120 (tau=0.95, diag)
[Quanto][INT8] Injected Triton int8 kernels are being used.
[WanGP][Triton] Preparing _rms_norm_rope_kernel (compiling, please wait)...
remark: qk_norm_rope.py:17:0: 98 instructions in function
[WanGP][Triton] Compiled _rms_norm_rope_kernel in 1350 ms.
[WanGP][Triton] Preparing _reduce_quant_k_kernel (compiling, please wait)...
remark: quant.py:29:0: 1236 instructions in function
[WanGP][Triton] Compiled _reduce_quant_k_kernel in 345 ms.
[WanGP][Triton] Preparing _reduce_vc_kernel (compiling, please wait)...
remark: preprocess.py:61:0: 302 instructions in function
[WanGP][Triton] Compiled _reduce_vc_kernel in 141 ms.
[WanGP][Triton] Preparing _forward_int8_ptr (compiling, please wait)...
remark: fwd.py:558:0: 3017 instructions in function
[WanGP][Triton] Compiled _forward_int8_ptr in 1164 ms.
[WanGP][Triton] Preparing _forward_int8_ptr (compiling, please wait)...
remark: fwd.py:558:0: 2287 instructions in function
[WanGP][Triton] Compiled _forward_int8_ptr in 859 ms.
[WanGP][Triton] Preparing _forward_int8_ptr (compiling, please wait)...
remark: fwd.py:558:0: 3185 instructions in function
[WanGP][Triton] Compiled _forward_int8_ptr in 1179 ms.
[WanGP][Triton] Preparing _forward_int8_ptr (compiling, please wait)...
remark: fwd.py:558:0: 2383 instructions in function
[WanGP][Triton] Compiled _forward_int8_ptr in 912 ms.
H3 denoising: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [14:02<00:00, 42.15s/steps]
Hooked to model 'model' (DepthAnythingV2)
Traceback (most recent call last):
File "D:\pinokio\api\wan.git\app\postprocessing\dlss5\runtime.py", line 491, in neural_render
depth = depth_guides.process(render_frame, reset) if depth_guides is not None else None
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\pinokio\api\wan.git\app\postprocessing\dlss5\runtime.py", line 335, in process
disparity = self.estimator.model.infer_image(bgr, input_size=min(self.inference_width, self.inference_height))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\pinokio\api\wan.git\app\venv\Lib\site-packages\torch\utils_contextlib.py", line 124, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "D:\pinokio\api\wan.git\app\preprocessing\depth_anything_v2\dpt.py", line 56, in infer_image
return depth.cpu().numpy()
^^^^^^^^^^^
File "D:\pinokio\api\wan.git\app\venv\Lib\site-packages\torch\utils_device.py", line 109, in torch_function
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^

torch.AcceleratorError: CUDA error: an illegal memory access was encountered
Search for cudaErrorIllegalAddress' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information. CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Compile with TORCH_USE_CUDA_DSA` to enable device-side assertions.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\pinokio\api\wan.git\app\postprocessing\spatial_upsamplers.py", line 425, in upscale_postprocessing
return handler.upscale(sample, spatial_upsampling, loaded_model_context=borrowed_context, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\pinokio\api\wan.git\app\postprocessing\dlss5\spatial_upsampler.py", line 84, in upscale
output = neural_render(sample, self.split_value(spatial_upsampling)[1], still_image=still_image, depth_resolution=config["depth_resolution"], motion_vector=config["motion_vector"], intensity=intensity, abort_callback=abort_callback, progress_callback=progress_callback)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\pinokio\api\wan.git\app\postprocessing\dlss5\runtime.py", line 509, in neural_render
offload_registry.unload_vram(list(_GUIDE_OFFLOADS))
File "D:\pinokio\api\wan.git\app\shared\utils\offload_registry.py", line 49, in unload_vram
offloadobj.unload_all()
File "D:\pinokio\api\wan.git\app\venv\Lib\site-packages\mmgp\offload.py", line 2847, in unload_all
torch.cuda.empty_cache()
File "D:\pinokio\api\wan.git\app\venv\Lib\site-packages\torch\cuda\memory.py", line 280, in empty_cache
torch._C._cuda_emptyCache()
torch.AcceleratorError: CUDA error: an illegal memory access was encountered
Search for cudaErrorIllegalAddress' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information. CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Compile with TORCH_USE_CUDA_DSA` to enable device-side assertions.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\pinokio\api\wan.git\app\wgp.py", line 8660, in queue_worker_func
success = generate_media(task, send_cmd, plugin_data=plugin_data, **filtered_params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\pinokio\api\wan.git\app\wgp.py", line 8248, in generate_media
sample = perform_spatial_upsampling(sample, spatial_upsampling, seed=seed, flashvsr_continue_cache=flashvsr_continue_cache, return_flashvsr_continue_cache=return_flashvsr_continue_cache, vae_tile_size=VAE_tile_size, fps=output_fps, prompt=prompt, negative_prompt=negative_prompt, audio_waveform=upsampler_audio, audio_sample_rate=upsampler_audio_sample_rate, reference_images=upsampler_reference_images, image_refs_relative_size=image_refs_relative_size, spatial_upsampler_prompt=spatial_upsampler_prompt, spatial_upsampler_reference_images=spatial_upsampler_reference_images, spatial_upsampler_face_count=spatial_upsampler_face_count, spatial_upsampler_parameters=spatial_upsampler_parameters, abort_callback=lambda: gen.get("abort", False), progress_callback=upsampler_progress)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\pinokio\api\wan.git\app\wgp.py", line 5828, in perform_spatial_upsampling
sample, upsampler_cache = upsampler_api.upscale_postprocessing(edit_upsampler, sample, spatial_upsampling, main_offloadobj=offloadobj, loaded_model_context=get_loaded_model_context(), seed=seed, continue_cache=flashvsr_continue_cache, return_continue_cache=return_flashvsr_continue_cache, vae_tile_size=vae_tile_size, process_files=process_files_def, vae_config=vae_config, init_pipe=init_pipe, profile=profile, still_image=still_image, fps=fps, frame_offset=frame_offset, prompt=prompt, negative_prompt=negative_prompt, audio_waveform=audio_waveform, audio_sample_rate=audio_sample_rate, source_audio_path=source_audio_path, reference_images=reference_images, image_refs_relative_size=image_refs_relative_size, abort_callback=abort_callback, progress_callback=progress_callback, **parameter_values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "D:\pinokio\api\wan.git\app\postprocessing\spatial_upsamplers.py", line 433, in upscale_postprocessing
_release_upsampler_handler(handler)
File "D:\pinokio\api\wan.git\app\postprocessing\spatial_upsamplers.py", line 253, in _release_upsampler_handler
handler.release_vram()
File "D:\pinokio\api\wan.git\app\postprocessing\dlss5\spatial_upsampler.py", line 88, in release_vram
release_flow_model()
File "D:\pinokio\api\wan.git\app\postprocessing\dlss5\runtime.py", line 88, in release_flow_model
offloadobj.release()
File "D:\pinokio\api\wan.git\app\venv\Lib\site-packages\mmgp\offload.py", line 3459, in release
self.unload_all()
File "D:\pinokio\api\wan.git\app\venv\Lib\site-packages\mmgp\offload.py", line 2847, in unload_all
torch.cuda.empty_cache()
File "D:\pinokio\api\wan.git\app\venv\Lib\site-packages\torch\cuda\memory.py", line 280, in empty_cache
torch._C._cuda_emptyCache()
torch.AcceleratorError: CUDA error: an illegal memory access was encountered
Search for cudaErrorIllegalAddress' in https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__TYPES.html for more information. CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1 Compile with TORCH_USE_CUDA_DSA` to enable device-side assertions.

Replies (0)

Up to 10 files, 25MB each. Images are optimized; GIFs -> MP4; videos 720p (max 120s).