1
stuck in upgrade loop with latest 1.8+ update
What happened?
Getting stuck in performance runtime upgrade loop
Steps to reproduce
- install fresh 1.8.5 or update to 1.8.5 (tried both, same issue
Your system (OS / GPU / RAM / VRAM / etc.)
Linux Mint 22.2 / Intel 24-core / rtx 4070 / 64GB RAM / 12GB VRAM / 2TB M.2 SSD
Logs / full error output
████
██ Starting Shell 8c75df96-bdb5-45a1-8caf-6bccf6c72dbb
<<PINOKIO_SHELL>>eval "$(conda shell.bash hook)" ; conda deactivate ; conda deactivate ; conda deactivate ; conda activate base && git pull
Already up to date.
(base) <<PINOKIO_SHELL>>
██ Terminated Shell 8c75df96-bdb5-45a1-8caf-6bccf6c72dbb
████
████
██ Starting Shell cc907bc4-cce7-4a75-b86e-15470bc9ec87
<<PINOKIO_SHELL>>eval "$(conda shell.bash hook)" ; conda deactivate ; conda deactivate ; conda deactivate ; conda activate base ; source /home/mlabs/pinokio/api/Maestro.git/app/env-sol/bin/activate /home/mlabs/pinokio/api/Maestro.git/app/env-sol && uv pip install -r requirements.txt
Using Python 3.11.14 environment at: env-sol
Resolved 296 packages in 2.67s
Uninstalled 1 package in 2ms
Installed 1 package in 8ms
~ decord==0.6.0
(env-sol) (base) <<PINOKIO_SHELL>>
██ Terminated Shell cc907bc4-cce7-4a75-b86e-15470bc9ec87
████
Installing Maestro's H3 Sol Engine / CUDA 13 acceleration runtime...
████
██ Starting Shell f324841b-68b1-41e2-9cba-d25d579a2635
<<PINOKIO_SHELL>>eval "$(conda shell.bash hook)" ; conda deactivate ; conda deactivate ; conda deactivate ; conda activate base ; source /home/mlabs/pinokio/api/Maestro.git/app/env-sol/bin/activate /home/mlabs/pinokio/api/Maestro.git/app/env-sol && uv pip install torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 --index-url https://download.pytorch.org/whl/cu130 --force-reinstall --no-deps
Using Python 3.11.14 environment at: env-sol
Resolved 3 packages in 140ms
Prepared 3 packages in 0.40ms
Uninstalled 3 packages in 86ms
Installed 3 packages in 73ms
~ torch==2.10.0+cu130
~ torchaudio==2.10.0+cu130
~ torchvision==0.25.0+cu130
(env-sol) (base) <<PINOKIO_SHELL>>
██ Terminated Shell f324841b-68b1-41e2-9cba-d25d579a2635
████
████
██ Starting Shell 8273c41d-c246-496f-a390-836aae94246d
<<PINOKIO_SHELL>>eval "$(conda shell.bash hook)" ; conda deactivate ; conda deactivate ; conda deactivate ; conda activate base ; source /home/mlabs/pinokio/api/Maestro.git/app/env-sol/bin/activate /home/mlabs/pinokio/api/Maestro.git/app/env-sol && uv pip install xformers==0.0.35 --index-url https://download.pytorch.org/whl/cu130 --force-reinstall --no-deps
Using Python 3.11.14 environment at: env-sol
Resolved 1 package in 49ms
Prepared 1 package in 0.65ms
Uninstalled 1 package in 4ms
Installed 1 package in 11ms
~ xformers==0.0.35
(env-sol) (base) <<PINOKIO_SHELL>>
██ Terminated Shell 8273c41d-c246-496f-a390-836aae94246d
████
████
██ Starting Shell 260f1ef0-440d-41d5-afce-02dadea880bb
<<PINOKIO_SHELL>>eval "$(conda shell.bash hook)" ; conda deactivate ; conda deactivate ; conda deactivate ; conda activate base ; source /home/mlabs/pinokio/api/Maestro.git/app/env-sol/bin/activate /home/mlabs/pinokio/api/Maestro.git/app/env-sol && uv pip install 'triton>=3.6,<3.7' --force-reinstall
Using Python 3.11.14 environment at: env-sol
Resolved 1 package in 34ms
Prepared 1 package in 0.22ms
Uninstalled 1 package in 2ms
Installed 1 package in 7ms
~ triton==3.6.0
(env-sol) (base) <<PINOKIO_SHELL>>
██ Terminated Shell 260f1ef0-440d-41d5-afce-02dadea880bb
████
████
██ Starting Shell 451da729-4145-46f3-9832-3918de1f63f5
<<PINOKIO_SHELL>>eval "$(conda shell.bash hook)" ; conda deactivate ; conda deactivate ; conda deactivate ; conda activate base ; source /home/mlabs/pinokio/api/Maestro.git/app/env-sol/bin/activate /home/mlabs/pinokio/api/Maestro.git/app/env-sol && uv pip install 'setuptools<=75.8.2' ninja wheel --force-reinstall
Using Python 3.11.14 environment at: env-sol
Resolved 4 packages in 39ms
Prepared 4 packages in 0.18ms
Uninstalled 4 packages in 7ms
Installed 4 packages in 12ms
~ ninja==1.13.0
~ packaging==26.3
~ setuptools==75.8.2
~ wheel==0.48.0
(env-sol) (base) <<PINOKIO_SHELL>>
██ Terminated Shell 451da729-4145-46f3-9832-3918de1f63f5
████
████
██ Starting Shell 7bcc20d3-59a8-45d3-9d00-5dee34f1a3ff
<<PINOKIO_SHELL>>eval "$(conda shell.bash hook)" ; conda deactivate ; conda deactivate ; conda deactivate ; conda activate base ; source /home/mlabs/pinokio/api/Maestro.git/app/env-sol/bin/activate /home/mlabs/pinokio/api/Maestro.git/app/env-sol && uv pip install --no-build-isolation git+https://github.com/thu-ml/SageAttention.git
Using Python 3.11.14 environment at: env-sol
Resolved 1 package in 434ms
× Failed to build `sageattention @ git+https://github.com/thu-ml/SageAttention.git@d1a57a546c3d395b1ffcbeecc66d81db76f3b4b5`
├─▶ The build backend returned an error
╰─▶ Call to `setuptools.build_meta.build_wheel` failed (exit status: 1)
[stdout]
Target compute capabilities: {'8.9'}
running bdist_wheel
running build
running build_py
copying sageattention/__init__.py -> build/lib.linux-x86_64-cpython-311/sageattention
copying sageattention/sm80_compile.py -> build/lib.linux-x86_64-cpython-311/sageattention
copying sageattention/sm90_compile.py -> build/lib.linux-x86_64-cpython-311/sageattention
copying sageattention/core.py -> build/lib.linux-x86_64-cpython-311/sageattention
copying sageattention/quant.py -> build/lib.linux-x86_64-cpython-311/sageattention
copying sageattention/fa3_wrapper.py -> build/lib.linux-x86_64-cpython-311/sageattention
copying sageattention/sm89_compile.py -> build/lib.linux-x86_64-cpython-311/sageattention
copying sageattention/triton/__init__.py -> build/lib.linux-x86_64-cpython-311/sageattention/triton
copying sageattention/triton/quant_per_block_varlen.py -> build/lib.linux-x86_64-cpython-311/sageattention/triton
copying sageattention/triton/attn_qk_int8_block_varlen.py -> build/lib.linux-x86_64-cpython-311/sageattention/triton
copying sageattention/triton/attn_qk_int8_per_block_causal_varlen.py -> build/lib.linux-x86_64-cpython-311/sageattention/triton
copying sageattention/triton/quant_per_thread.py -> build/lib.linux-x86_64-cpython-311/sageattention/triton
copying sageattention/triton/attn_qk_int8_per_block_causal.py -> build/lib.linux-x86_64-cpython-311/sageattention/triton
copying sageattention/triton/attn_qk_int8_per_block.py -> build/lib.linux-x86_64-cpython-311/sageattention/triton
copying sageattention/triton/quant_per_block.py -> build/lib.linux-x86_64-cpython-311/sageattention/triton
running build_ext
[stderr]
nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
nvcc warning : incompatible redefinition for option 'compiler-bindir', the last value of this option was used
Traceback (most recent call last):
File "<string>", line 11, in <module>
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/build_meta.py", line 435, in build_wheel
return _build(['bdist_wheel'])
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/build_meta.py", line 426, in _build
return self._build_with_temp_dir(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/build_meta.py", line 407, in _build_with_temp_dir
self.run_setup()
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/build_meta.py", line 320, in run_setup
exec(code, locals())
File "<string>", line 272, in <module>
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/__init__.py", line 117, in setup
return distutils.core.setup(**attrs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 186, in setup
return run_commands(dist)
^^^^^^^^^^^^^^^^^^
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 202, in run_commands
██ Detached from Shell 7bcc20d3-59a8-45d3-9d00-5dee34f1a3ff
dist.run_commands()
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 983, in run_commands
self.run_command(cmd)
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/dist.py", line 999, in run_command
super().run_command(command)
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1002, in run_command
cmd_obj.run()
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/command/bdist_wheel.py", line 369, in run
self.run_command("build")
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 339, in run_command
self.distribution.run_command(command)
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/dist.py", line 999, in run_command
super().run_command(command)
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1002, in run_command
cmd_obj.run()
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 136, in run
self.run_command(cmd_name)
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 339, in run_command
self.distribution.run_command(command)
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/dist.py", line 999, in run_command
super().run_command(command)
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 1002, in run_command
cmd_obj.run()
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/command/build_ext.py", line 99, in run
_build_ext.run(self)
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/setuptools/_distutils/command/build_ext.py", line 365, in run
self.build_extensions()
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 695, in build_extensions
_check_cuda_version(compiler_name, compiler_version)
File "/home/mlabs/pinokio/api/Maestro.git/app/env-sol/lib/python3.11/site-packages/torch/utils/cpp_extension.py", line 524, in _check_cuda_version
raise RuntimeError(CUDA_MISMATCH_MESSAGE, cuda_str_version, torch.version.cuda)
RuntimeError: ('The detected CUDA version (%s) mismatches the version that was used to compilePyTorch (%s). Please make sure to use the same CUDA versions.', '12.8', '13.0')
hint: Build failures usually indicate a problem with the package or the build environment
===================================================
# input.event
[
"Error:"
]
===================================================
(env-sol) (base) <<PINOKIO_SHELL>>
████
██ Starting Shell 3df62037-706e-4521-9269-ac9ddba96c68
██ Terminated Shell 7bcc20d3-59a8-45d3-9d00-5dee34f1a3ff
████
<<PINOKIO_SHELL>>eval "$(conda shell.bash hook)" ; conda deactivate ; conda deactivate ; conda deactivate ; conda activate base ; source /home/mlabs/pinokio/api/Maestro.git/app/env-sol/bin/activate /home/mlabs/pinokio/api/Maestro.git/app/env-sol && python scripts/install_gguf_kernels.py
[GGUF kernels] llamacpp_gguf_cuda already installed - skipping.
(env-sol) (base) <<PINOKIO_SHELL>>
██ Terminated Shell 3df62037-706e-4521-9269-ac9ddba96c68
████
████
██ Starting Shell 95c3dc84-09bb-4334-b72f-4f9e35aa9379
<<PINOKIO_SHELL>>eval "$(conda shell.bash hook)" ; conda deactivate ; conda deactivate ; conda deactivate ; conda activate base && npm install
up to date, audited 199 packages in 478ms
50 packages are looking for funding
run `npm fund` for details
8 vulnerabilities (2 low, 1 moderate, 5 high)
To address all issues, run:
npm audit fix
Run `npm audit` for details.
npm warn allow-scripts 1 package has install scripts not yet covered by allowScripts:
npm warn allow-scripts esbuild@0.27.3 (install: (install scripts present))
npm warn allow-scripts
npm warn allow-scripts Run `npm approve-scripts --allow-scripts-pending` to review, or `npm approve-scripts <pkg>` to allow.
(base) <<PINOKIO_SHELL>>
██ Terminated Shell 95c3dc84-09bb-4334-b72f-4f9e35aa9379
████
████
██ Starting Shell c9ff09d0-ec67-479e-b538-7d0de2c20d8f
<<PINOKIO_SHELL>>eval "$(conda shell.bash hook)" ; conda deactivate ; conda deactivate ; conda deactivate ; conda activate base && npm run build
> ui@0.0.0 build
> tsc -b && vite build
vite v7.3.2 building client environment for production...
✓ 1807 modules transformed.
dist/index.html 5.14 kB │ gzip: 2.22 kB
dist/assets/index-DnFlDDc1.css 78.64 kB │ gzip: 13.08 kB
dist/assets/index-tK8ONRpK.js 965.98 kB │ gzip: 251.25 kB
(!) Some chunks are larger than 500 kB after minification. Consider:
- Using dynamic import() to code-split the application
- Use build.rollupOptions.output.manualChunks to improve chunking: https://rollupjs.org/configuration-options/#output-manualchunks
- Adjust chunk size limit for this warning via build.chunkSizeWarningLimit.
✓ built in 1.33s
(base) <<PINOKIO_SHELL>>
██ Terminated Shell c9ff09d0-ec67-479e-b538-7d0de2c20d8f
████
Disconnected...
Replies (1)
Up to 10 files, 25MB each. Images are optimized; GIFs -> MP4; videos 720p (max 120s).