Skip to content

[WIP] Nightly signal on torch #30146

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
atalman wants to merge 1 commit into
base: main
Choose a base branch
from
+24 −24
Open

[WIP] Nightly signal on torch #30146

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,8 @@ endif()
# requirements.txt files and should be kept consistent. The ROCm torch
# versions are derived from docker/Dockerfile.rocm
#
set(TORCH_SUPPORTED_VERSION_CUDA "2.9.0")
set(TORCH_SUPPORTED_VERSION_ROCM "2.9.0")
set(TORCH_SUPPORTED_VERSION_CUDA "2.10.0")
set(TORCH_SUPPORTED_VERSION_ROCM "2.10.0")
Comment on lines +59 to +60
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

With the move to nightly builds, the version check for CUDA at line 125 will consistently fail and produce warnings. Nightly versions (e.g., 2.10.0.dev...) will not match 2.10.0 with VERSION_EQUAL.

This will create unnecessary noise in build logs, potentially hiding real issues. Please consider updating the check to be more flexible for nightly versions, for example by using VERSION_GREATER_EQUAL or VERSION_MATCHES, similar to how the ROCm version is checked.


#
# Try to find python package with an executable that exactly matches
Expand Down Expand Up @@ -426,7 +426,7 @@ if(VLLM_GPU_LANG STREQUAL "CUDA")
endif()
list(APPEND VLLM_EXT_SRC ${MARLIN_TEMPLATE_BF16_KERNEL_SRC})

if (MARLIN_FP8_ARCHS)
if (MARLIN_FP8_ARCHS)
file(GLOB MARLIN_TEMPLATE_FP8_KERNEL_SRC "csrc/quantization/gptq_marlin/sm89_kernel_*.cu")
set_gencode_flags_for_srcs(
SRCS "${MARLIN_TEMPLATE_FP8_KERNEL_SRC}"
Expand Down
2 changes: 1 addition & 1 deletion docker/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ ARG UV_INDEX_URL=${PIP_INDEX_URL}
ARG UV_EXTRA_INDEX_URL=${PIP_EXTRA_INDEX_URL}

# PyTorch provides its own indexes for standard and nightly builds
ARG PYTORCH_CUDA_INDEX_BASE_URL=https://download.pytorch.org/whl
ARG PYTORCH_CUDA_INDEX_BASE_URL=https://download.pytorch.org/whl/nightly
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This change to use the nightly PyTorch index will cause the Docker build to fail. The CUDA_VERSION is set to 12.9.1 at the top of the file, which leads to the construction of an invalid URL .../nightly/cu129 for PyTorch wheels on line 143. The PyTorch nightly builds are not available for cu129 (the latest is cu128).

To fix this, you should either:

  1. Change CUDA_VERSION to a version that has a corresponding PyTorch nightly build (e.g., 12.8.x). This might require changing the base image.
  2. Hardcode a valid CUDA version for the PyTorch index URL in the uv pip install command. Given that CUDA_VERSION is used for the base image, this option is likely safer.

For example, you could change line 143 to:

-    --extra-index-url ${PYTORCH_CUDA_INDEX_BASE_URL}/cu$(echo $CUDA_VERSION | cut -d. -f1,2 | tr -d '.')
+    --extra-index-url ${PYTORCH_CUDA_INDEX_BASE_URL}/cu128


# PIP supports multiple authentication schemes, including keyring
# By parameterizing the PIP_KEYRING_PROVIDER variable and setting it to
Expand Down
2 changes: 1 addition & 1 deletion docker/Dockerfile.cpu
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ FROM ubuntu:22.04 AS base-common
WORKDIR /workspace/

ARG PYTHON_VERSION=3.12
ARG PIP_EXTRA_INDEX_URL="https://download.pytorch.org/whl/cpu"
ARG PIP_EXTRA_INDEX_URL="https://download.pytorch.org/whl/nightly/cpu"

# Install minimal dependencies and uv
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ requires = [
"packaging>=24.2",
"setuptools>=77.0.3,<81.0.0",
"setuptools-scm>=8.0",
"torch == 2.9.0",
"torch",
"wheel",
"jinja2",
]
Expand Down
2 changes: 1 addition & 1 deletion requirements/build.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ ninja
packaging>=24.2
setuptools>=77.0.3,<81.0.0
setuptools-scm>=8
torch==2.9.0
torch
wheel
jinja2>=3.1.6
Comment on lines 5 to 9
Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Unpinned build torch resolves to unsupported release

The build requirements now allow any torch version (requirements/build.txt line 7) while CMake declares 2.10 as the supported CUDA/ROCm version (CMakeLists.txt lines 59-60). With the default PyPI index this resolves to the current stable CPU wheel (e.g., 2.x) rather than the expected 2.10 GPU nightly, so a developer running uv pip install -r requirements/build.txt will compile the extensions against an unsupported CPU build that lacks the CUDA ABI and typically fails at build or import time. Please bound the requirement to the supported 2.10 nightly channel instead of leaving it unconstrained.

Useful? React with 👍 / 👎.

regex
Expand Down
6 changes: 3 additions & 3 deletions requirements/cuda.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@ numba == 0.61.2 # Required for N-gram speculative decoding

# Dependencies for NVIDIA GPUs
ray[cgraph]>=2.48.0 # Ray Compiled Graph, required for pipeline parallelism in V1.
torch==2.9.0
torchaudio==2.9.0
torch
torchaudio
# These must be updated alongside torch
torchvision==0.24.0 # Required for phi3v processor. See https://github.com/pytorch/vision?tab=readme-ov-file#installation for corresponding version
torchvision # Required for phi3v processor. See https://github.com/pytorch/vision?tab=readme-ov-file#installation for corresponding version
# FlashInfer should be updated together with the Dockerfile
flashinfer-python==0.5.3
10 changes: 5 additions & 5 deletions requirements/rocm-build.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# Common dependencies
-r common.txt

--extra-index-url https://download.pytorch.org/whl/rocm6.4
torch==2.9.0
torchvision==0.24.0
torchaudio==2.9.0
--extra-index-url https://download.pytorch.org/whl/nightly/rocm6.4
torch
torchvision
torchaudio

triton==3.5.0
pytorch-triton-rocm
cmake>=3.26.1,<4
packaging>=24.2
setuptools>=77.0.3,<80.0.0
Expand Down
6 changes: 3 additions & 3 deletions requirements/test.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,9 @@ soundfile # required for audio tests
jiwer # required for audio tests
tblib # for pickling test exceptions
timm >=1.0.17 # required for internvl and gemma3n-mm test
torch==2.9.0
torchaudio==2.9.0
torchvision==0.24.0
torch
torchaudio
torchvision
transformers_stream_generator # required for qwen-vl test
matplotlib # required for qwen-vl test
mistral_common[image,audio] >= 1.8.5 # required for voxtral test
Expand Down
8 changes: 4 additions & 4 deletions requirements/test.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1123,7 +1123,7 @@ tomli==2.2.1
# via schemathesis
tomli-w==1.2.0
# via schemathesis
torch==2.9.0+cu129
torch
# via
# -r requirements/test.in
# accelerate
Expand Down Expand Up @@ -1152,7 +1152,7 @@ torch==2.9.0+cu129
# torchvision
# vector-quantize-pytorch
# vocos
torchaudio==2.9.0+cu129
torchaudio
# via
# -r requirements/test.in
# encodec
Expand All @@ -1165,7 +1165,7 @@ torchmetrics==1.7.4
# pytorch-lightning
# terratorch
# torchgeo
torchvision==0.24.0+cu129
torchvision
# via
# -r requirements/test.in
# lightly
Expand Down Expand Up @@ -1206,7 +1206,7 @@ transformers==4.57.3
# transformers-stream-generator
transformers-stream-generator==0.0.5
# via -r requirements/test.in
triton==3.5.0
pytorch-triton
# via torch
tritonclient==2.51.0
# via
Expand Down
4 changes: 2 additions & 2 deletions requirements/xpu.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,8 @@ wheel
jinja2>=3.1.6
datasets # for benchmark scripts
numba == 0.61.2 # Required for N-gram speculative decoding
--extra-index-url=https://download.pytorch.org/whl/xpu
torch==2.9.0+xpu
--extra-index-url=https://download.pytorch.org/whl/nightly/xpu
torch
torchaudio
torchvision

Expand Down
Loading