kunthawat/microfish
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

build: strip NVIDIA CUDA runtime libs from image (no local GPU)

Browse Source 
camel-oasis (dep of camel-ai/sentence-transformers) pulls in torch on
linux-x86_64, which drags several GB of nvidia-cuda-* / cudnn / triton
packages into the production image even though this deployment never runs an
LLM locally — all LLM calls go through an API (OpenAI-compatible) and the
server has no GPU. The nvidia-* packages are pure bloat.

After 'uv sync', uninstall all nvidia-* runtime libs + triton (torch itself
stays as a CPU runtime). Then verify the stripped env still imports psycopg,
torch, sentence-transformers and the app, so the build fails loudly if the
strip breaks anything instead of failing silently at container runtime.

Verified locally (no CUDA libs present): psycopg/torch/sentence_transformers/
create_app all import fine.
This commit is contained in:
Kunthawat Greethong
2026-08-31 20:38:03 +07:00
parent fb9275818e
commit 2fe8482ad4
1 changed files with 15 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files

17
Dockerfile
View File

@@ -43,9 +43,22 @@ COPY backend ./backend
COPY locales ./locales COPY locales ./locales
COPY package.json ./ COPY package.json ./
# Install backend deps against the final pyproject.toml/uv.lock # Install backend deps against the final pyproject.toml/uv.lock, then strip the
# NVIDIA CUDA runtime libraries that torch drags in (via camel-oasis ->
# sentence-transformers). The server has no GPU and all LLM calls go through an
# API, so CUDA libs are pure bloat (many GB). torch itself stays (CPU runtime)
# but without the nvidia-* packages the image is dramatically smaller. We then
# verify the stripped env still imports torch/sentence-transformers and the app
# so the build fails loudly instead of a silent runtime break.
RUN cd backend && uv sync --frozen --no-dev \ RUN cd backend && uv sync --frozen --no-dev \
&& uv run --frozen python -c "import psycopg" || (echo "FATAL: psycopg not installed — package sync missing the Postgres driver" >&2 && exit 1) && uv pip uninstall \
nvidia-cublas-cu12 nvidia-cuda-cupti-cu12 nvidia-cuda-nvrtc-cu12 \
nvidia-cuda-runtime-cu12 nvidia-cudnn-cu12 nvidia-cufft-cu12 \
nvidia-cufile-cu12 nvidia-curand-cu12 nvidia-cusolver-cu12 \
nvidia-cusparse-cu12 nvidia-cusparselt-cu12 nvidia-nccl-cu12 \
nvidia-nvjitlink-cu12 nvidia-nvshmem-cu12 nvidia-nvtx-cu12 triton \
&& uv run --frozen python -c "import psycopg, torch; import sentence_transformers; from app import create_app; print('deps-after-strip-ok')" \
|| (echo "FATAL: dependency import failed after CUDA strip (torch/sentence-transformers/app). Revisit the nvidia-* uninstall list." >&2 && exit 1)
# Make the migration-runner entrypoint executable # Make the migration-runner entrypoint executable
RUN chmod +x /app/backend/docker_entrypoint.sh RUN chmod +x /app/backend/docker_entrypoint.sh