Install dependencies from pyproject.toml but not the package

Question

I have a python package that I want to install inside a docker file.

pyproject.toml looks like:

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "bar"
dependencies = [
    "pandas",
]

[project.optional-dependencies]
foo = [
    "matplotlib",
]

... and Dockerfile:

# ...
WORKDIR /app
COPY . /app
RUN pip install /app

This installs the dependencies (in this example, pandas) on every build, which I want to avoid to save developer time, since pyproject.toml is rarely touched.

How can I install only pandas (the dependencies listed pyproject.toml) without having to COPY . and installing bar.

I want to avoid:

• adopt other tools like poetry
• create a requirements.txt and use dynamic keyword in pyproject.toml, because I have optional-dependencies and I want to keep the list of dependencies as close together (i.e. same file) as possible.

Something like:

# ...
WORKDIR /app
COPY ./pyproject.toml /app/
RUN pip install --requirements-from /app/pyproject.toml  # <-- HERE
COPY . /app
RUN pip install /app  # <-- Installs `bar` only. All dependencies already installed.

Answer 1

Sounds like you want to do a multi stage docker build

This way you need to install the dependencies from the package once, then you can just copy them for the subsequent builds.

It may not be exactly what you want, but it would be my first go to.

FROM python:3.11-slim AS builder

WORKDIR /app
COPY ./pyproject.toml /app/
RUN pip install --target /deps

FROM python:3.11-slim

COPY --from=builder /deps /deps

COPY . /app
RUN pip install --target /deps

If that is not satisfactory, you may want to do some pyproject.toml hacking using the https://pypi.org/project/pyproject-parser/ lib.

This could deteriorate faster, and I'd advice against it, but it could solve your problem.

Something like

from pyproject_parser import PyProjectTOML

# Load the source pyproject.toml
source_pyproject = PyProjectTOML.load('path/to/source/pyproject.toml')

# Load the destination pyproject.toml
destination_pyproject = PyProjectTOML.load('path/to/destination/pyproject.toml')

# Copy dependencies from source to destination
destination_pyproject.tool.poetry.dependencies.update(source_pyproject.tool.poetry.dependencies)

# Save the updated destination pyproject.toml
destination_pyproject.save('path/to/destination/pyproject.toml')