I use Poetry for dependency management and want to build a Docker container - should I be running Poetry in the Docker container?

Question

Should I be running Poetry inside Docker, or should I be using pip, pipx, or something similar instead?

Should I be building wheels, or tarballs and passing them into the Docker container instead of running pip or poetry inside the Docker container?

Here's an example of one of the Dockerfiles that I tried, but never got to work - the environment is broken, and I have no idea why:

FROM python:3.10

# Configure Poetry
ENV POETRY_VERSION=1.2.0
ENV POETRY_HOME=/opt/poetry
ENV POETRY_VENV=/opt/poetry-venv
ENV POETRY_CACHE_DIR=/opt/.cache

# Install poetry separated from system interpreter
RUN python3 -m venv $POETRY_VENV \
    && $POETRY_VENV/bin/pip install -U pip setuptools \
    && $POETRY_VENV/bin/pip install poetry==${POETRY_VERSION}

# Add `poetry` to PATH
ENV PATH="${PATH}:${POETRY_VENV}/bin"

WORKDIR /app

# Install dependencies
COPY poetry.lock pyproject.toml ./
RUN poetry install

# Run your app
COPY . /app

ENTRYPOINT [ "./entrypoint.sh" ]

entrypoint.sh contains:

poetry run blah

In this case, the problem is that the virtualenv is broken, and I have no idea why:

$ docker run blah/blah:latest
The virtual environment found in /app/.venv seems to be broken.
Recreating virtualenv blah-9TtSrW0h-py3.10 in /opt/.cache/virtualenvs/blah-9TtSrW0h-py3.10
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  ...
    import dateutil.parser
ModuleNotFoundError: No module named 'dateutil'

Should I even be using Poetry within Docker? Should I use pip or pipx instead?

What are you supposed to do when you want to run a Python application in a container? Are wheels the answer? Is something else the answer?

Answer 1

I've come up with an approach that seems to work:

1. Use poetry to create a requirements.txt file:

$ poetry export -f requirements.txt -o requirements.txt --without-hashes

2. Install the application's dependencies from requirements.txt rather than using Poetry:

FROM python:3.10

WORKDIR /app
COPY . .

RUN pip install -r requirements.txt

ENTRYPOINT [ "./entrypoint.sh" ]

Where entrypoint.sh contains:

#!/bin/bash
python3 -m blah.cli "$@"

To ensure that my requirements.txt is in sync with poetry.lock I build the container using a Makefile which has been simplified to:

VERSION := $(shell poetry version | cut -d ' ' -f 2)
CONTAINER_NAME := blah/blah

requirements:
    poetry export -f requirements.txt -o requirements.txt --without-hashes

container: requirements
    docker build -t $(CONTAINER_NAME):$(VERSION) -t $(CONTAINER_NAME):latest .

.PHONY: requirements container

This translates poetry.lock into a requirements.txt file at build time when the build Makefile goal is called and ensures that the two files are in sync when building container images.