How can I use Testcontainers to build a Dockerfile?

Question

I want to build an application. For testing it uses testcontainers. The build will run on CI and on the developers' machines. The Dockerfile is more or less:

FROM amazoncorretto:17-alpine as builder
add . .
run ./gradlew build

from amazoncorretto:17-alpine
copy --from=builder build/libs/*.jar app.jar
ENTRYPOINT ["java","-jar","/app.jar"]

And I run the build using docker build .

Part of the ./gradlew build runs tests with Testscontainers and uses

val sftpDocker = GenericContainer(DockerImageName.parse("atmoz/sftp:alpine"))

And it returns

java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration

I know that:

• Testcontainers has its own docker API client and doesn't requires installed docker inside the Alpine container 3
• Someone made it using "docker:20.10.14-dind" image. But I don't know how it fits in my problem 4
• I can mount the /var/run/docker.sock during docker run ... but I'm using RUN command inside dockerfile and docker build ... instead
• I can expose DOCKER_HOST and testcontainers should use the default gateway's IP address. But it's way less secure than using socket

So is there a way to use a socket in this setup? If not, how should I run my host Docker to expose TCP instead of a socket?

Answer 1

This is complicated, but possible. You need to mount the docker socket inside the docker build environment. However, you can't use a volume mount as you would when running a docker container. When you are building an image, you can only inject a network, e.g., docker build --network=host. You need to have the docker server exposed in that network, and then you can access it from within the docker build step. To achieve that, you can, for example, use socat twice, once to expose the host docker socket over the network, and a second time inside the Dockerfile to mount it under /var/run/docker.sock, where testcontainers will use it.

Here is an example. Let us start with a simple test container test.

import unittest
import psycopg2
from testcontainers.postgres import PostgresContainer

class MyIntegrationTest(unittest.TestCase):
    def test_postgres_version(self):
      pin = "sha256:36ed71227ae36305d26382657c0b96cbaf298427b3f1eaeb10d77a6dea3eec41"
      with PostgresContainer("postgres:16-alpine@" + pin, driver=None) as postgres:
        cursor = psycopg2.connect(postgres.get_connection_url()).cursor()
        cursor.execute('SELECT version()')
        version = cursor.fetchone()
        self.assertEqual(version[0][:13], "PostgreSQL 16")

Now let us write the Dockerfile that will run it as part of a build step.

FROM mcr.microsoft.com/devcontainers/base:alpine
RUN apk add py3-pip socat docker-cli
RUN pip install testcontainers[postgres]==4.7.2 psycopg2-binary==2.9.9 --break-system-packages
COPY test.py ./
ENV TESTCONTAINERS_HOST_OVERRIDE=gateway.docker.internal
RUN socat UNIX-LISTEN:/var/run/docker.sock,fork TCP:gateway.docker.internal:2375 & \
        python -m unittest discover

And use socat to guarantee that docker can be locally accessed through the host network.

socat -d0 TCP-LISTEN:2375,fork UNIX-CONNECT:/var/run/docker.sock&
DOCKER_HOST=tcp://localhost:2375 docker ps

And finally we can build our docker image, with testcontainers being part of the build step.

docker build --network=host --add-host gateway.docker.internal:host-gateway .

That is it.