Gitlab docker run cannot mount ${CI_PROJECT_DIR} folder

Question

The build container has access to the docker socket through a bind mount. In a pipeline, the following will not work (resulting in an empty /project directory):

    - docker run
      --volume ${CI_PROJECT_DIR}:/project
      image-name

Instead, I had to find the container name of the build container and attach its volumes to my new container:

    - |
      docker_ps_output=$(docker ps --format "{{.Names}}" --filter "label=com.gitlab.gitlab-runner.job.id=${CI_JOB_ID}")
      readarray -t gitlab_container_names <<<"$docker_ps_output"

      for container_name in "${gitlab_container_names[@]}"
      do
        if [[ $container_name == *"build"* ]]; then
          echo "Found build container $container_name"
          export BUILD_CONTAINER_NAME=$container_name
          break
        else
          echo "Ignoring ${container_name}. We are looking for the build container".
        fi
      done

      if [ -z $BUILD_CONTAINER_NAME ]; then
        echo "Could not find build container name"
        exit 1
      fi
    - docker run
      --volume-from=$BUILD_CONTAINER_NAME
      image-name

Is this a problem on my end, a limitation of Docker (a container not being able to mount one of its volumes onto a new container) or an issue with Gitlab?

Answer 1

First, definitions:

• host - the GitLab Runner machine. When a new job is created, this machine is creating a new container in itself and the job is running in it.
• job's container - the container the started in the host and the job's script commands are running in it.

The behavior you see is what ew expect to see. This is because, as you said, the docker socket of the host is mounted to the job's container.
When you do docker run in the job, the new container is created inside the host (the GitLab Runner machine) and not inside the job's container. So now both the job's container and the new container are containers in the host.

This means that the --volume ${CI_PROJECT_DIR}:/project in the command is actually mounting the folder ${CI_PROJECT_DIR} of the host to the folder /project in the new container. Since such folder does not exist in the host. The folder /project in the new container will be empty. Each file that will be created in it during the run will be in the folder ${CI_PROJECT_DIR} in the host, not in the folder ${CI_PROJECT_DIR} in the job's container.

Your solution is a valid one. I used it myself.

A better way is to avoid using docker run in a job. Instead you can use the image image-name as the job's image (image: keyword). This is better but not always possible.

Another solution, if you have control over the Runner, is to mount a folder from the Runner to the job's container. This is done in the Runner's toml configuration. You can also mount the base folder of the jobs (usually /builds, I think) from the host to the the job's containers. Then, your original command

    - docker run
      --volume ${CI_PROJECT_DIR}:/project
      image-name

will work as well.