How to create a docker image for shell scripts suite

Question

I want to create a docker image for BBmap suite which has a folder with multiple shell scripts (${SHELLTOOLS}). I have created a docker file below to create the docker image, but I also want to use helper script, entrypoint.sh. How do I make this docker file run with entrypoint.sh? Can someone please suggest me what I need to add/change to this docker file and entrypoint.sh so I can create a docker image and run it with the command below?

My Docker file:

FROM openjdk:8-jre
RUN    apt-get update \
    && apt-get install -y \
        build-essential \
        openjdk-11-jre \
        wget \
    && rm -rf /var/lib/apt/lists/*

ENV BBMAP_VERSION 38.20
ENV BBMAP_DIR /usr/local/bbmap

WORKDIR /usr/local
RUN set -eux; \
       wget -O bbmap.tar.gz https://sourceforge.net/projects/bbmap/files/BBMap_${BBMAP_VERSION}.tar.gz/download \
    && tar -zxf bbmap.tar.gz \
    && rm bbmap.tar.gz

ENV PATH ${BBMAP_DIR}:${PATH}
COPY ./entrypoint.sh /usr/local/bin/
## Add ENV for Shell scripts from BBMAP
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]

My entrypoint.sh:

#!/bin/bash
${SHELLTOOL} -Xmx${MEM}g in=${sortedFQ} delimiter=${DELIM} prefixmode=f column=${COLNUM} out=${sampleFQ}%_#.fq

My command to run this on a LSF server:

export SHELLTOOL="/usr/local/bbmap/demuxbyname.sh"; \
export MEM=20; \
export sortedFQ="/${PR}/${FULLSM}/paire_test_sorted.fastq"; \
export column=2; \
export DELIM=":"; \
export COLNUM=2; \
export sampleFQ="/${PR}/${FULLSM}/ADNI_1380"
bsub \
    -J "${FULLSM}_bbduk_demulti" \
    -o "${BASE}/${FULLSM}_bbduk_demulti.%J" \
    -n1 -W 1440 \
    -R "rusage[mem=20192]" \
    -q research-hpc \
    -a "docker(username/bbmap:latest)" \
entrypoint.sh

Answer 1

There are several ways you can add all those env vars to the executed container. I'll go with plain Docker and DockerCompose solutions. It depends on your needs which one is better for you. Whichever option you take, there are a couple of variables I've not seen: PR and FULLSM. You'll probably want to add them to the top of the variables definition blocks.

Plain Docker

You need those variables to be defined before entrypoint.sh is executed. You can add them at build time or at run time. To add them at build time, you'll need to modify the Dockerfile. You can hardcode it or parameterize, even with default values:

Hardcoded

This option is good if this variables NEVER change, and NEVER will change. Just add the env vars to the Dockerfile, as seen in the docs:

# Only one ENV to add only one extra layer
ENV \
  SHELLTOOL="/usr/local/bbmap/demuxbyname.sh" \
  MEM=20 \
  sortedFQ="/${PR}/${FULLSM}/paire_test_sorted.fastq" \
  column=2 \
  DELIM=":" \
  COLNUM=2 \
  sampleFQ="/${PR}/${FULLSM}/ADNI_1380"

Finally, rebuild your Docker container and run it. It should work.

Parameterized

This allows you to change the values with the build command, using the arg option. This is good if those variables are almost always the same, but can change from one environment to another, or may change in the future as they have already changed in the past. Furthermore, we will set them with default values, so you will have almost the same work to build it as in the first option:

ARG SHELLTOOL=/usr/local/bbmap/demuxbyname.sh
ARG MEM=20
ARG sortedFQ=/${PR}/${FULLSM}/paire_test_sorted.fastq
ARG column=2
ARG DELIM=:
ARG COLNUM=2
ARG sampleFQ=/${PR}/${FULLSM}/ADNI_1380

ENV \
  SHELLTOOL="${SHELLTOOL}" \
  MEM=${MEM} \
  sortedFQ="${sortedFQ}" \
  column=${column} \
  DELIM="${DELIM}" \
  COLNUM=${COLNUM} \
  sampleFQ="${sampleFQ}"

docker-compose

With this tool, you can define variables to use with a container, during build time or runtime, using declarative yaml format. If you want to define them during build time, you'll need to use also the parameterized option explained above, allowing docker-compose to use them.

Build time

docker-compose has an option to define build arguments:

version: "3"
services:

  bbmap:
    build:
      context: .
      args:                                                                     
        - SHELLTOOL=/usr/local/bbmap/demuxbyname.sh
        - MEM=20
        # etc...

You can even include an .env file and use it from the yaml:

.env:

SHELLTOOL="/usr/local/bbmap/demuxbyname.sh"
MEM=20
sortedFQ="/${PR}/${FULLSM}/paire_test_sorted.fastq"
column=2
DELIM=":"
COLNUM=2
sampleFQ="/${PR}/${FULLSM}/ADNI_1380"

docker-compose.yml:

version: "3"
services:

  bbmap:
    build:
      context: .
      args:
        - SHELLTOOL=${SHELLTOOL}
        - MEM=${MEM}
        # etc...

Run time

docker-compose has an option to define environment variables:

version: "3"
services:

  bbmap:
    build:
      context: .
    environment:
      - SHELLTOOL=/usr/local/bbmap/demuxbyname.sh
      - MEM=20
      # etc...

As with the build args option, you can use an .env file to include them:

.env:

SHELLTOOL="/usr/local/bbmap/demuxbyname.sh"
MEM=20
# etc...

docker-compose.yml:

version: "3"
services:

  bbmap:
    build:
      context: .
    environment:
      - SHELLTOOL=${SHELLTOOL}
      - MEM=${MEM}
      # etc...