Bumping package.json version without invalidating docker cache

Question

I'm using a pretty standard Dockerfile to containerize a Node.js application:

# Simplified version
FROM node:alpine

# Copy package.json first for docker build's layer caching
COPY package.json package-lock.json foo/
RUN npm install

COPY src/ foo/
RUN npm run build

Breaking up my COPY into two parts was advantageous because it allowed Docker to cache the (long) npm install step.

Recently, however, I started bumping my package.json version using semver. This had the side effect of invalidating the Docker cache for the npm install step, lengthening my build times significantly.

Is there an alternative caching strategy I can use so that npm install only runs when my dependencies change?

Answer 1

Here's my take on this, based off other answers, but shorter and with usage of jq:

Dockerfile:

FROM endeveit/docker-jq AS deps

# https://stackoverflow.com/a/58487433
# To prevent cache invalidation from changes in fields other than dependencies

COPY package.json /tmp

RUN jq '{ dependencies, devDependencies }' < /tmp/package.json > /tmp/deps.json

FROM node:12-alpine

WORKDIR /app

COPY --from=deps /tmp/deps.json ./package.json
COPY package-lock.json .

RUN npm ci
# https://docs.npmjs.com/cli/ci.html#description

COPY . .

RUN npm run build

LABEL maintainer="Alexey Vishnyakov <[email protected]>"

I extract dependencies and devDependencies fields to a separate file, then on next build step I copy it from the previous step as package.json (COPY --from=deps /tmp/deps.json ./package.json).

After RUN npm ci, COPY . . will overwrite gutted package.json with the original one (you can test it by adding RUN cat package.json after COPY . . command.

Note that npm-scripts commands like postinstall won't run since they're not present in the file during npm ci and also if npm ci is running from root and without --unsafe-perm

Either run commands after COPY . . or/and (if needed) include them via jq (changing command will invalidate cache layer) or add --unsafe-perm

Dockerfile:

FROM endeveit/docker-jq AS deps

COPY package.json /tmp

RUN jq '{ dependencies, devDependencies, peerDependencies, scripts: (.scripts | { postinstall }) }' < /tmp/package.json > /tmp/deps.json
# keep postinstall script 

FROM node:12-alpine

WORKDIR /app

COPY --from=deps /tmp/deps.json ./package.json
COPY package-lock.json .

# RUN npm ci --unsafe-perm 
# allow postinstall to run from root (security risk)

RUN npm ci
# https://docs.npmjs.com/cli/ci.html#description

RUN npm run postinstall

...