Cache dependencies between GitLab CI build stages

Question

I have the following GitLab yml file and I have written it in stages assuming that every stage would retain the dependencies, but seems it is not to be the case!

# This file is a template, and might need editing before it works on your project.
# Official Java image. Look for the different tagged releases at
# https://hub.docker.com/r/library/java/tags/ . A Java image is not required
# but an image with a JVM speeds up the build a bit.
image: java:8

variables:
  FILE_TARGET_PATH: $FILE_TARGET_PATH

stages:
  - test
  - run

cache:
  key: "$CI_COMMIT_REF_NAME" # contains either the branch or the tag, so it's caching per branch
  untracked: true
  paths:
    - "sbt-cache/.ivy.cache"
    - "sbt-cache/.boot"
    - "sbt-cache/.sbtboot"
    - "sbt-cache/target"

before_script:
  # Enable the usage of sources over https
  - apt-get update -yqq
  - apt-get install apt-transport-https -yqq
  # Add keyserver for SBT
  - echo "deb http://dl.bintray.com/sbt/debian /" | tee -a /etc/apt/sources.list.d/sbt.list
  - apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 2EE0EA64E40A89B84B2DF73499E82A75642AC823
  # Install SBT
  - apt-get update -yqq
  - apt-get install sbt -yqq

Run unit Tests:
  stage: test
  tags:
    - master
  script:
    # Execute your project's tests
    - sbt -Denv=test clean test

Run Pipeline:
  stage: run
  tags:
    - master
  script:
    # Execute the pipeline
    - sbt -Denv=test run

How can I cache the dependencies in this multi stage set up? I have a local runner on my machine that runs the pipeline. Would artifacts help me?

Answer 1

If you are using a newer sbt version than 1.0.4 the caching won't work for you as the compiler will always invalidate all the files. This compiler issue has already been reported here: https://github.com/sbt/sbt/issues/4168

My suggestion would be to downgrade sbt version to 1.0.4 for CI. If that doesn't help you can also check the reason why the cache is invalidated by adding to your build.sbt:

// Debug incremental zinc compiler
logLevel := Level.Debug
incOptions := incOptions.value.withApiDebug(true).withRelationsDebug(true)

I had the same issue for Bitbucket Pipelines CI and managed to successfully make it work here