Using GitHub cache action with multiple cache paths?

Question

I'm trying to use the official GitHub cache action (https://github.com/actions/cache) to cache some binary files to speed up some of my workflows, however I've been unable to get it working when specifying multiple cache paths.

Here's a simple, working test I've set up using a single cache path: There is one action for writing the cache, and one for reading it (both executed in separate workflows, but on the same repository and branch). The write-action is executed first, and creates a file "subdir/a.txt", and then caches it with the "actions/cache@v2" action:

    # Test with single path
    - name: Create file
      shell: bash
      run: |
        mkdir subdir
        cd subdir
        printf '%s' "Lorem ipsum" >> a.txt
        
    - name: Write cache (Single path)
      uses: actions/cache@v2
      with:
        path: "D:/a/cache_test/cache_test/**/*.txt"
        key: test-cache-single-path

The read-action retrieves the cache, prints a list of all files in the directory recursively to confirm it has restored the file from the cache, and then prints the contents of the cached txt-file:

    - name: Get cached file
      uses: actions/cache@v2
      id: get-cache
      with:
        path: "D:/a/cache_test/cache_test/**/*.txt"
        key: test-cache-single-path
    
    - name: Print files
      shell: bash
      run: |
        echo "Cache hit: ${{steps.get-cache.outputs.cache-hit}}"
        cd "D:/a/cache_test/cache_test"
        ls -R
        cat "D:/a/cache_test/cache_test/subdir/a.txt"

This works without any issues.

Now, the description of the cache action contains an example for specifying multiple cache paths:

  - uses: actions/cache@v2
    with:
      path: | 
        path/to/dependencies
        some/other/dependencies 
      key: ${{ runner.os }}-${{ hashFiles('**/lockfiles') }}

But when I try that for my example actions, it fails to work. In the new write-action, I create two files, "subdir/a.txt" and "subdir/b.md", and then cache them by specifying two paths:

    # Test with multiple paths
    - name: Create files
      shell: bash
      run: |
        mkdir subdir
        cd subdir
        printf '%s' "Lorem ipsum" >> a.txt
        printf '%s' "dolor sit amet" >> b.md

    #- name: Write cache (Multi path)
      uses: actions/cache@v2
      with:
        path: |
          "D:/a/cache_test/cache_test/**/*.txt"
          "D:/a/cache_test/cache_test/**/*.md"
        key: test-cache-multi-path

The new read-action is the same as the old one, but also specifies both paths:

    # Read cache
    - name: Get cached file
      uses: actions/cache@v2
      id: get-cache
      with:
        path: |
          "D:/a/cache_test/cache_test/**/*.txt"
          "D:/a/cache_test/cache_test/**/*.md"
        key: test-cache-multi-path
    
    - name: Print files
      shell: bash
      run: |
        echo "Cache hit: ${{steps.get-cache.outputs.cache-hit}}"
        cd "D:/a/cache_test/cache_test"
        ls -R
        cat "D:/a/cache_test/cache_test/subdir/a.txt"
        cat "D:/a/cache_test/cache_test/subdir/b.md"

This time I still get the confirmation that the cache has been read:

Cache restored successfully
Cache restored from key: test-cache-multi-path
Cache hit: true

However "ls -R" does not list the files, and the "cat" commands fail because the files do not exist.

Where is my error? What is the proper way of specifying multiple paths with the cache action?

Answer 1

I was able to make it work with a few modifications;

• use relative paths instead of absolute
• use a hash of the content for the key

It looks like with at least bash the absolute paths look like this:

• /d/a/so-foobar-cache/so-foobar-cache/cache_test/cache_test/subdir

Where so-foobar-cache is the name of the repository.

.github/workflows/foobar.yml

name: Store and Fetch cached files
on: [push]
jobs:
  store:
    runs-on: windows-2019
    steps:
      - name: Create files
        shell: bash
        id: store
        run: |
          mkdir -p 'cache_test/cache_test/subdir'
          cd 'cache_test/cache_test/subdir'
          echo pwd $(pwd)
          printf '%s' "Lorem ipsum" >> a.txt
          printf '%s' "dolor sit amet" >> b.md
          cat a.txt b.md
      - name: Store in cache
        uses: actions/cache@v2
        with:
          path: |
            cache_test/cache_test/**/*.txt
            cache_test/cache_test/**/*.md
          key: multiple-files-${{ hashFiles('cache_test/cache_test/**') }}
      - name: Print files (A)
        shell: bash
        run: |
          echo "Cache hit: ${{steps.store.outputs.cache-hit}}"
          find cache_test/cache_test/subdir
          cat cache_test/cache_test/subdir/a.txt
          cat cache_test/cache_test/subdir/b.md


  fetch:
    runs-on: windows-2019
    needs: store
    steps:
      - name: Restore
        uses: actions/cache@v2
        with:
          path: |
            cache_test/cache_test/**/*.txt
            cache_test/cache_test/**/*.md
          key: multiple-files-${{ hashFiles('cache_test/cache_test/**') }}
          restore-keys: |
            multiple-files-${{ hashFiles('cache_test/cache_test/**') }}
            multiple-files-
      - name: Print files (B)
        shell: bash
        run: |
          find cache_test -type f | xargs -t grep -e.

Log

$ gh run view 1446486801 

✓ master Store and Fetch cached files · 1446486801
Triggered via push about 3 minutes ago

JOBS
✓ store in 5s (ID 4171907768)
✓ fetch in 10s (ID 4171909690)

First job

$ gh run view 1446486801 --log --job=4171907768 | grep -e Create  -e Store -e Print
store   Create files    2021-11-10T22:59:32.1396931Z ##[group]Run mkdir -p 'cache_test/cache_test/subdir'
store   Create files    2021-11-10T22:59:32.1398025Z mkdir -p 'cache_test/cache_test/subdir'
store   Create files    2021-11-10T22:59:32.1398695Z cd 'cache_test/cache_test/subdir'
store   Create files    2021-11-10T22:59:32.1399360Z echo pwd $(pwd)
store   Create files    2021-11-10T22:59:32.1399936Z printf '%s' "Lorem ipsum" >> a.txt
store   Create files    2021-11-10T22:59:32.1400672Z printf '%s' "dolor sit amet" >> b.md
store   Create files    2021-11-10T22:59:32.1401231Z cat a.txt b.md
store   Create files    2021-11-10T22:59:32.1623649Z shell: C:\Program Files\Git\bin\bash.EXE --noprofile --norc -e -o pipefail {0}
store   Create files    2021-11-10T22:59:32.1626211Z ##[endgroup]
store   Create files    2021-11-10T22:59:32.9569082Z pwd /d/a/so-foobar-cache/so-foobar-cache/cache_test/cache_test/subdir
store   Create files    2021-11-10T22:59:32.9607728Z Lorem ipsumdolor sit amet
store   Store in cache  2021-11-10T22:59:33.9705422Z ##[group]Run actions/cache@v2
store   Store in cache  2021-11-10T22:59:33.9706196Z with:
store   Store in cache  2021-11-10T22:59:33.9706815Z   path: cache_test/cache_test/**/*.txt
store   Store in cache  cache_test/cache_test/**/*.md
store   Store in cache  
store   Store in cache  2021-11-10T22:59:33.9708499Z   key: multiple-files-25c0e6413e23766a3681413625169cee1ca3a7cd2186cc1b1df5370fb43bce55
store   Store in cache  2021-11-10T22:59:33.9709961Z ##[endgroup]
store   Store in cache  2021-11-10T22:59:35.1757943Z Received 260 of 260 (100.0%), 0.0 MBs/sec
store   Store in cache  2021-11-10T22:59:35.1761565Z Cache Size: ~0 MB (260 B)
store   Store in cache  2021-11-10T22:59:35.1781110Z [command]C:\Windows\System32\tar.exe -z -xf D:/a/_temp/653f7664-e139-4930-9710-e56942f9fa47/cache.tgz -P -C D:/a/so-foobar-cache/so-foobar-cache
store   Store in cache  2021-11-10T22:59:35.2069751Z Cache restored successfully
store   Store in cache  2021-11-10T22:59:35.2737840Z Cache restored from key: multiple-files-25c0e6413e23766a3681413625169cee1ca3a7cd2186cc1b1df5370fb43bce55
store   Print files (A) 2021-11-10T22:59:35.3087596Z ##[group]Run echo "Cache hit: "
store   Print files (A) 2021-11-10T22:59:35.3088324Z echo "Cache hit: "
store   Print files (A) 2021-11-10T22:59:35.3088983Z find cache_test/cache_test/subdir
store   Print files (A) 2021-11-10T22:59:35.3089571Z cat cache_test/cache_test/subdir/a.txt
store   Print files (A) 2021-11-10T22:59:35.3090176Z cat cache_test/cache_test/subdir/b.md
store   Print files (A) 2021-11-10T22:59:35.3104465Z shell: C:\Program Files\Git\bin\bash.EXE --noprofile --norc -e -o pipefail {0}
store   Print files (A) 2021-11-10T22:59:35.3106449Z ##[endgroup]
store   Print files (A) 2021-11-10T22:59:35.3494703Z Cache hit: 
store   Print files (A) 2021-11-10T22:59:35.4456032Z cache_test/cache_test/subdir
store   Print files (A) 2021-11-10T22:59:35.4456852Z cache_test/cache_test/subdir/a.txt
store   Print files (A) 2021-11-10T22:59:35.4459226Z cache_test/cache_test/subdir/b.md
store   Print files (A) 2021-11-10T22:59:35.4875011Z Lorem ipsumdolor sit amet
store   Post Store in cache 2021-11-10T22:59:35.6109511Z Post job cleanup.
store   Post Store in cache 2021-11-10T22:59:35.7899690Z Cache hit occurred on the primary key multiple-files-25c0e6413e23766a3681413625169cee1ca3a7cd2186cc1b1df5370fb43bce55, not saving cache.

Second job

$ gh run view 1446486801 --log --job=4171909690  | grep -e Restore -e Print
fetch   Restore 2021-11-10T22:59:50.8498516Z ##[group]Run actions/cache@v2
fetch   Restore 2021-11-10T22:59:50.8499346Z with:
fetch   Restore 2021-11-10T22:59:50.8499883Z   path: cache_test/cache_test/**/*.txt
fetch   Restore cache_test/cache_test/**/*.md
fetch   Restore 
fetch   Restore 2021-11-10T22:59:50.8500449Z   key: multiple-files-
fetch   Restore 2021-11-10T22:59:50.8501079Z   restore-keys: multiple-files-
fetch   Restore multiple-files-
fetch   Restore 
fetch   Restore 2021-11-10T22:59:50.8501644Z ##[endgroup]
fetch   Restore 2021-11-10T22:59:53.1143793Z Received 257 of 257 (100.0%), 0.0 MBs/sec
fetch   Restore 2021-11-10T22:59:53.1145450Z Cache Size: ~0 MB (257 B)
fetch   Restore 2021-11-10T22:59:53.1163664Z [command]C:\Windows\System32\tar.exe -z -xf D:/a/_temp/30b0dc24-b25f-4713-b3d3-cecee7116785/cache.tgz -P -C D:/a/so-foobar-cache/so-foobar-cache
fetch   Restore 2021-11-10T22:59:53.1784328Z Cache restored successfully
fetch   Restore 2021-11-10T22:59:53.5197756Z Cache restored from key: multiple-files-
fetch   Print files (B) 2021-11-10T22:59:53.5483939Z ##[group]Run find cache_test -type f | xargs -t grep -e.
fetch   Print files (B) 2021-11-10T22:59:53.5484730Z find cache_test -type f | xargs -t grep -e.
fetch   Print files (B) 2021-11-10T22:59:53.5498140Z shell: C:\Program Files\Git\bin\bash.EXE --noprofile --norc -e -o pipefail {0}
fetch   Print files (B) 2021-11-10T22:59:53.5498674Z ##[endgroup]
fetch   Print files (B) 2021-11-10T22:59:55.8119800Z grep -e. cache_test/cache_test/subdir/a.txt cache_test/cache_test/subdir/b.md
fetch   Print files (B) 2021-11-10T22:59:56.1777887Z cache_test/cache_test/subdir/a.txt:Lorem ipsum
fetch   Print files (B) 2021-11-10T22:59:56.1784138Z cache_test/cache_test/subdir/b.md:dolor sit amet
fetch   Post Restore    2021-11-10T22:59:56.3890391Z Post job cleanup.
fetch   Post Restore    2021-11-10T22:59:56.5481739Z Cache hit occurred on the primary key multiple-files-, not saving cache.