Install private python package in Gitlab CI using CI token

Question

From my Gitlab CI I'm trying to install a Python package from a private registry with this line:

pip install pipelinewriter --index-url https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/api/v4/projects/.../packages/pypi/simple

This fails: ERROR: Could not find a version that satisfies the requirement.

When I use my PAT, it succeeds:

pip install pipelinewriter --index-url https://__token__:<MY_PAT>@gitlab.com/api/v4/projects/.../packages/pypi/simple

Preferably I'd use the CI token instead of my PAT. Is this possible?

I was able to push the package using the CI token, following these instructions. This makes me think it should be possible to install the package using the same credentials.

I also tried adding some lines to the .netrc file as suggested by others:

.install_python_dependencies:
  before_script:
    - echo "Hacky credentials..."
    - echo "machine gitlab.com" > ~/.netrc
    - echo "login gitlab-ci-token" >> ~/.netrc
    - echo "password ${CI_JOB_TOKEN}" >> ~/.netrc

    - echo "Installing Python dependencies..."
    - pip install -r requirements.txt
    - pip install pipelinewriter --index-url https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.com/api/v4/projects/.../packages/pypi/simple
    # - pip install pipelinewriter --index-url https://__token__:${PAT}@gitlab.com/api/v4/projects/.../packages/pypi/simple >>> This works
    - export PYTHONPATH="."

But I keep facing the some error.

Answer 1

If the pipeline trying to pull the package from the registry it's in another project, then the issue lies in the permissions that $CI_JOB_TOKEN has.

The project that attempts to pull from the pipeline needs to be allowlisted to have cross-project permissions. That option is located at Settings > CI/CD menu.

More details can be found here