Fail to active conda in Azure DevOps pipeline

Question

Testing the azure devops pipeline on a python project build by conda

jobs:
  - job: pre_build_setup
    displayName: Pre Build Setup
    pool:
      vmImage: 'ubuntu-18.04'
    steps:
      - bash: echo "##vso[task.prependpath]$CONDA/bin"
        displayName: Add conda to PATH

  - job: build_environment
    displayName: Build Environment
    dependsOn: pre_build_setup
    steps:
      - script: conda env create --file environment.yml --name build_env
        displayName: Create Anaconda environment
      - script: conda env list
        displayName:  environment installation verification

  - job: unit_tests
    displayName: Unit Tests
    dependsOn: build_environment
    strategy:
      maxParallel: 2
    steps:
      - bash: conda activate build_env

The last step - bash: conda activate build_env fails on me with the following error

Script contents:
conda activate build_env
========================== Starting Command Output ===========================
/bin/bash --noprofile --norc /home/vsts/work/_temp/d5af1b5c-9135-4984-ab16-72b82c91c329.sh

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.
To initialize your shell, run

    $ conda init <SHELL_NAME>

Currently supported shells are:
  - bash
  - fish
  - tcsh
  - xonsh
  - zsh
  - powershell

See 'conda init --help' for more information and options.

IMPORTANT: You may need to close and restart your shell after running 'conda init'.


##[error]Bash exited with code '1'.
Finishing: Bash

How can I active conda? seems the path is wrong so that it is unable to find conda.

Answer 1

CommandNotFoundError: Your shell has not been properly configured to use 'conda activate'.

Here the issue is your script is run in a sub-shell, but condahasn't been initialized in this sub-shell.

You need change your active script as：

steps:
  - task: Bash@3
    inputs:
      targetType: 'inline'
      script: |
        eval "$(conda shell.bash hook)"
        conda activate build_env
    displayName: Active

---

In addition, please do not split the Add PATH, create environment and active the environment into different jobs.

For Azure devops pipeline, agent job is the basic unit of the pipeline running process and each agent job has its own independent running environment and work logic.

For more detailed, you were using Hosted agent to apply your scripts in this issue scenario.

While there's one agent job starts to run, our pool system will assign an VM to this agent job. And, this VM will be recycled back once the agent job finished. When next agent job start to run , a completely new VM will be randomly reassign.

dependsOn can only share files and pass variables between jobs. It can not keep the VM continued in next job.

I believe you should be able to guess what problem you are going to encounter. Yes, even you can succeed to apply that activate script, you will continue face another error: Could not find conda environment: build_env. That's because the environment which is using by this activate script is a brand new vm, the VM that previous build_environment job used has been recycled by the system.

So, do not split the create environment and activate into 2 agent jobs:

  - job: build_environment
    displayName: Build Environment
    dependsOn: pre_build_setup
    steps:
      - script: conda env create --file environment.yml --name build_env
        displayName: Create Anaconda environment
      - script: conda env list
        displayName:  environment installation verification
      - task: Bash@3
        inputs:
          targetType: 'inline'
          script: |
            eval "$(conda shell.bash hook)"
            conda activate build_env
        displayName: Active

Answer 2

There's one more approach proposed by Microsoft which seems to be more robust.

In every step where you want the environment to be activated, you should run

source $CONDA/bin/activate <myEnv>

or just

source activate <myEnv>

if you've already added $CONDA/bin to the PATH variable. You may check the link above to find examples for Ubuntu, macOS and Windows.

In your case it would look as follows:

steps:
  - task: Bash@3
    inputs:
      targetType: 'inline'
      script: source activate build_env
    displayName: Active

Important note: as for now, if you pass the name of the environment to activate script, the environment must be created in the same job. However, if you're using prefix (i.e. path to the environment directory), it doesn't matter.