Capture snapshot of current python environment and recreate on another machine

Question

I have an environment created using miniconda with python 3.6.8, called basepy_3_6_8.

I want to save the environment snapshot to a file and then recreate it later on another machine:

There are different commands to capture the environment snapshot, with slightly different outputs. Which of these can I use to guarantee that the exact environment used by the user is recreated in the target?

I was hoping pip freeze > requirements.txt and pip install -r requirements.txt would work independent of the source environment, but I noticed that pip freeze from within a conda environment does not capture the python version.

Here is the code to create the conda environment, and output of different commands:

$ conda create -n myenv python=3.6.8
$ conda activate myenv

(myenv)$ pip freeze
astroid==2.1.0
autopep8==1.4.3
certifi==2018.11.29
colorama==0.4.1
isort==4.3.4
lazy-object-proxy==1.3.1
mccabe==0.6.1
pycodestyle==2.4.0
pylint==2.2.2
six==1.12.0
typed-ast==1.1.1
wincertstore==0.2
wrapt==1.11.0

(myenv)$ pip list
Package           Version
----------------- ----------
astroid           2.1.0
autopep8          1.4.3
certifi           2018.11.29
colorama          0.4.1
isort             4.3.4
lazy-object-proxy 1.3.1
mccabe            0.6.1
pip               18.1
pycodestyle       2.4.0
pylint            2.2.2
setuptools        40.6.3
six               1.12.0
typed-ast         1.1.1
wheel             0.32.3
wincertstore      0.2
wrapt             1.11.0

(myenv)$ conda list
# packages in environment at C:\Users\alias\AppData\Local\Continuum\miniconda3\envs\myenv:
#
# Name                    Version                   Build  Channel
certifi                   2018.11.29               py36_0
pip                       18.1                     py36_0
python                    3.6.8                h9f7ef89_0
setuptools                40.6.3                   py36_0
sqlite                    3.26.0               he774522_0
vc                        14.1                 h0510ff6_4
vs2015_runtime            14.15.26706          h3a45250_0
wheel                     0.32.3                   py36_0
wincertstore              0.2              py36h7fe50ca_0

(myenv)$ conda list --export
# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: win-64
certifi=2018.11.29=py36_0
pip=18.1=py36_0
python=3.6.8=h9f7ef89_0
setuptools=40.6.3=py36_0
sqlite=3.26.0=he774522_0
vc=14.1=h0510ff6_4
vs2015_runtime=14.15.26706=h3a45250_0
wheel=0.32.3=py36_0
wincertstore=0.2=py36h7fe50ca_0

I am eventually interested in a general tool that can capture the current environment of a specified type (conda, virtualenv, venv, global python environment) so as to install it uniformly on another machine. What is the best approach for this?

Answer 1

I've never used conda, but I'd try to use two different tools to manage the python version and your project dependencies.

To install a specific python version, I'd use pyenv: https://github.com/pyenv/pyenv.

pyenv also has a plugin to manage virtualenvs (https://github.com/pyenv/pyenv-virtualenv) that should support Anaconda and Miniconda: https://github.com/pyenv/pyenv-virtualenv#anaconda-and-miniconda

To manage your dependencies (packages you install in your virtual env), you have a few alternatives:

• Pip freeze: it doesn't automatically guarantee reproducibility though, because it doesn't have a lock file to pinpoint the exact dependency tree

• Poetry: https://github.com/sdispater/poetry (supports a lock file)

• Pipenv: https://github.com/pypa/pipenv (supports a lock file)

Hope this is helpful.