How to use virtual environments without wasting disk space? [closed]

Question

I've watched many videos where they say we should have a new virtual environment for every Python project we make. When making new environments, I have to install my packages (numpy, matplotlib, etc.) over and over again. What can I do to keep these environments from using a lot of my computer's storage?

Answer 1

Well, you are right, virtual environments are somehow a waste of disk space because they are meant to create isolated environments that have -almost- no dependencies outside themselves.

E.g. if you have venvA and venvB, both can be using the same version of pckgX but, none of them will share it with the other and you will have the same pckgX installed in two different environments. However this is not an awful drawback, as in most cases you use python environments to have different versions of the same package in your machine and use them interchangeably, or that is why I use virtual environments.

It makes you comfortable when you modify or delete packages in your virtual environment with no worries that you may damage other ones.

However, we can overcome this by using caches and other methods:

• In pip we can cache downloads in ~/.pip/cache so it won't need to download them again next time by adding the following to $HOME/.pip/pip.conf:

[global]
download_cache = ~/.pip/cache

• In conda, you can use a shared package cache by creating a directory on your system where the shared users have read and write access.

Then, for each user who will have access, edit the .condarc file found in their home directory with the following entry, specifying the full path to that shared directory:

pkgs_dirs:
    - /path/to/shared_directory

Windows - C:\Users\username.condarc

macOS and Linux - /home/username/.condarc

Verify the package cache by running conda info.