Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Using git submodules with python

I've read a lot of blog posts and questions on this site about the usage of git submodules and still have no idea how to better use them with python.

I mean, what is the easier way to manage dependencies if I have such a package:

├── mypkg
│   └── __init__.py
├── setup.py
└── submodules
    ├── subm1
    └── subm2

Then, what if I need to use "mypkg" as a submodule for "top_level_pkg":

├── setup.py
├── submodules
│   └── mypkg
└── top_level_package
    └── __init__.py

, I want to run pip install . and have all resolved correctly (have each submodule installed to the VENV in correct order).

What I've tried:

  • Install each submodule using "pip" running in a subprocess. But it seems to be a hacky way and hard to manage (Unexpected installation of GIT submodule)
  • Use "install_requires" with "setuptools.find_packages()" but without success
  • Use requirements.txt file for each submodule, but I can't find a way how to automate it so "pip" could automatically install all requirements for all submodules.

Ideally, I imagine a separate setup.py file for each submodule with install_requires=['submodules/subm1', 'submodules/submn'], but setuptools does not support it.

like image 864
garrywreck Avatar asked Oct 20 '25 03:10

garrywreck


2 Answers

I'm not saying it's impossible, but very hard and very tricky. A safer way is to turn each submodule into an installable Python module (with it's own setup.py) and install the submodules from Git.

This link describes how to install packages from Git with setup.py: https://stackoverflow.com/a/32689886/2952185

like image 167
Gijs Wobben Avatar answered Oct 21 '25 18:10

Gijs Wobben


Thankfully to Gijs Wobben and sinoroc I came up with solution that works for my case:

install_requires=['subm1 @ file://localhost/<CURENT_DIR>/path/to/subm1']
like image 30
garrywreck Avatar answered Oct 21 '25 18:10

garrywreck



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!