Git: push all objects to remote repository (not just refs, but absolutely all blobs, trees and commits)

Question

Context: I am (mis)using Git not for version control, but to keep a record of the relationship between files. Git's internal data structure (basically a graph of objects) has exactly what I need (cryptographic hashes, etc...) and there are plenty of very good tools to manipulate the data.

One thing I have so far not been able to do is to push the entire graph to a remote repository. As far as I understand, git push only acts on refs, not on objects. Is there an easy way to push all of the objects (commits, trees, blobs) to a remote or do I have to add a ref (e.g. a branch) on each of the commits first, then push all branches, then delete all branches (locally and on the remote)?

Answer 1

The short answer is that you can't, not quite. While git push pushes / uploads various Git objects, it ends by setting one or more names in the other Git repository. The receiving Git then tends to run git gc immediately afterward.

In particular, you run:

git push remote refspec1 refspec2 ... refspecN

on the command line, and your Git calls up some other Git at remote (via URL taken from remote.remote.pushurl for instance). Then your Git delivers, to their Git, the objects identified by the source parts of each of your refspecs, or more precisely, offers them those objects by hash ID. You can put hash IDs here:

git push origin a123456:name

The a123456 is an abbreviated hash ID. Whatever object that names, your Git will offer its full hash ID to the other Git. The other Git will then check to see whether it has object a123456. If so, it tells you it doesn't need the actual content; if not, your Git will package the content, and now must offer any object that a123456 requires as well. For instance, if a123456 is a commit, your Git must offer its tree and parent commits; the other Git tells you whether it has them already, or needs them, which triggers yet more object-following as needed. (There are some optimizations here to cut down on the back-and-forth communications, but the idea should be clear by this point.)

Eventually, your Git and their Git agree that if your Git sends some set of objects, they will have all the objects they need to accept (or reject!) your Git's set some names request. Your Git then packages up those objects, typically as a thin pack, which stores those objects, but delta-compressed against objects your Git knows or assumes their Git already has based on the earlier have/want object-by-hash-ID conversation that led to this point. (For instance, if you offered a1234356 and they said they need it, then you offered its parent a000000 and they said ah, I already have that one, your Git knows that they not only have object a000000 itself, but also its tree and all of its blobs. Moreover, if their repository is not shallow, they have all of a000000's parent commits and all of their trees and blobs! So your Git can delta-compress a123456's blobs against all such earlier blobs, to make this thin pack. It can completely omit any blobs in a123456 that are also in a000000, and so on.)

Once your Git has sent the thin pack, the other side fixes it (fattens it and adds an index). They now take your Git's set of requests of the form: please set <name> to <hash> (non-forced push) or the same without the please (forced push). If you ran git push origin a123456:name, that request says set name to a123456.

They either obey the request/command, or reject it. If they reject it, then a123456—which you just sent them—is pretty likely not reachable from any of their other references! (The one case where it is, is when something else in the same push set or updated some other reference so as to make a123456 reachable.)

When the conversation ends, after all of your set <name> to <hash> operations, they run git gc. If the objects you sent them were loose objects, those objects would be protected by the default 14-day grace period. But you sent them a pack file. If they repack the pack file while incorporating the objects they're going to retain—and they almost always do that—they will drop any unreferenced object.

This leads to the somewhat sad conclusion that makes up the first sentence of this answer.