[wishlist] Make Git() serializable (i.e. pickleable) #504
Description
I decided to become fancy and use some multiprocessing in our datalad, but quickly was bombarded with
File "/usr/lib/python2.7/multiprocessing/pool.py", line 567, in get
raise self._value
PicklingError: Can't pickle <class 'git.cmd.AutoInterrupt'>: attribute lookup git.cmd.AutoInterrupt failed
()
so I looked into cmd.py to see that AutoInterupt is actuall a sub-class of the Git class... so something went hairy here and I have decided to reproduce on a more obvious snippet:
$> python -c 'from git import Git; r = Git("/tmp/123"); import pickle; pickle.dumps(r)'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python2.7/pickle.py", line 1380, in dumps
Pickler(file, protocol).dump(obj)
File "/usr/lib/python2.7/pickle.py", line 224, in dump
self.save(obj)
File "/usr/lib/python2.7/pickle.py", line 306, in save
rv = reduce(self.proto)
File "/usr/lib/python2.7/copy_reg.py", line 77, in _reduce_ex
raise TypeError("a class that defines __slots__ without "
TypeError: a class that defines __slots__ without defining __getstate__ cannot be pickledso failed to pickle directly but differently... but overall story is that as long as I have a Git object involved anyhow -- can't pickle my beast, can't use multiprocessing :-/
Activity
Byron commented on Sep 11, 2016
A fix could be as easy as to not use slots at all. Considering the difficulties slots add to subclasses, and that it seems to be generally frowned upon in the python community for all their weaknesses, it would be a good move at least in this case.
I tried it and it seems to not break anything, thus I'd just push the respective change.
fix(repo): make it serializable with pickle