Repository navigation
Trying to pickle an exception crashes the interpreter #284
Description
The following code apparently tries to write to a null pointer:
import clr import pickle from System import Exception pickle.dumps(Exception())
I'm investigating, where it fails exactly but I'm open to hints :)
Small addition: This works fine for simple objects like System.Object()
Activity
filmor commented on Nov 7, 2016
The reason for this seems to be the implementation of BaseException's reduce:
/* Pickling support */ static PyObject * BaseException_reduce(PyBaseExceptionObject *self) { if (self->args && self->dict) return PyTuple_Pack(3, Py_TYPE(self), self->args, self->dict); else return PyTuple_Pack(2, Py_TYPE(self), self->args); }In our hacked implementation that injects BaseException into the tp_bases of .NET exceptions, args and dict are both NULL, so the Decref on the result of this function will fail, could be that even the cast to PyBaseExceptionObject is already broken.
den-run-ai commented on Nov 7, 2016
Can't you use dill instead pickle?
I just tried and it works!
In [1]: import clr
In [2]: import dill
In [3]: dill.dumps(Exception())
Out[3]: '\x80\x02cexceptions\nException\nq\x00)Rq\x01.'
On Mon, Nov 7, 2016, 11:00 AM Benedikt Reinartz notifications@github.com
wrote:
The reason for this seems to be the implementation of BaseException's reduce
:
/* Pickling support */
static PyObject *
BaseException_reduce(PyBaseExceptionObject *self)
{
if (self->args && self->dict)
return PyTuple_Pack(3, Py_TYPE(self), self->args, self->dict);
else
return PyTuple_Pack(2, Py_TYPE(self), self->args);
}
In our hacked implementation that injects BaseException into the tp_bases
of .NET exceptions, args and dict are both NULL, so the Decref on the
result of this function will fail, could be that even the cast to
PyBaseExceptionObject is already broken.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#284 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHgZ5WhAoHLvz5WLP6LHqpk84jwvgP4zks5q71lBgaJpZM4KrQec
.
filmor commented on Nov 7, 2016
I could maybe do that to silence this particular error on my end, yes. However, this doesn't really fix the problem that you can easily kill the interpreter using this ;)
It happened to me when returning an exception object from a function run in a ProcessPoolExecutor, very difficult to debug ...
den-run-ai commented on Nov 7, 2016
I agree that crashing interpreter is not good :)
If dill does not work for your case, then there is also cloudpickle used by
PySpark.
On Mon, Nov 7, 2016 at 11:11 AM, Benedikt Reinartz <notifications@github.com
den-run-ai commented on Nov 8, 2016
this also works with cPickle (pickle speed up with C), but gives a different hash:
filmor commented on Nov 8, 2016
2 comments: