| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| Fix C implementation of pickle.loads to use importlib's locking | ||
| mechanisms, and thereby avoid using partially-loaded modules. | ||
| Patch by Tim Burgess. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| Expand Up | @@ -6623,13 +6623,13 @@ _pickle_Unpickler_find_class_impl(UnpicklerObject *self, | |
| } | ||
| } | ||
|
|
||
| module = PyImport_GetModule(module_name); | ||
| /* | ||
| * we don't use PyImport_GetModule here, because it can return partially- | ||
| * initialised modules, which then cause the getattribute to fail. | ||
| */ | ||
| module = PyImport_Import(module_name); | ||
|
Comment thread
Copy link
Copy Markdown
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low QualityCould you benchmark how slower this is compared with just using PyImport_GetModule?
Sorry, something went wrong.
All reactions
Copy link
Copy Markdown
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Qualitygit master: $ ./python -m timeit -s "import pickle, logging; d = pickle.dumps((logging.Logger, logging.Handler, logging.Filter, logging.Formatter))" "pickle.loads(d)" 100000 loops, best of 5: 2.55 usec per loop This PR: $ ./python -m timeit -s "import pickle, logging; d = pickle.dumps((logging.Logger, logging.Handler, logging.Filter, logging.Formatter))" "pickle.loads(d)" 50000 loops, best of 5: 4.84 usec per loop So this is a sizable performance drop, but on a micro-benchmark. I don't think this will be significant in real-world cases. Usually, pickle performance becomes important when large pieces of data are serialized, and by construction you don't get many STACK_GLOBAL opcodes in it. cc'ing @pierreglaser @ogrisel out of curiosity.
Sorry, something went wrong.
All reactions
Copy link
Copy Markdown
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality+1. Also, STACK_GLOBAL should tend to progressively vanish in the pickle string, in favor of BINGET as functions/classes get progressively added to the memo (if used).
Sorry, something went wrong.
All reactions
Comment thread
pitrou marked this conversation as resolved.
|
||
| if (module == NULL) { | ||
| if (PyErr_Occurred()) | ||
| return NULL; | ||
| module = PyImport_Import(module_name); | ||
| if (module == NULL) | ||
| return NULL; | ||
| return NULL; | ||
| } | ||
| global = getattribute(module, global_name, self->proto >= 4); | ||
| Py_DECREF(module); | ||
| Expand Down | ||
| Back | FazBrowse Home | New Git URL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low QualityThis is a neat test :-)
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.