Repository navigation
Python.exe crashes if __pycache__ directories exist (garbage collection trashed) #481
Description
Environment
- Pythonnet version: 2.3.0
- Python version: 3.5 64-bit
- Operating System: Windows 7 Pro 64-bit SP1
Details
- Describe what you were trying to get done.
I am trying to import .NET assemblies and use them together. Unfortunately, python interpreter always crashes if .py files are already precompiled and lay inside pycache directories. I tried breifly python 3.6 and newer pythonnet, but it still crashes. However it doesn't crash with old python 2.7.13 64-bit (need to recompile pythonnet for 2.7).
I used Debugger and see that python garbage collection is trashed. Mostly object->tp_is_gc field is invalid not showing to real function.
Deleteing pycache directories allow test.py to run once.
Please, help in identify the issue and in making the fix.
- What commands did you run to trigger this issue?
- Unpack zip, open Visual Studio 2015 Update 3 Command Prompt in this directory
- Need to compile pythonnet for python 3.5 64-bit windows. Copy clr.pyd and Python.Runtime.dll to the directory of run.bat
- start run.bat
- python.exe crashes after printing "0"
- If there was a crash, please include the traceback here.
The crash happens mostly inside visit_decref() function in PyObject_IS_GC(op) macro. It shows either "op" is 0 or "op->tp_is_gc" has a wrong address.
Activity
den-run-ai commented on May 31, 2017
did you try running python with -B flag?
testrunner123 commented on Jun 1, 2017
I've tried this: in 4 of 5 attempts, python interpreter 3.5 64-bit crashes at the exit. No pycache directores exist in this case.
vmuriart commented on Jun 1, 2017
I haven't seen yoyr sample code, this isn't related to the sub classing bug right? I'm curious if pycache existed when the subclass tests failed
den-run-ai commented on Jun 1, 2017
You zip file does not include Axa.* .NET dependencies.
testrunner123 commented on Jun 1, 2017
@vmuriart
I do subclass in sample and I believe it is related. In real assembly I have subclassing and strange effects. If I remove subclassing then it works.
@denfromufa
I do not understand. Well, I am not .NET pro. I compile with run.bat in Visual Studio 2015 Upd3 Developer Prompt and see only:
C:\dev\job3>csc /target:library /debug:full class.cs
Microsoft (R) Visual C# Compiler version 1.3.1.60616
Copyright (C) Microsoft Corporation. All rights reserved.
and class.dll is made somehow.
testrunner123 commented on Jun 1, 2017
@vmuriart
I tested and removing subclassing in sample code or just removing __namespace__ in adapter.py fixes the issue. Do you have an idea what this bug is ? Or may be it is fixed already??
den-run-ai commented on Jun 1, 2017
I see that it is self-contained now - Aka namespace is in the same class.cs.
testrunner123 commented on Jun 2, 2017
I nailed bug in src\runtime\typemananger.cs in CreateSubType(): method Runtime.XIncref() is not called, but PyObject is created and added to disposeList. Correct code:
if (0 != Runtime.PyMapping_HasKey(py_dict, assemblyKey.Handle)) { var pyAssembly = new PyObject(Runtime.PyDict_GetItem(py_dict, assemblyKey.Handle)); Runtime.XIncref(pyAssembly.Handle); ... } ... if (0 != Runtime.PyMapping_HasKey(py_dict, namespaceKey.Handle)) { var pyNamespace = new PyObject(Runtime.PyDict_GetItem(py_dict, namespaceKey.Handle)); Runtime.XIncref(pyNamespace.Handle); ... }Now it doesn't crash anymore.
Would be not easy to make regression test out of it. I realized that no crash if this code is completely in one .py. It must be spread among several modules to provoke garbage collector crash.
vmuriart commented on Jun 2, 2017
Nice! The subclass tests might catch this. Even though it's intermittent I run it multiple times each version to check for it.
testrunner123 commented on Jun 6, 2017
If it works in your tests then you can submit this patch to repo.
den-run-ai commented on Jun 6, 2017
@testrunner123 would you like to submit a pull request? otherwise someone can do this for you and attribute the commit to you git commit --author="???"
filmor commented on Jun 7, 2017
Hmm, we had quite a few of those. Maybe we should replace the current new PyObject by more explicit constructors like PyObject.FromPtrIncref, PyObject.BorrowFromPtr? I don't really have good names, though.
den-run-ai commented on Jun 7, 2017
@filmor i agree about PyObject.FromPtrIncref, when would PyObject.BorrowFromPtr happen?
filmor commented on Jul 11, 2017
I didn't think much about the names, I'll try to get a PR up with a proposal for explicit constructors.
lostmsu commented on Apr 23, 2020
@filmor the fix seems to be merged. Can this bug be closed?
filmor commented on Apr 23, 2020
I think so, yes.