Repository navigation
Not possible to run embed_tests on Ubuntu #489
Description
Environment
- Pythonnet version: 2.3.0
- Python version: 3.5 3.4
- Operating System: Ubuntu 14.04.04 64-bit
- Mono 4.6
Details
- Describe what you were trying to get done.
I am trying to run embed_tests, but they do not run and I get 136 errors like:
- SetUp Error : Python.EmbeddingTest.TestPyInt
System.EntryPointNotFoundException : Py_IsInitialized
at (wrapper managed-to-native) Python.Runtime.Runtime:Py_IsInitialized ()
at Python.Runtime.Runtime.Initialize () [0x00001] in <6f2b25faa20d4740b2ac949a9317c3ed>:0
at Python.Runtime.PythonEngine.Initialize (System.Collections.Generic.IEnumerable`1[T] args, System.Boolean setSysArgv) [0x00016] in <6f2b25faa20d4740b2ac949a9317c3ed>:0
at Python.Runtime.PythonEngine.Initialize (System.Boolean setSysArgv) [0x00006] in <6f2b25faa20d4740b2ac949a9317c3ed>:0
at Python.Runtime.PythonEngine.Initialize () [0x00001] in <6f2b25faa20d4740b2ac949a9317c3ed>:0
at Python.EmbeddingTest.TestPyInt.SetUp () [0x00001] in :0
at (wrapper managed-to-native) System.Reflection.MonoMethod:InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&)
at System.Reflection.MonoMethod.Invoke (System.Object obj, System.Reflection.BindingFlags invokeAttr, System.Reflection.Binder binder, System.Object[] parameters, System.Globalization.CultureInfo culture) [0x00038] in <8f2c484307284b51944a1a13a14c0266>:0
The main point is here: "Py_IsInitialized" entry point is not found. The bug is that pythonnet does load "__Internal" library instead of libpython3.4m.so. This happens because setup.py detects if /usr/bin/python3 does link to ".so" and if not then load "__Internal". But this is wrong logic, because loading "__Internal" doesn't make any sense. Never. On Ubuntu Python Interpreter do not load libpythonXXX.so (is statically linked), but library libpythonXXX.so exists !
There are generally 2 cases on Linux:
- pythonnet is loaded from Python Interpreter (for example /usr/bin/python3 application)
How to check: try to resolve entry "Py_IsInitialized" in main application. If resolved then skip (2) - pythonnet is loaded from .NET Application
How to check: if (1) fails then it is the case. Try to load Python library (libpythonXXX.so) in this case.
- What commands did you run to trigger this issue? If you can provide a
Minimal, Complete, and Verifiable example
this will help us understand the issue.
nunit3-console.exe --noresult Python.EmbeddingTest.dll
Activity
den-run-ai commented on Jun 13, 2017
On Ubuntu are you using stock system Python 3.5 or you installed from somewhere else (e.g. Anaconda)?
If you are using custom Python can you check the output of ldd for libpython?
testrunner123 commented on Jun 14, 2017
@denfromufa
Sorry a bit of mess: the output that I reported is on Ubuntu 64-bit with stock python 3.4 from Ubuntu.org.
Please:
ldd /usr/lib/x86_64-linux-gnu/libpython3.4m.so:
ldd /usr/bin/python3.4:
testrunner123 commented on Jul 10, 2017
So what happens is pretty simple. There are 2 usecases:
In runtime.cs using "DllImport" all Python/C API functions are hardwired to "__Internal" library that is in "Mono" main application. This works in (1), but not in (2).
testrunner123 commented on Jul 10, 2017
I added code to load /usr/lib/x86_64-linux-gnu/libpython3.4m.so in Runtime.Initialize(). But now I get another error:
NUnit Console Runner 3.6.0 Copyright (C) 2017 Charlie Poole Runtime Environment OS Version: Linux 3.19.0.37 CLR Version: 4.0.30319.42000 Test Files Python.EmbeddingTest.dll => Python.EmbeddingTest.DynamicTest.AssignNone Fatal Python error: auto-releasing thread-state, but no thread-state for this thread Stacktrace: at <unknown> <0xffffffff> at (wrapper managed-to-native) Python.Runtime.Runtime.PyGILState_Release (intptr) <0x0005f> at Python.Runtime.PythonEngine.ReleaseLock (intptr) <0x0000f> at Python.EmbeddingTest.PyImportTest.Dispose () <0x00013> at (wrapper runtime-invoke) object.runtime_invoke_void__this__ (object,intptr,intptr,intptr) <0x000c8> at <unknown> <0xffffffff> at (wrapper managed-to-native) System.Reflection.MonoMethod.InternalInvoke (System.Reflection.MonoMethod,object,object[],System.Exception&) <0x0006c> at System.Reflection.MonoMethod.Invoke (object,System.Reflection.BindingFlags,System.Reflection.Binder,object[],System.Globalization.CultureInfo) <0x000b7> at System.Reflection.MethodBase.Invoke (object,object[]) <0x0002a> at NUnit.Framework.Internal.Reflect.InvokeMethod (System.Reflection.MethodInfo,object,object[]) <0x0006e> at NUnit.Framework.Internal.Reflect.InvokeMethod (System.Reflection.MethodInfo,object) <0x00013> at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunNonAsyncMethod (System.Reflection.MethodInfo,NUnit.Framework.Internal.TestExecutionContext) <0x0004f> at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunSetUpOrTearDownMethod (NUnit.Framework.Internal.TestExecutionContext,System.Reflection.MethodInfo) <0x00047> at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunTearDown (NUnit.Framework.Internal.TestExecutionContext) <0x0007f> at NUnit.Framework.Internal.Commands.SetUpTearDownCommand.Execute (NUnit.Framework.Internal.TestExecutionContext) <0x001d3> at NUnit.Framework.Internal.Execution.SimpleWorkItem.PerformWork () <0x00028> at NUnit.Framework.Internal.Execution.WorkItem.RunTest () <0x000ec> at NUnit.Framework.Internal.Execution.WorkItem.Execute () <0x0014f> at NUnit.Framework.Internal.Execution.ParallelWorkItemDispatcher.Execute (NUnit.Framework.Internal.Execution.WorkItem) <0x00088> at NUnit.Framework.Internal.Execution.ParallelWorkItemDispatcher.Dispatch (NUnit.Framework.Internal.Execution.WorkItem) <0x000ff> at NUnit.Framework.Internal.Execution.CompositeWorkItem.RunChildren () <0x00205> at NUnit.Framework.Internal.Execution.CompositeWorkItem.PerformWork () <0x0016f> at NUnit.Framework.Internal.Execution.WorkItem.RunTest () <0x000ec> at NUnit.Framework.Internal.Execution.WorkItem.Execute () <0x0014f> at NUnit.Framework.Internal.Execution.TestWorker.TestWorkerThreadProc () <0x00272> at System.Threading.ThreadHelper.ThreadStart_Context (object) <0x00083> at System.Threading.ExecutionContext.RunInternal (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) <0x00233> at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object,bool) <0x00028> at System.Threading.ExecutionContext.Run (System.Threading.ExecutionContext,System.Threading.ContextCallback,object) <0x00061> at System.Threading.ThreadHelper.ThreadStart () <0x00039> at (wrapper runtime-invoke) object.runtime_invoke_void__this__ (object,intptr,intptr,intptr) <0x000c8> Native stacktrace: /usr/bin/mono() [0x4accca] /lib/x86_64-linux-gnu/libpthread.so.0(+0x10340) [0x7fa19e41a340] /lib/x86_64-linux-gnu/libc.so.6(gsignal+0x39) [0x7fa19de65cc9] /lib/x86_64-linux-gnu/libc.so.6(abort+0x148) [0x7fa19de690d8] /usr/lib/x86_64-linux-gnu/libpython3.4m.so(Py_FatalError+0x50) [0x7fa198c5abf0] /usr/lib/x86_64-linux-gnu/libpython3.4m.so(PyGILState_Release+0x84) [0x7fa198df6a34] [0x4045c170] Debug info from gdb: Could not attach to process. If your uid matches the uid of the target process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try again as the root user. For more details, see /etc/sysctl.d/10-ptrace.conf ptrace: Operation not permitted. No threads. ================================================================= Got a SIGABRT while executing native code. This usually indicates a fatal error in the mono runtime or one of the native libraries used by your application. ================================================================= Errors, Failures and Warnings 1) Error : Tcp transport error. Server stack trace: at System.Runtime.Remoting.Channels.Tcp.TcpMessageIO.ReceiveMessageStatus (System.IO.Stream networkStream, System.Byte[] buffer) [0x0001a] in <55bb602902a54ad2b6e076d3b4161540>:0 at System.Runtime.Remoting.Channels.Tcp.TcpClientTransportSink.ProcessMessage (System.Runtime.Remoting.Messaging.IMessage msg, System.Runtime.Remoting.Channels.ITransportHeaders requestHeaders, System.IO.Stream requestStream, System.Runtime.Remoting.Channels.ITransportHeaders& responseHeaders, System.IO.Stream& responseStream) [0x00061] in <55bb602902a54ad2b6e076d3b4161540>:0 at System.Runtime.Remoting.Channels.BinaryClientFormatterSink.SyncProcessMessage (System.Runtime.Remoting.Messaging.IMessage msg) [0x0006c] in <55bb602902a54ad2b6e076d3b4161540>:0 Exception rethrown at [0]: at (wrapper managed-to-native) System.Object:__icall_wrapper_mono_remoting_wrapper (intptr,intptr) at (wrapper remoting-invoke) NUnit.Engine.Agents.RemoteTestAgent:Run (NUnit.Engine.ITestEventListener,NUnit.Engine.TestFilter) at NUnit.Engine.Runners.ProcessRunner.RunTests (NUnit.Engine.ITestEventListener listener, NUnit.Engine.TestFilter filter) [0x00025] in <f6948bd2223a41cc9ef1546388fbcb5e>:0 Test Run Summary Overall result: Failed Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0 Start time: 2017-07-10 09:45:34Z End time: 2017-07-10 09:45:35Z Duration: 0.779 secondstestrunner123 commented on Jul 11, 2017
Solution that I have found so far for Ubuntu users (I have 14.04 64-bit) is to preload libpython by starting tests with bash command:
Using this command execution of "src\embed_tests" succeed without errors.
Note that libpython is inside /usr/lib/x86_64-linux-gnu/ that is not a part of standard ldd search path. Further sysconfig or distutils in Python itself report wrong library path ("LIBDIR" is "/usr/lib").