Repository navigation
3.2.0 crashes on shutdown when binary formatter serialization is enabled #2748
Description
The tests in arrow-dotnet started crashing on shutdown after updating to pythonnet 3.2.0. After investigating, I found that this was caused by changing ModuleObject.cache and ModuleObject.allNames to be ConcurrentDictionary instances, which are not serializable.
We had set EnableUnsafeBinaryFormatterSerialization to true to work around errors previously, but we don't need the runtime serialization functionality so have now disabled this (apache/arrow-dotnet#457).
Minimal repro:
PythonNetRepro.csproj:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net8.0</TargetFramework>
<EnableUnsafeBinaryFormatterSerialization>true</EnableUnsafeBinaryFormatterSerialization>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="pythonnet" Version="3.2.0" />
</ItemGroup>
</Project>The net8.0 target framework is required. dotnet 9 removed the BinaryFormatter implementation and it now always throws a PlatformNotSupportedException.
Program.cs:
using Python.Runtime;
PythonEngine.Initialize();
PythonEngine.Shutdown()Reproducing the error:
python -m venv venv ./venv/bin/python -m pip install find-libpython PYTHONNET_PYDLL=$(./venv/bin/python -m find_libpython) dotnet run
This crashes with:
Unhandled exception. System.Runtime.Serialization.SerializationException: Type 'System.Collections.Concurrent.ConcurrentDictionary`2[[System.String, System.Private.CoreLib, Version=8.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e],[Python.Runtime.PyObject, Python.Runtime, Version=3.2.0.0, Culture=neutral, PublicKeyToken=5000fea6cba702dd]]' in Assembly 'System.Collections.Concurrent, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' is not marked as serializable. at System.Runtime.Serialization.FormatterServices.InternalGetSerializableMembers(Type type) at System.Runtime.Serialization.FormatterServices.<>c.<GetSerializableMembers>b__5_0(MemberHolder mh) at System.Collections.Concurrent.ConcurrentDictionary`2.GetOrAdd(TKey key, Func`2 valueFactory) at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.InitMemberInfo() at System.Runtime.Serialization.Formatters.Binary.WriteObjectInfo.InitSerialize(Object obj, ISurrogateSelector surrogateSelector, StreamingContext context, SerObjectInfoInit serObjectInfoInit, IFormatterConverter converter, ObjectWriter objectWriter, SerializationBinder binder) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Write(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo) at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Serialize(Object graph, BinaryFormatterWriter serWriter) at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream serializationStream, Object graph) at Python.Runtime.RuntimeData.Stash() at Python.Runtime.Runtime.Shutdown() at Python.Runtime.PythonEngine.Shutdown() at Program.<Main>$(String[] args) in /home/adam/dev/pythonet-repro/Program.cs:line 4
Activity
filmor commented on Oct 6, 2026
@adamreeve I have verified that my PR fixes the issue for the reproducible example (thanks for that!) and will release a 3.2.1 tomorrow. Feel free to reopen if it is not fixed for you.