Repository navigation
Use Python buffer protocol when converting Python objects to .NET arrays #1838
Description
Copying data from NumPy array to C# is a common task. Python.NET implements a simple sequence-to-array copying mechanism, which uses iterators. It is extremely inefficient for large arrays of primitive types.
Now we have a better way - using PyBuffer and buffer protocol. We could detect, that Python object implements the buffer protocol, use it to copy data when possible, and fallback to iteration otherwise.
Activity
lostmsu commented on Jun 27, 2022
If somebody wants to take on it, this test shows simple use of PyBuffer: TestBufferRead
And this is the method that needs to be changed: Converter.ToArray
filmor commented on Jun 27, 2022
This would not break API/ABI, so we can integrate it in a normal point release after 3.0, right?
lostmsu commented on Jun 27, 2022
Yep.
bruceYangxianglin commented on Aug 14, 2022
Copying data from C# array to Python list or Numpy array is also a common task. Is there a more efficient way?
bjudkewitz commented on Mar 25, 2023
Agreed, converting a python array to a C# array is a very common task, and it is currently extremely slow. It would be fantastic to make it faster. Is it currently possibly with Python (and Python.NET) only, without any C# programming?
yahav6893 commented on Nov 19, 2023
Hi! I’m looking for some guidance on how to speed up copying from Python to C#.
Would anyone be able to point me in the right way to do that?
Thanks ahead.
chickenservice commented on Apr 15, 2024
Hi, is someone currently working on this? If not I'd be happy to try. I've been using pythonnet at work successfully for years and would be glad to help. I'm familiar with the topic because I have already dealt with converting numpy arrays from/to C# arrays in an efficient manner. It'd be great to have this natively in pythonnet.
filmor commented on Apr 15, 2024
@chickenservice To my knowledge, no one is currently actively working on this, so give it a shot! An implementation would be greatly appreciated!
chickenservice commented on Apr 15, 2024
@filmor Perfect, is there anything besides the steps in CONTRIBUTING.md that I need to be aware of?