FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Support comparison operators by christabella · Pull Request #1347 · pythonnet/pythonnet · GitHub

Support comparison operators - #1347

Merged
lostmsu merged 12 commits into
pythonnet:masterfrom
christabella:feat/comparison-operators
Jan 12, 2021
Merged

Support comparison operators#1347
lostmsu merged 12 commits into
pythonnet:masterfrom
christabella:feat/comparison-operators

Conversation

christabella commented Jan 6, 2021
edited
Loading

Copy link
Copy Markdown
Contributor

What does this implement/fix? Explain your changes.

Continuation of https://github.com/pythonnet/pythonnet/pull/1324/files but specifically for comparison operators e.g. >, >=, <, == etc. based on a discussion with @tminka --- in a nutshell, we want to check each C# class for any comparison operator methods when calling ClassBase.tp_richcompare, before proceeding with the usual logic (see #294) which handles C# classes that implement an IComparable interface.

Would appreciate a review from @lostmsu who helped with the previous PR that this builds on. Thanks!

Does this close any currently open issues?

Closes #1312
More concrete examples can also be found in other Infer.NET tutorials e.g. having to use op_GreaterThan https://github.com/dotnet/infer/blob/67b4f80d97018460bcb817f76ec874d0f33f1651/test/TestPython/test_tutorials.py#L31

Any other comments?

Some remaining tasks could be:

Checklist

Check all those that are applicable and complete.

  • Make sure to include one or more tests for your change
  • If an enhancement PR, please create docs and at best an example
  • Add yourself to AUTHORS
  • Updated the CHANGELOG

The tests pass in net472 but not on netcoreapp3.1

christabella marked this pull request as draft January 6, 2021 15:55
christabella force-pushed the feat/comparison-operators branch from 6196140 to 3c4ea26 Compare January 7, 2021 07:13
christabella marked this pull request as ready for review January 7, 2021 07:45
christabella mentioned this pull request Jan 7, 2021
4 tasks
Comment thread src/runtime/classbase.cs Outdated
[NonSerialized]
internal List<string> dotNetMembers;
internal Indexer indexer;
internal Hashtable richcompare;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Why not have it as Dictionary<int, MethodObject>, mapping directly from {Py_EQ, ...} to corresponding operator implementation?

Comment thread src/runtime/classbase.cs Outdated
// otherwise fallback to checking if an IComparable interface is handled.
if (PyToCilOpMap.ContainsKey(op)) {
string CilOp = PyToCilOpMap[op];
if (cls.richcompare.Contains(CilOp)) {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Generally, you'd do cls.richcompare.TryGetValue(op, out var methodObject) instead of having separate ContainsKey check.

Comment thread src/runtime/classbase.cs Outdated
var methodObject = (MethodObject)cls.richcompare[CilOp];
IntPtr args = other;
var free = false;
if (!Runtime.PyTuple_Check(other))

lostmsu Jan 7, 2021
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I don't think the code inside this if should ever be skipped. How would a > (1,2) work if a was a .NET object with > operator? Because (1,2) is a tuple, they'd be passed to Invoke as two args instead of 1.

Consequently, free will alwasy be true and not needed.

Probably a good idea to add a test. I believe you'd need an operator >(SomeClass a, PyObject b) defined and to check b indeed receives the tuple (1,2).

christabella Jan 8, 2021
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Indeed if b were a tuple this if would be skipped. Once removing the if, the operator method does receive b as PyObject, but I get an AccessViolationException : Attempted to read or write protected memory. so I think b is wrongly parsed? I suppose a tuple when converted into a PyObject should probably not look like this:

christabella Jan 8, 2021
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think that error message might have been a flaky bug, because now the variable states are the same but the error is:

Message: 
    Python.Runtime.PythonException : TypeError : '>=' not supported between instances of 'int' and 'tuple'

lostmsu Jan 9, 2021
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

This is because to work with PyObject instances in C# callbacks you must hold GIL. E.g. the method body must be inside using (Py.GIL()) { ... block. It applies to debugging too.

christabella Jan 11, 2021
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Thanks, I didn't know about that. I put the operator >(Obj a, PyObject b) body in a Py.GIL block and can now see that b has a value of {(1, 2)}

I added 2/6 of the tuple comparison tests but it's done in a bit of a strange way. In the test I'm assuming that the PyObject is a tuple; is that okay?


(Below is resolved)

However, the same error message persists, and when I try to debug

result = binding.info.Invoke(binding.inst, BindingFlags.Default, null, binding.args, null);

I get this error

Maybe I'm missing something simple again, should the GIL block should also be wrapping that statement? But that would make the change bigger than I thought. Are tuples commonly used enough for comparison operators, or can I put the tuple test in a different PR?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I don't think you need to add a test with a tuple for each comparison operator. One is enough.

lostmsu Jan 11, 2021
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

The failure on the screenshot is only happening because you have debugger attached. It tries to call ToString on some PyObject to show in Watches window, but it does not work without Py.GIL

Comment thread src/runtime/operatormethod.cs Outdated
Comment on lines +53 to +58
["op_Equality"] = new SlotDefinition("__eq__", TypeOffset.tp_richcompare),
["op_Inequality"] = new SlotDefinition("__ne__", TypeOffset.tp_richcompare),
["op_LessThanOrEqual"] = new SlotDefinition("__le__", TypeOffset.tp_richcompare),
["op_GreaterThanOrEqual"] = new SlotDefinition("__ge__", TypeOffset.tp_richcompare),
["op_LessThan"] = new SlotDefinition("__lt__", TypeOffset.tp_richcompare),
["op_GreaterThan"] = new SlotDefinition("__gt__", TypeOffset.tp_richcompare),

lostmsu Jan 7, 2021
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I think this is error-prone. op_LessThan does not really implement tp_richcompare. It would be better to have a separate ComparisonOpMap, string -> string, and fix IsOperatorMethod correspondingly.

It would also remove the need to change FixupSlots below.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Makes sense, thanks! Should that ComparisonOpMap also be merged with the other map for op_LessThan -> Py_LT?

christabella Jan 8, 2021
edited
Loading

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

I added this OperatorMethod.ComparisonOpMap in the latest commit, without merging it with the ClassBase.CilToPyOpMap, although they both have the same keys so perhaps they should be together in OperatorMethod.

Comment thread src/runtime/operatormethod.cs Outdated
foreach (var method in clrType.GetMethods(flags))
{
if (!IsOperatorMethod(method))
if (!IsOperatorMethod(method) || IsComparisonOp(method)) // We don't want to override ClassBase.tp_richcompare.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Direct comment to the reason why we don't want to override, otherwise it does not explain anything and simply says what code in this method does (which is generally not very useful).

Something like "comparison operators are handled by ClassBase.tp_richcompare"

Comment thread src/embed_tests/TestOperator.cs Outdated

public override int GetHashCode()
{
return 159832395 + Num.GetHashCode();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

unchecked(159832395 + Num.GetHashCode())

Comment thread src/runtime/classbase.cs Outdated
Comment on lines +94 to +95
if (true)
{

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Please, remove unnecessary if and free boolean (always true)

Comment thread src/runtime/operatormethod.cs Outdated
if (!IsOperatorMethod(method))
// We don't want to override slots for either non-operators or
// comparison operators, which are handled by ClassBase.tp_richcompare.
if (!IsOperatorMethod(method) || IsComparisonOp(method))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

NIT: !OpMethodMap .ContainsKey(method)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Of course, thanks!

christabella force-pushed the feat/comparison-operators branch from a9e28ba to 7a2b5e2 Compare January 12, 2021 08:00
lostmsu merged commit e44aa46 into pythonnet:master Jan 12, 2021
christabella deleted the feat/comparison-operators branch January 13, 2021 08:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

operator overloading

3 participants


Back | FazBrowse Home | New Git URL