Fix Hash#inspect of Symbol keys with non-ASCII encoding - #9410
Conversation
Sorry, something went wrong.
isSimpleName classified names via a Java String built by RubyEncoding.decodeRaw (byte-as-Latin-1), misclassifying multibyte characters. Use the existing isSymbolName(String) when ASCII and use the encoding-aware SymbolNameType when the encoding is not ASCII. Re-enables TestHash#test_inspect; also drops a stale no-op exclude in TestHashOnly (no test_inspect method exists in that class).
|
Nice, thank you! Will wait to see how the tests shake out. |
Sorry, something went wrong.
|
Thanks! All the tests pass. The challenging part was the differences between isSymbolName(String) and SymbolNameType. isSymbolName(String) has checks for global symbols like :$-w and other weird Ruby-stuff, but SymbolNameType (in IdUtil.determineSymbolNameType) sets these as OTHER. However SymbolNameType does work with non-ASCII characters. So I guess a follow up PR would be to look more into SymbolNameType possibly miscategorizing symbols. |
Sorry, something went wrong.
|
Those methods are intended to mimic the same symbol classification logic in cruby but they have never been exactly the same and have diverged frequently. Every so often I run into a case where we need them to work correctly for some optimization or error check. This is a good step in the right direction. |
Sorry, something went wrong.
isSymbolName was classifying names via a Java String built by RubyEncoding.decodeRaw (byte-as-Latin-1), losing multi-byte character grouping. Use the encoding-aware SymbolNameType when the string is not ASCII instead.
Re-enables TestHash#test_inspect; also drops a stale no-op exclude in TestHashOnly (no test_inspect method exists in that class).