Commit 553308c
committed
BUG: Fix weak hash function in np.isin().
In a build of NumPy against libcxx, we found the following code
regressed from almost instantaneous with NumPy 2.3 to over 40s with
NumPy 2.4.
```
import numpy as np
full_list = np.array("2015-12-01T00:00:00.000000000", 'datetime64[ns]') + np.arange(162143) * np.timedelta64(4, 'h').astype('timedelta64[ns]')
sampled_dates = full_list[10:28]
np.isin(sampled_dates, full_list)
```
Our belief is the following:
* std::unordered_set in libcxx uses power of two hash buckets.
* std::hash is the identity function.
* in this particular example, we are hashing integers (datetime64
values) separated by multiples of a power of two.
* the net result is that all of the integers end up in the same hash
bucket.
We can make the code more robust simply by using the same npy_fnv1a hash
used elsewhere in the same file since it will do a better job of
distributing hash bits.1 parent 75a7eba commit 553308c
1 file changed
Lines changed: 1 addition & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
@@ -63,7 +63,7 @@ empty_array_like(PyArrayObject *arr, npy_intp length) | |||
| 63 | 63 | ||
| 64 | 64 | template <typename T> | |
| 65 | 65 | size_t hash_integer(const T *value, npy_bool equal_nan) { | |
| 66 | - return std::hash<T>{}(*value); | ||
| 66 | + return npy_fnv1a(reinterpret_cast<const unsigned char*>(value), sizeof(T)); | ||
| 67 | 67 | } | |
| 68 | 68 | ||
| 69 | 69 | template <typename S, typename T, S (*real)(T), S (*imag)(T)> | |
0 commit comments