| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
…orting of mixins Also add regression tests for multi-key sorting and warnings on 'kind' argument.
|
Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.
|
Sorry, something went wrong.
There was a problem hiding this comment.
@taldcroft - the crucial bits are great, and my comments are just a request to reconsider the warning added, and one little thing that can be removed.
Sorry, something went wrong.
| t = Table() | ||
| t["a"] = [2, 1, 3, 2, 3, 1] | ||
| a_struct = t["a"].copy() | ||
| with warnings.catch_warnings(): |
There was a problem hiding this comment.
This is the default when we run tests, so no need to add
(aside: can do catch_warnings(action="error"))
Sorry, something went wrong.
| The column name(s) to order the table by | ||
| kind : {'quicksort', 'mergesort', 'heapsort', 'stable'}, optional | ||
| Sorting algorithm used by ``numpy.argsort``. | ||
| Sorting algorithm used by `numpy.argsort`. This is only used |
There was a problem hiding this comment.
It is not a big deal, but my sense would be never to warn and here just write
Passed on to `numpy.argsort` if sorting is on a single column, ignored for sorting on multiple columns (including mixin columns that expand into multiple sortable arrays, like |Time|).
My logic is that for sorting a warning is really not that useful, and for people who, say write a package that wraps table sorting, and who just want stable sorts (which is not the default for argsort), it is nice to be able to give it and not worry about how many columns are going to be sorted.
Note that np.argsort is not that different; the kind parameter is only a request (and mergesort doesn't really exist as a separate option anymore):
Sorting algorithm. The default is 'quicksort'. Note that both 'stable'
and 'mergesort' use timsort under the covers and, in general, the
actual implementation will vary with data type. The 'mergesort' option
is retained for backwards compatibility.
Sorry, something went wrong.
| kwargs = {"kind": kind} if kind else {} | ||
| idx = np.argsort(sortable_arrays[0], **kwargs) | ||
| else: | ||
| if kind is not None and kind not in ("stable", "mergesort"): |
There was a problem hiding this comment.
So, I'd remove the warning here (which obviously simplifies the tests too).
Sorry, something went wrong.
| jd1, jd2 = parent.jd1, parent.jd2 | ||
| jd_approx = jd1 + jd2 | ||
| jd_remainder = (jd1 - jd_approx) + jd2 | ||
| return [jd_approx, jd_remainder] |
There was a problem hiding this comment.
I really don't know why we didn't do this before!
Sorry, something went wrong.
|
@mhvk - thanks for the review. I implemented your suggestion to remove the warning, excellent idea. |
Sorry, something went wrong.
|
FYI the codecov/patch 75% of diff hit is wrong. It shows no coverage for code that is obviously run, and I explicitly confirmed that the exception handling is also covered. So I don't know why codecov is busted like that. |
Sorry, something went wrong.
|
When did you last rebase? Might help. |
Sorry, something went wrong.
|
I see you have enabled auto-merge but it is not the CI blocking the merge. This PR needs at least one approval. FYI. |
Sorry, something went wrong.
There was a problem hiding this comment.
All ok now!
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Description
This PR use col.info.get_sortable_arrays() in Table.argsort() for efficient mixin sorting. This fixes the first bullet point of #14942.
Rationale
Table.argsort() (and hence Table.sort()) currently sorts by converting the key columns to a NumPy array via Table.as_array(). For a single plain key column this is just the column data, but for multiple key columns it builds a structured array over all of them, and if any key is a mixin column (like ~astropy.time.Time) that can't be represented as native structured-array dtype, it gets stored as a slow Python object array. Every comparison during the sort then falls back to Python-level object comparisons instead of vectorized NumPy operations, which is extremely slow — see the benchmark below, where combining a Time column with a plain int column is over 1000x slower than sorting on the int column alone.
get_sortable_arrays() already exists as an info method (used internally by table joins) that returns one or more plain NumPy arrays which can be lexically sorted to reproduce the sort order of a column, without needing an object array. For a normal Column this is just [self.data]. For a Time column it is two float arrays (jd1/jd2 split into an approximate value and a remainder) that fully capture time ordering without loss of precision.
This PR reworks Table.argsort() to call get_sortable_arrays() on each key column and flatten the results into a single list of arrays. If there is only one sortable array overall (e.g. a single plain-column key), np.argsort is used directly, preserving support for the kind argument. Otherwise (multiple keys, and/or a mixin column that expands into multiple arrays) np.lexsort is used, which is always stable and does not accept kind; in that case a non-default, non-stable kind is ignored and triggers an AstropyUserWarning.
If a column's get_sortable_arrays() raises NotImplementedError (e.g. SkyCoord, which has no well-defined sort order), Table.argsort() now raises TypeError rather than falling through to a slow/incorrect object array comparison.
As part of this work, Time.info.get_sortable_arrays() was also optimized: it previously computed the remainder term by constructing a new Time object and subtracting, which is roughly as expensive as computing a full Time.argsort(). It now uses the same cheap jd1/jd2 float arithmetic that Time.argsort() already uses internally, avoiding that redundant object construction.
Changes
Backwards compatibility notes
Performance
Table.argsort() performance: main vs table-unified-sort
Benchmark script: perf_test_argsort_time.py
N = 100,000 rows, 5 repeats per case, median times reported.
Results
Key findings
AI disclosure
Essentially all of this PR was generated using GitHub CoPilot with Claude Sonnet 5. I reviewed everything carefully and understand all the changes.