feat: write support for sequences and accessors - #2658
flying-sheep wants to merge 1 commit into
Conversation
Sorry, something went wrong.
Codecov Report❌ Patch coverage is 98.78049% with 1 line in your changes missing coverage. Please review.
@@ Coverage Diff @@
## feat/write-compat-profiles #2658 +/- ##
==============================================================
+ Coverage 85.82% 85.93% +0.11%
==============================================================
Files 50 50
Lines 8182 8243 +61
==============================================================
+ Hits 7022 7084 +62
+ Misses 1160 1159 -1
|
Sorry, something went wrong.
ilan-gold
left a comment
There was a problem hiding this comment.
So I'm not sure I like this design...I'd like to ask in the zarr Zulip about this...
Sorry, something went wrong.
| Together with mappings, | ||
| this allows storing arbitrary JSON-like structures in `uns`, | ||
| with leaves of any encodable type. | ||
|
|
||
| Writing these requires the `"0.14"` {doc}`write compatibility profile <write-compat>`, | ||
| since older anndata versions cannot read them. | ||
|
|
||
| ```python | ||
| >>> store["uns/hetero"].visititems(print) # uns["hetero"] == [1, "b", df] | ||
| 0 <HDF5 dataset "0": shape (), type "<i8"> | ||
| 1 <HDF5 dataset "1": shape (), type "|O"> | ||
| 2 <HDF5 group "/uns/hetero/2" (2 members)> | ||
| 2/_index <HDF5 dataset "_index": shape (2,), type "|O"> | ||
| 2/a <HDF5 dataset "a": shape (2,), type "<i8"> | ||
| ``` |
There was a problem hiding this comment.
My point in #1979 (and I think to some degree the author's) was that we would no longer store a bunch of stuff to disk anymore as arrays that can be more succinctly just be JSON. I think a key in uns called as-json or similar could make sense. Then we dump there whatever the "user wants" to be dumped there for JSON-like access.
Sorry, something went wrong.
|
It might be worth jamming zarr-developers/zarr-extensions#9 through in advance of this if there is appetite from the zarr maintainers (which there seems to be) |
Sorry, something went wrong.
|
Can you link to your zarr zulip post please? /edit: found it #Zarr-Python > json in zarr I think I’d be totally OK with not having heterogeneous sequences but instead a JSON type. The only question is how to decide when to use it. adata.uns["foo"]["ad"] = AnnData(...) stays serializable, so when looking at the .uns["foo"] dict, how do we decide if we store it as JSON or not? And of course that doesn’t mean we don’t need #2662. Regarding zarr-developers/zarr-extensions#9: I’m not looking forward to adding more blockers to the stack. I planned to finish up this PR stack during the conference, then scanpy 1.13 right after. If they can commit to this next week I’d say that’s fine, but I don’t want to wait in case that issue is asleep for a few more months. |
Sorry, something went wrong.
TODO: