FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

feat: advanced indexing by JohannesMessner · Pull Request #1074 · docarray/docarray · GitHub

/ docarray Public

feat: advanced indexing - #1074

Merged
JohannesMessner merged 24 commits into
feat-rewrite-v2from
feat-advanced-indexing
Feb 7, 2023
ConversationCommits (24)ChecksFiles changed
Merged

feat: advanced indexing#1074
JohannesMessner merged 24 commits into
feat-rewrite-v2from
feat-advanced-indexing

Conversation

JohannesMessner commented Feb 2, 2023 •
edited
Loading

Copy link
Copy Markdown
Member

Goals:

PyTorch like indexing into DocumentArray and DocumentArrayStacked

da = DocumentArray([... for _ in range(5)]
da[0]  # index by offset; returns doc at this position
da[0:5:2]  # index by slice; returns every other doc
da[0, 1, 2]  # index by sequence of positions; returns DocumentArray with docs at these positions
da[[0, 1, 2]]  # same as above
da[[True, False, True, True, False]]  # index by boolean mask; returns DocumentArray with "True" docs
da[None]  # returns `da`, just to conform with PyTorch convention

This also includes a refactor where in DocumentArrayStacked ._columns is now separated into _doc_columns and _tensor_columns, instead of having both types in the same dict. It makes everything a bit clearer.

closes #1073

feat: indexing by iterable of ints
61db273 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
fix: accept all iterables as index
dda16e8 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
JohannesMessner added the DocArray v2 This issue is part of the rewrite; not to be merged into main label Feb 2, 2023
JohannesMessner self-assigned this Feb 2, 2023
JohannesMessner changed the base branch from main to feat-rewrite-v2 February 2, 2023 14:11
JoanFM requested changes Feb 2, 2023
View reviewed changes
Comment thread docarray/array/array.py Outdated
feat: index by boolean mask
a193e45 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
feat: allow indexing with torch or numpy
e0546d3 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
feat: add setitem
3c61c91 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
fix: set by mask
023fc07 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
test: add tests
815efe3 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
test: fix some tests
bc1c54b 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
fix: some mypy issues
53282ab 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
JohannesMessner marked this pull request as ready for review February 3, 2023 16:26
samsja reviewed Feb 6, 2023
View reviewed changes
Comment thread docarray/array/array.py Outdated
samsja reviewed Feb 6, 2023
View reviewed changes
Comment thread docarray/array/array.py Outdated
fix: remove uneeded optimization
67718a4 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
JohannesMessner and others added 3 commits February 6, 2023 13:58
fix: index by numpy int type
87a0f8a 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
refactor: make torch available check a util function
2103d6a 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
Merge branch 'feat-rewrite-v2' into feat-advanced-indexing
738b93a 
Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>
github-actions Bot added size/l and removed size/m labels Feb 6, 2023
fix: np indexing
4ca255d 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
Merge remote-tracking branch 'origin/feat-advanced-indexing' into fea…
0241c03 
…t-advanced-indexing

JoanFM commented Feb 6, 2023

Copy link
Copy Markdown
Member

Would u mind adding a description of what it enables?

fix: mypy stuff
5478b0a 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
docs: add docstring
eddf528 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
docs: fix docstring example
0693d35 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
refactor: split columns dict
50984e2 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
anna-charlotte approved these changes Feb 7, 2023
View reviewed changes
Comment thread docarray/array/array_stacked.py
JohannesMessner mentioned this pull request Feb 7, 2023
Closed
47 tasks
JohannesMessner and others added 2 commits February 7, 2023 10:01
docs: tweak docstring
2f2f4f2 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
Merge branch 'feat-rewrite-v2' into feat-advanced-indexing
acabda4 
Signed-off-by: Johannes Messner <44071807+JohannesMessner@users.noreply.github.com>
test: add test for none indexing
27f3167 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
fix: adapt proto to changes
1a058dd 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>
github-actions Bot added size/xl and removed size/l labels Feb 7, 2023

github-actions Bot commented Feb 7, 2023

Copy link
Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

refactor: apply black
eb87fc6 
Signed-off-by: Johannes Messner <messnerjo@gmail.com>

github-actions Bot commented Feb 7, 2023

Copy link
Copy Markdown

This PR exceeds the recommended size of 1000 lines. Please make sure you are NOT addressing multiple issues with one PR. Note this PR might be rejected due to its size.

github-actions Bot commented Feb 7, 2023

Copy link
Copy Markdown

📝 Docs are deployed on https://ft-feat-advanced-indexing--jina-docs.netlify.app 🎉

JohannesMessner merged commit de8f70e into feat-rewrite-v2 Feb 7, 2023
JohannesMessner deleted the feat-advanced-indexing branch February 7, 2023 10:07
JohannesMessner mentioned this pull request Feb 7, 2023
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

samsja samsja left review comments
JoanFM JoanFM approved these changes
+1 more reviewer
anna-charlotte anna-charlotte approved these changes
Reviewers whose approvals may not affect merge requirements

Assignees

JohannesMessner

Labels

area/core area/testing area/typing component/array component/proto DocArray v2 This issue is part of the rewrite; not to be merged into main size/xl

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

toch/np like indexig for DocumentArrayStacked

4 participants

Back | FazBrowse Home | New Git URL