FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

docs: explain how to set document config by JohannesMessner · Pull Request #1773 · docarray/docarray · GitHub

/ docarray Public

docs: explain how to set document config #1773

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JohannesMessner merged 2 commits into from
Sep 5, 2023
Conversation 6 Commits 2 Checks 0 Files changed
Merged

docs: explain how to set document config #1773

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ea6e9bd
docs: explain how to set document config
JohannesMessner Sep 5, 2023
d587a6a
test: test config subclassing
JohannesMessner Sep 5, 2023
File filter

Filter by extension

Filter by extension .md  (1) .py  (1) All 2 file types selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
15 changes: 15 additions & 0 deletions docs/user_guide/representing/first_step.md
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,21 @@ This representation can be used to [send](../sending/first_step.md) or [store](.

[BaseDoc][docarray.base_doc.doc.BaseDoc] can be nested to represent any kind of data hierarchy.

## Setting a Pydantic `Config` class

Documents support setting a `Config` [like any other Pydantic `BaseModel`](https://docs.pydantic.dev/latest/usage/model_config/).

However, if you set a config, you should inherit from the `BaseDoc` config class:

```python
from docarray import BaseDoc


class MyDoc(BaseDoc):
class Config(BaseDoc.Config):

deepankarm Sep 5, 2023

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Minor comment. In pydantic v1, we only had to pass a class Config in the model. It didn't need to be inherited from another config class.

JohannesMessner Sep 5, 2023

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Choose a reason Spam Abuse Off Topic Outdated Duplicate Resolved Low Quality

Yes, in pydantic that is not needed. But we set some pydantic configs in BaseDoc, so to preserve them, inheritance is needed. We could probably find some magic way around this, but until then this is the safest option imo.
arbitrary_types_allowed = True # just an example setting
```

See also:

* The [next part](./array.md) of the representing section
Expand Down
31 changes: 26 additions & 5 deletions tests/units/document/test_any_document.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
@@ -1,10 +1,14 @@
from typing import Dict, List

import numpy as np
import pytest
from typing import Dict, List
from orjson import orjson

from docarray import DocList
from docarray.base_doc import AnyDoc, BaseDoc
from docarray.base_doc.io.json import orjson_dumps_and_decode
from docarray.typing import NdArray
from docarray.typing.tensor.abstract_tensor import AbstractTensor


def test_any_doc():
Expand Down Expand Up @@ -36,7 +40,7 @@ class InnerDoc(BaseDoc):
class DocTest(BaseDoc):
text: str
tags: Dict[str, int]
l: List[int]
l_: List[int]
d: InnerDoc
ld: DocList[InnerDoc]

Expand All @@ -46,14 +50,14 @@ class DocTest(BaseDoc):
DocTest(
text='type1',
tags={'type': 1},
l=[1, 2],
l_=[1, 2],
d=inner_doc,
ld=DocList[InnerDoc]([inner_doc]),
),
DocTest(
text='type2',
tags={'type': 2},
l=[1, 2],
l_=[1, 2],
d=inner_doc,
ld=DocList[InnerDoc]([inner_doc]),
),
Expand All @@ -71,7 +75,7 @@ class DocTest(BaseDoc):
for i, d in enumerate(aux):
assert d.tags['type'] == i + 1
assert d.text == f'type{i + 1}'
assert d.l == [1, 2]
assert d.l_ == [1, 2]
if protocol == 'proto':
assert isinstance(d.d, AnyDoc)
assert d.d.text == 'I am inner' # inner Document is a Dict
Expand All @@ -89,3 +93,20 @@ class DocTest(BaseDoc):
assert isinstance(d.ld[0], dict)
assert d.ld[0]['text'] == 'I am inner'
assert d.ld[0]['t'] == {'a': 'b'}


def test_subclass_config():
class MyDoc(BaseDoc):
x: str

class Config(BaseDoc.Config):
arbitrary_types_allowed = True # just an example setting

assert MyDoc.Config.json_loads == orjson.loads
assert MyDoc.Config.json_dumps == orjson_dumps_and_decode
assert (
MyDoc.Config.json_encoders[AbstractTensor](3) == 3
) # dirty check that it is identity
assert MyDoc.Config.validate_assignment
assert not MyDoc.Config._load_extra_fields_from_protobuf
assert MyDoc.Config.arbitrary_types_allowed
Back | FazBrowse Home | New Git URL