FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

feat: add BaseDocWoId by samsja · Pull Request #1803 · docarray/docarray · GitHub

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension .lock  (1) .py  (1) .toml  (1) All 3 file types selected
Only manifest files
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
71 changes: 40 additions & 31 deletions docarray/base_doc/doc.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -61,39 +61,14 @@
ExcludeType = Optional[Union['AbstractSetIntStr', 'MappingIntStrAny']]


class BaseDoc(BaseModel, IOMixin, UpdateMixin, BaseNode):
class BaseDocWithoutId(BaseModel, IOMixin, UpdateMixin, BaseNode):
"""
BaseDoc is the base class for all Documents. This class should be subclassed
to create new Document types with a specific schema.

The schema of a Document is defined by the fields of the class.

Example:
```python
from docarray import BaseDoc
from docarray.typing import NdArray, ImageUrl
import numpy as np


class MyDoc(BaseDoc):
embedding: NdArray[512]
image: ImageUrl


doc = MyDoc(embedding=np.zeros(512), image='https://example.com/image.jpg')
```


BaseDoc is a subclass of [pydantic.BaseModel](
https://docs.pydantic.dev/usage/models/) and can be used in a similar way.
BaseDocWoId is the class behind BaseDoc, it should not be used directly unless you know what you are doing.
Comment thread
samsja marked this conversation as resolved.
It is basically a BaseDoc without the ID field.
!!! warning
This class cannot be used with DocumentIndex. Only BaseDoc is compatible
"""

id: Optional[ID] = Field(
description='The ID of the BaseDoc. This is useful for indexing in vector stores. If not set by user, it will automatically be assigned a random value',
default_factory=lambda: ID(os.urandom(16).hex()),
example=os.urandom(16).hex(),
)

if is_pydantic_v2:

class Config:
Expand Down Expand Up @@ -545,7 +520,7 @@ def parse_raw(
:param allow_pickle: allow pickle protocol
:return: a document
"""
return super(BaseDoc, cls).parse_raw(
return super(BaseDocWithoutId, cls).parse_raw(
b,
content_type=content_type,
encoding=encoding,
Expand Down Expand Up @@ -582,3 +557,37 @@ def _exclude_docarray(
)

to_json = BaseModel.model_dump_json if is_pydantic_v2 else json


class BaseDoc(BaseDocWithoutId):
"""
BaseDoc is the base class for all Documents. This class should be subclassed
to create new Document types with a specific schema.

The schema of a Document is defined by the fields of the class.

Example:
```python
from docarray import BaseDoc
from docarray.typing import NdArray, ImageUrl
import numpy as np


class MyDoc(BaseDoc):
embedding: NdArray[512]
image: ImageUrl


doc = MyDoc(embedding=np.zeros(512), image='https://example.com/image.jpg')
```


BaseDoc is a subclass of [pydantic.BaseModel](
https://docs.pydantic.dev/usage/models/) and can be used in a similar way.
"""

id: Optional[ID] = Field(
description='The ID of the BaseDoc. This is useful for indexing in vector stores. If not set by user, it will automatically be assigned a random value',
default_factory=lambda: ID(os.urandom(16).hex()),
example=os.urandom(16).hex(),
)
35 changes: 17 additions & 18 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion pyproject.toml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Original file line number Diff line number Diff line change
Expand Up @@ -105,7 +105,8 @@ pytest-asyncio = ">=0.20.2"


[tool.poetry.group.docs.dependencies]
mkdocstrings = {extras = ["python"], version = ">=0.20.0"}
mkdocstrings = {extras = ["python"], version = ">=0.23.0"}
mkdocstrings-python= ">=1.7.0"
mkdocs-material= ">=9.1.2"
mkdocs-awesome-pages-plugin = ">=2.8.0"
mktestdocs= ">=0.2.0"
Expand Down

Back | FazBrowse Home | New Git URL