FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

feat: validate file formats in url by makram93 · Pull Request #1606 · docarray/docarray · GitHub

/ docarray Public

feat: validate file formats in url - #1606

Merged
jupyterjazz merged 4 commits into
docarray:feat-file-validationfrom
makram93:feat-file-format-validation
Jun 26, 2023
ConversationCommits (4)ChecksFiles changed
Merged

feat: validate file formats in url#1606
jupyterjazz merged 4 commits into
docarray:feat-file-validationfrom
makram93:feat-file-format-validation

Conversation

makram93 commented May 31, 2023 •
edited
Loading

Copy link
Copy Markdown
Contributor

Solves: #1555

Solution:

  • rely on the mime-types library to detect the mime type from the URL and raise an error if it is not the expected type for that class
  • if auto-detection of mime-type fails then we check if the file ending is among the extra extensions
    • Check for exceptional cases, if there is still no match then raise the error

CAVEAT: Any file without an extension will raise an error, for example, a wav audio file with no extension will raise an error.

fix: validate file formats in url
5243e23 
Signed-off-by: Mohammad Kalim Akram <kalim.akram@jina.ai>
makram93 changed the title fix: validate file formats in url feat: validate file formats in url May 31, 2023

JoanFM commented May 31, 2023

Copy link
Copy Markdown
Member

I think doing a request to the URL is an overkill and it will make it very slow

JoanFM requested changes May 31, 2023
View reviewed changes
Comment thread docarray/typing/url/any_url.py Outdated
JoanFM linked an issue Jun 14, 2023 that may be closed by this pull request
Url types are not aware of extension during validation #1555
Closed
makram93 and others added 3 commits June 23, 2023 03:11
Merge branch 'docarray:main' into feat-file-format-validation
7e52fff
fix: apply review suggestions
aa07312 
Signed-off-by: Mohammad Kalim Akram <kalim.akram@jina.ai>
fix: mypy and unit tests
a7f515d 
Signed-off-by: Mohammad Kalim Akram <kalim.akram@jina.ai>
jupyterjazz changed the base branch from main to feat-file-validation June 26, 2023 08:59
jupyterjazz requested a review from JoanFM June 26, 2023 09:02
jupyterjazz marked this pull request as ready for review June 26, 2023 09:07
jupyterjazz merged commit 49fd592 into docarray:feat-file-validation Jun 26, 2023
jupyterjazz mentioned this pull request Jun 26, 2023
Merged
JoanFM pushed a commit that referenced this pull request Jun 27, 2023
feat: validate file formats in url (#1606) (#1669)
e0e5cd8 
Signed-off-by: Mohammad Kalim Akram <kalim.akram@jina.ai>
Signed-off-by: jupyterjazz <saba.sturua@jina.ai>
Co-authored-by: Mohammad Kalim Akram <kalim.akram@jina.ai>
JoanFM mentioned this pull request Jul 3, 2023
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

JoanFM JoanFM approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Url types are not aware of extension during validation

3 participants

Back | FazBrowse Home | New Git URL