Repository navigation
Pull MinIO from a registry that still serves it - #1559
Merged
dimitri-yatsenko merged 1 commit intoOct 1, 2026
Merged
dimitri-yatsenko merged 1 commit into
dimitri-yatsenko merged 1 commit into
Conversation
Copy link
Copy Markdown
Member
Sorry, something went wrong.
MinIO withdrew its public images. `minio/minio` on Docker Hub was deleted on 2026-09-11 -- the whole repository, not just `latest`, so pinning a tag does not help -- and `quay.io/minio/minio` began requiring authentication on 2026-09-24. Free community builds stopped in October 2025 and the open-source repository was archived in February 2026, so this is a withdrawal rather than an outage: re-running a job will not clear it. Every object-storage suite fails at collection in CI as a result -- test_gc, test_npy_codec, test_codecs, test_attach, test_object -- while passing locally for anyone with the old image still in their Docker cache, which is why this did not show up until CI ran on a clean machine. Switches the fixture to Chainguard's build, which is public, needs no credentials, and is drop-in for testcontainers' MinioContainer. It is also the safer choice on its own merits: the last community image is a year stale and carries unpatched high-severity CVEs, while the Chainguard build is rebuilt continuously. The image is now read from DJ_TEST_MINIO_IMAGE, matching DJ_TEST_MYSQL_IMAGE, so a deployment can point at its own mirror without editing tests. Verified with the local MinIO cache deleted, so the pull is cold as it is in CI: 118 passed across the five object-storage suites.
dimitri-yatsenko
requested review from
MilagrosMarin,
lum-agilyti and
ttngu207
September 30, 2026 22:28
This was referenced Sep 30, 2026
Copy link
Copy Markdown
Member
Author
|
Follow-up filed as #1560 (v2.4): whether to keep depending on MinIO in the test fixture at all. This PR stays the right fix for the release — it changes no test code and is verified cold. #1560 carries the evaluation, including the constraint that decides it: the storage layer goes through fsspec/s3fs rather than boto3, and gc.py walks stores recursively, so isdir and walk ride on s3fs's emulation of directories over list_objects_v2. A thin mock that diverges there will not raise — it will return a different key set and let a GC bug through green. |
Sorry, something went wrong.
ttngu207
approved these changes
Sep 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Reviewers
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Unblocks CI on #1555, #1556 and #1558. This is not a flake and re-running will not clear it — I said otherwise on #1555 before diagnosing it, and that was wrong.
What happened
MinIO withdrew its public images:
Verified directly rather than inferred: while authenticated to Docker Hub and with datajoint/mysql:8.0 pulling fine, every minio/minio tag returns pull access denied ... repository does not exist, the Hub API returns object not found for the repository itself, and quay.io/minio/minio returns 401 while a control public Quay image pulls normally.
Pinning a tag does not help — the repository is gone, not the tag.
Why it surfaced only now
tests/conftest.py hardcoded minio/minio:latest. Anyone with that image already in their Docker cache — I had a copy from Sept 2025 — keeps passing locally while CI, which starts clean, fails at collection on all five object-storage suites: test_gc, test_npy_codec, test_codecs, test_attach, test_object. That is the whole of the 33 errors on the release PRs.
The fix
cgr.dev/chainguard/minio:latest — public, no credentials, drop-in for testcontainers' MinioContainer. Now read from DJ_TEST_MINIO_IMAGE, matching the existing DJ_TEST_MYSQL_IMAGE, so a deployment can point at its own mirror without editing tests.
It is also the safer image on its own merits. The last community build is a year stale (2025-09-07) and carries unpatched high-severity CVEs with no upstream fix coming; the Chainguard build is rebuilt continuously — the copy I tested was built 2026-09-30.
Verification
Deleted the local MinIO images first, so the pull is cold exactly as in CI:
What this does not settle
Whether the test suite should depend on MinIO at all, given the project's direction. Candidates that pull publicly today are LocalStack, SeaweedFS and adobe/s3mock. Worth noting before anyone reaches for the lightest option: DataJoint's storage layer goes through fsspec/s3fs, not boto3, and the GC suite leans on prefix-listing semantics — the area where a thin mock is most likely to diverge. That evaluation belongs in its own issue, not on the eve of a release.
Sources for the timeline: MinIO removal reporting, Quay anonymous-pull change.