gh-51067: add `ZipFile.remove()` by danny0838 · Pull Request #103033 · python/cpython · GitHub

This is a revision of #19358 (for issue #51067) as the original author seems not keeping working.

Notable changes:

Added docs and tests.
Support mode 'w' and 'x', as noted by remove/delete method for zipfile objects #51067 (comment)
Support removing multiple members and removing non-physically with the internal _remove_members method, as they may be used by some interested people, as noted by remove/delete method for zipfile objects #51067 (comment) and remove/delete method for zipfile objects #51067 (comment).

They are not currently introduced in the public remove API, as it would involve more complicated changes to the public APIs (e.g. introducing error handling for multiple members, and a extra method that purges stale data by non-physical removing) and other ZipFile related APIs do not support similar operations.
Move physical data in chunks, to prevent a memory issue for large files.
Fixed a flaw of the previous implementation that self.NameToInfo gets a missing key when removing one of duplicated arcnames.

Issue: remove/delete method for zipfile objects #51067

📚 Documentation preview 📚: https://cpython-previews--103033.org.readthedocs.build/

All commit authors signed the Contributor License Agreement.

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

Automatically compacting the .zip file seems like overkill. Suggestion: split this into two methods:

ZipInfo.remove() removes the record for the file, and by default zeroes out its data.
ZipFile.repack() reclaims free space.

The latter method is dangerous for self-extracting .zip files, which have an executable header before the the zip data begins. That header would be stripped out, I think.

Also, please don't force-push to an open PR. It makes it harder for reviewers to follow changes! Thanks

Refer to: python/cpython#103033

* Add remove method to ZipFile Refer to: python/cpython#103033 * Make use of `ZipFileWithRemove`

All commit authors signed the Contributor License Agreement.

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

This is a revision of commit 659eb04 (PR python#19358), notably with following changes: - Add documentation and tests. - Raise `ValueError` for a bad mode, as in other methods. - Support multi-member removal in `_remove_members()`. - Support non-physical removal in `_remove_members()`. - Move physical file data in chunks to prevent excessive memory usage on large files. - Fix missing entry in `self.NameToInfo` when removing a duplicated archive name. - Also update `ZipInfo._end_offset` for physically moved files. Co-authored-by: Éric <merwok@netwok.org> (cherry picked from commit e6bc82a (PR python#103033))

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

This is a revision of commit 659eb04 (PR python#19358), notably with following changes: - Add documentation and tests. - Raise `ValueError` for a bad mode, as in other methods. - Support multi-member removal in `_remove_members()`. - Support non-physical removal in `_remove_members()`. - Move physical file data in chunks to prevent excessive memory usage on large files. - Fix missing entry in `self.NameToInfo` when removing a duplicated archive name. - Also update `ZipInfo._end_offset` for physically moved files. Co-authored-by: Éric <merwok@netwok.org> (cherry picked from commit e6bc82a (PR python#103033))

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

The PR is being old and the base python executable can be hardly compiled. Rebased onto the latest Python and make sure it builds, with few trivial commits squashed together.

This is a revision of commit 659eb04 (PR python#19358), notably with following changes: - Add documentation and tests. - Raise `ValueError` for a bad mode, as in other methods. - Support multi-member removal in `_remove_members()`. - Support non-physical removal in `_remove_members()`. - Move physical file data in chunks to prevent excessive memory usage on large files. - Fix missing entry in `self.NameToInfo` when removing a duplicated archive name. - Also update `ZipInfo._end_offset` for physically moved files. Co-authored-by: Éric <merwok@netwok.org> (cherry picked from commit e6bc82a (PR python#103033))

- File is not truncated in mode 'w'/'x', which results non-shrinked file. - This cannot be simply resolved by adding truncation for mode 'w'/'x', which may be used on an unseekable file buffer and truncation is not allowed.

- The seek will be automatically called in `ZipFile.close`.

Rebased to use the same base as PR #134627 for easier comparison.

In favor of #134627, I'm probably not going to keep working on this PR, unless it proves to be the final accepted approach for the issue.

NOTE: Compared to #134627, this PR is rather quick and naive, and some cares may need to be aware of:

Overlapping entries are not checked. It may be unsafe to call remove on a ZIP file having such entries.
When stripping a local file entry, the algorithm simply strips bytes between its starting and next local file entry's starting instead of actually calculate its size. This means extra bytes after the entry will be removed, which may be a problem in some cases. (See c80d21b for details)

Unfortunately, fixing any of them requires a non-trivial code rework.

This PR is stale because it has been open for 30 days with no activity.

Co-authored-by: Éric <merwok@netwok.org>

#134627 implemented a version of this.

bedevere-bot mentioned this pull request Mar 25, 2023

remove/delete method for zipfile objects #51067

Closed

bedevere-bot added the awaiting review label Mar 25, 2023

danny0838 force-pushed the gh-51067 branch from 2219d90 to 314916e Compare March 25, 2023 11:31

danny0838 force-pushed the gh-51067 branch 3 times, most recently from 76722fa to f3450f1 Compare March 26, 2023 12:37

arhadthedev added the stdlib Standard Library Python modules in the Lib/ directory label Mar 30, 2023

bpepple added a commit to Metron-Project/darkseid that referenced this pull request Aug 31, 2024

Add remove method to ZipFile

343c935

Refer to: python/cpython#103033

bpepple added a commit to Metron-Project/darkseid that referenced this pull request Aug 31, 2024

Add remove method to ZipFile (#111 )

c779b46

* Add remove method to ZipFile Refer to: python/cpython#103033 * Make use of `ZipFileWithRemove`

merwok reviewed Apr 24, 2025

View reviewed changes

Comment thread Doc/library/zipfile.rst Outdated

Comment thread Lib/zipfile/__init__.py

Comment thread Lib/zipfile/__init__.py Outdated

merwok added the type-feature A feature request or enhancement label Apr 24, 2025

danny0838 force-pushed the gh-51067 branch from 692260e to 4556a74 Compare May 22, 2025 12:36

danny0838 force-pushed the gh-51067 branch from 4556a74 to 7632c7b Compare May 22, 2025 12:45

danny0838 force-pushed the gh-51067 branch from a43b29f to f5325ad Compare May 22, 2025 12:51

danny0838 changed the title gh-51067: add ZipInfo.remove() gh-51067: add ZipFile.remove() May 22, 2025

danny0838 mentioned this pull request May 24, 2025

gh-51067: Add remove() and repack() to ZipFile #134627

Merged

merwok reviewed Jun 19, 2025

View reviewed changes

Comment thread Misc/NEWS.d/next/Core_and_Builtins/2025-05-22-12-52-35.gh-issue-51067.tJxGGF.rst Outdated

danny0838 and others added 10 commits June 19, 2025 23:44

Fix and improve tests

96a07a2

Update ZipInfo._end_offset

602b909

No more allow mode 'w'/'x'

bb90f48

- File is not truncated in mode 'w'/'x', which results non-shrinked file. - This cannot be simply resolved by adding truncation for mode 'w'/'x', which may be used on an unseekable file buffer and truncation is not allowed.

Test for zip64 more thoroughly

2d74501

Add tests for zstd

63d56e5

Remove unneeded seek

ab3a187

- The seek will be automatically called in `ZipFile.close`.

Add lock for thread safety

1c074c5

📜🤖 Added by blurb_it.

8b4f709

Suppress duplicated name warning during generating test files

7b7f54a

Fix rst format

c95ef53

danny0838 force-pushed the gh-51067 branch from 0d82d00 to c95ef53 Compare June 19, 2025 15:53

github-actions Bot added the stale Stale PR or inactive for long period of time. label Apr 12, 2026

Apply suggestion from @merwok

6418f91

Co-authored-by: Éric <merwok@netwok.org>

github-actions Bot removed the stale Stale PR or inactive for long period of time. label May 12, 2026

gpshead closed this Jun 20, 2026

github-project-automation Bot moved this to Done in Zipfile issues 📦 Jun 20, 2026

danny0838 deleted the gh-51067 branch June 21, 2026 17:05

gh-51067: add `ZipFile.remove()` by danny0838 · Pull Request #103033 · python/cpython · GitHub

Repository navigation

gh-51067: add ZipFile.remove() - #103033

gh-51067: add ZipFile.remove()#103033danny0838 wants to merge 12 commits intopython:mainpython/cpython:mainfrom danny0838:gh-51067danny0838/cpython:gh-51067Copy head branch name to clipboard

Conversation

danny0838 commented Mar 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Mar 25, 2023 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-bot commented Mar 25, 2023

Uh oh!

bedevere-bot commented Mar 25, 2023

Uh oh!

barneygale commented Mar 30, 2023

Uh oh!

barneygale commented Mar 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

python-cla-bot Bot commented May 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-app Bot commented May 22, 2025

Uh oh!

bedevere-app Bot commented May 22, 2025

Uh oh!

bedevere-app Bot commented May 22, 2025

Uh oh!

danny0838 commented May 22, 2025

Uh oh!

Uh oh!

danny0838 commented Jun 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Apr 12, 2026

Uh oh!

gpshead commented Jun 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

gh-51067: add ZipFile.remove()#103033
danny0838 wants to merge 12 commits into
python:mainfrom
danny0838:gh-51067

danny0838 commented Mar 25, 2023 •
edited
Loading

ghost commented Mar 25, 2023 •
edited by ghost
Loading

barneygale commented Mar 30, 2023 •
edited
Loading

python-cla-bot Bot commented May 22, 2025 •
edited
Loading

danny0838 commented Jun 19, 2025 •
edited
Loading