FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

bpo-39389: gzip: fix compression level metadata by wchargin · Pull Request #18077 · python/cpython · GitHub

  1. python
  2. cpython

Repository navigation

bpo-39389: gzip: fix compression level metadata #18077

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
serhiy-storchaka merged 2 commits into from
Jan 21, 2020
Conversation 4 Commits 2 Checks 0 Files changed
Merged

bpo-39389: gzip: fix compression level metadata #18077

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
b1f707e
bpo-39389: gzip: fix compression level metadata
wchargin Jan 20, 2020
82ab31f
Add news entry
wchargin Jan 20, 2020
File filter

Filter by extension

Filter by extension .py  (2) All 1 file type selected
Viewed files
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Unified
Split
Hide whitespace
Diff view
Unified
Split
Hide whitespace
Next Next commit
bpo-39389: gzip: fix compression level metadata 
As described in RFC 1952, section 2.3.1, the XFL (eXtra FLags) byte of a
gzip member header should indicate whether the DEFLATE algorithm was
tuned for speed or compression ratio. Prior to this patch, archives
emitted by the `gzip` module always indicated maximum compression.

Test Plan:
Unit tests added; they fail before this patch (in 2 of 3 cases) and pass
after it.

wchargin-branch: gzip-compresslevel-metadata
  • Loading branch information
wchargin committed Jan 20, 2020
commit b1f707eebf8696103d1db4b00e75cb66e68d6fb5
12 changes: 9 additions & 3 deletions Lib/gzip.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ def __init__(self, filename=None, mode=None,
self.fileobj = fileobj

if self.mode == WRITE:
self._write_gzip_header()
self._write_gzip_header(compresslevel)

@property
def filename(self):
Expand All @@ -236,7 +236,7 @@ def _init_write(self, filename):
self.bufsize = 0
self.offset = 0 # Current file offset for seek(), tell(), etc

def _write_gzip_header(self):
def _write_gzip_header(self, compresslevel):
self.fileobj.write(b'\037\213') # magic header
self.fileobj.write(b'\010') # compression method
try:
Expand All @@ -257,7 +257,13 @@ def _write_gzip_header(self):
if mtime is None:
mtime = time.time()
write32u(self.fileobj, int(mtime))
self.fileobj.write(b'\002')
if compresslevel == _COMPRESS_LEVEL_BEST:
xfl = b'\002'
elif compresslevel == _COMPRESS_LEVEL_FAST:
xfl = b'\004'
else:
xfl = b'\000'
self.fileobj.write(xfl)
self.fileobj.write(b'\377')
if fname:
self.fileobj.write(fname + b'\000')
Expand Down
20 changes: 20 additions & 0 deletions Lib/test/test_gzip.py
Show comments
View file
Open in desktop
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Original file line number Diff line number Diff line change
Expand Up @@ -358,6 +358,26 @@ def test_metadata(self):
isizeBytes = fRead.read(4)
self.assertEqual(isizeBytes, struct.pack('<i', len(data1)))

def test_compresslevel_metadata(self):
# see RFC 1952: http://www.faqs.org/rfcs/rfc1952.html
# specifically, discussion of XFL in section 2.3.1
cases = [
('fast', 1, b'\x04'),
('best', 9, b'\x02'),
('tradeoff', 6, b'\x00'),
]
xflOffset = 8

for (name, level, expectedXflByte) in cases:
with self.subTest(name):
fWrite = gzip.GzipFile(self.filename, 'w', compresslevel=level)
with fWrite:
fWrite.write(data1)
with open(self.filename, 'rb') as fRead:
fRead.seek(xflOffset)
xflByte = fRead.read(1)
self.assertEqual(xflByte, expectedXflByte)

def test_with_open(self):
# GzipFile supports the context management protocol
with gzip.GzipFile(self.filename, "wb") as f:
Expand Down
Back | FazBrowse Home | New Git URL