| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
### Motivation `JdkZlibDecompressor#processOutput` sizes every output buffer as `inflater.getRemaining() << 1`. The number of remaining input bytes is only a hint: the inflater may still hold decoded data that did not fit into the previous output buffer, and by then it may have pulled the last input bytes into its internal state, so `getRemaining()` is `0` and the proposed buffer has zero capacity. `inflate(...)` cannot make progress with it, and because `status()` maps `inflater.needsInput()` straight to `NEED_INPUT`, the caller is asked for more input instead of draining the pending output. At the end of the stream `endOfInput()` then fails with `Compressed stream ended before the end-of-stream marker`, even though the stream was complete and valid. Measured with 100000 bytes of `'a'` deflated raw (115 bytes, ratio ~870:1): only 99848 bytes are produced, the remaining 152 bytes stay inside the `Inflater`, and the call throws. The legacy `JdkZlibDecoder` decodes the same bytes to all 100000, because it inflates into one buffer that it keeps expanding, so `inflate(...)` always has room. `ZlibWrapper.ZLIB` and `ZlibWrapper.GZIP` hide the problem, as their trailer keeps `getRemaining()` above zero until the inflater is finished. ### Modification - Never propose a zero-sized output buffer: `processOutput` floors the proposed capacity at `MIN_OUTPUT_BUFFER_SIZE` (512 bytes). `maxAllocation` still caps it exactly as before. - Remember whether the last `inflate(...)` filled the output buffer completely and, if it did, report `NEED_OUTPUT` from `status()`, so pending output is drained before more input is requested. `needsInput()` only says that all input bytes were consumed, not that all output was produced. ### Result A valid deflate stream is fully decompressed regardless of its compression ratio.
|
@renechoi did you sign our icla ? https://netty.io/s/icla |
Sorry, something went wrong.
There was a problem hiding this comment.
one nit.
Sorry, something went wrong.
Motivation: Review feedback on netty#17191: JdkZlibDecompressorTest#jdkDecompress closed the InflaterInputStream it read from but left the ByteArrayOutputStream it wrote into open. Modification: Close the output stream as well. Result: No behaviour change; the helper now closes both streams it owns.
|
Signed the ICLA just now, thanks. |
Sorry, something went wrong.
✅ All tests passed ✅🏷️ Commit: 5285742 Learn more about TestLens at testlens.app. |
Sorry, something went wrong.
|
Could not create auto-port PR. |
Sorry, something went wrong.
…ressible streams (#17196) Auto-port of #17191 to 5.0 Cherry-picked commit: 7681aff --- ### Motivation `JdkZlibDecompressor#processOutput` sizes every output buffer as `inflater.getRemaining() << 1`. The number of remaining input bytes is only a hint: the inflater may still hold decoded data that did not fit into the previous output buffer, and by then it may have pulled the last input bytes into its internal state, so `getRemaining()` is `0` and the proposed buffer has zero capacity. `inflate(...)` cannot make progress with it, and because `status()` maps `inflater.needsInput()` straight to `NEED_INPUT`, the caller is asked for more input instead of draining the pending output. At the end of the stream `endOfInput()` then fails with `Compressed stream ended before the end-of-stream marker`, even though the stream was complete and valid. Measured with 100000 bytes of `'a'` deflated raw (115 bytes, ratio ~870:1), driving the decompressor exactly as its own tests do: ``` new JdkZlibDecompressor (ZlibWrapper.NONE): 99848 of 100000 bytes, then DecompressionException: Compressed stream ended before the end-of-stream marker legacy JdkZlibDecoder (ZlibWrapper.NONE): 100000 bytes java.util.zip.Inflater : 100000 bytes ``` Replaying the same buffer sizing against a bare `Inflater` shows the state that is mis-read: ``` iteration 2148: inflate returned 0 cap=0 getRemaining=0 needsInput=true finished=false producedSoFar=99848 drained with a 64K buffer afterwards: 152B, finished=true ``` So the tail is inside the `Inflater` the whole time and only the buffer sizing keeps it there. The legacy `JdkZlibDecoder` is not affected because it inflates into a single buffer that it keeps expanding, so `inflate(...)` always has room. `ZlibWrapper.ZLIB` and `ZlibWrapper.GZIP` hide the problem as well, because their trailer keeps `getRemaining()` above zero until the inflater is finished; raw deflate has no trailer, which is what `permessage-deflate` and `Content-Encoding: deflate` payloads look like. ### Modification - Never propose a zero-sized output buffer: `processOutput` floors the proposed capacity at `MIN_OUTPUT_BUFFER_SIZE` (512 bytes). `maxAllocation` still caps it exactly as before, so a configured limit keeps its meaning. - Remember whether the last `inflate(...)` filled the output buffer completely and, if it did, report `NEED_OUTPUT` from `status()` so the pending output is drained before more input is requested. `needsInput()` only says that all input bytes were consumed, not that all output was produced. The flag is cleared once a call does not fill the buffer or the inflater is finished, so this cannot loop. ### Result A valid deflate stream is fully decompressed regardless of its compression ratio. **Verification done:** added `JdkZlibDecompressorTest#testHighlyCompressibleStreamIsFullyDecompressed`. It runs for all three wrappers, cross-checks the compressed input with the JDK inflater first so a failure cannot be blamed on the fixture, and then asserts the decompressor returns the same bytes. It fails on unpatched `4.2` for `ZlibWrapper.NONE` with the exception above and passes with this change. `./mvnw -pl codec-compression clean install` is green on JDK 21 (499 tests, checkstyle, forbidden-apis and revapi included). I also ran the other `Decompressor` implementations through the same battery (round trip at chunk sizes 1..64K, `addInput` ownership on malformed input, `close()` idempotency, concatenated frames, and hand-crafted gzip headers covering every `FLG` combination against `GZIPInputStream`) and found no other divergence, so this change is limited to the zlib one. Co-authored-by: renechoi <115696395+renechoi@users.noreply.github.com>
Sorry, something went wrong.
|
No, this is only for the new implementation which does not exist in 4.1 |
Sorry, something went wrong.
|
@yawkat so the old implementation does not have the same logical bug ? |
Sorry, something went wrong.
|
According to the author no, because the decoder doesn't really stream the output like the decompressor does. But I haven't double checked. |
Sorry, something went wrong.
|
Short answer: yes, and I need to correct my own PR description. I wrote there that the legacy JdkZlibDecoder "inflates into a single buffer that it keeps expanding, so inflate(...) always has room". That is wrong. JdkZlibDecoder has the same logical bug, it just truncates silently instead of throwing. And it is not really a 4.1 question: the same class is on 4.2 and 5.0, and its decode(...) body is byte-identical on all three. Current 4.2 (035d76e, so with this PR already in), raw deflate of 'a' repeated: size=65537 chunk=all compressed=80 decoded=65536 lost=1 isTruncatedPrefix=true error=none size=33333 chunk=7 compressed=50 decoded=33286 lost=47 isTruncatedPrefix=true error=none size=100000 chunk=all compressed=115 decoded=100000 lost=0 isTruncatedPrefix=true error=none Nothing is thrown, checkException() is empty, and the output is a clean prefix of the input, so a peer just sees a short body. The last line is the payload I used in this PR, which is why I read the legacy path as unaffected. What saves it most of the time is that prepareDecompressBuffer(ctx, decompressed, ...) only ever grows an existing buffer, and ByteBuf growth is geometric, so there is normally slack left. That protection is gone whenever the buffer is reallocated from null, because then the capacity is inflater.getRemaining() << 1 again. Instrumented tail of the 65537 case on 4.1 f4f1b9d: ITER cap=65536 writable=32768 out=32768 remainingIn=0 needsInput=true finished=false REALLOC fresh=true asked=0 cap=0 writable=0 -> loopContinues=false The buffer had just gone to ctx.fireChannelRead(...) because it reached maxForwardBytes, so decompressed was null, the fresh one gets capacity 0, and while (!inflater.needsInput()) ends the loop with finished == false and the tail still inside the inflater. The second way in needs no threshold at all, since every decode(...) call starts from null as well. That is the 33333-byte line above, arriving in 7-byte reads. Scope, measured on 4.1 f4f1b9d over 88 payload sizes x {NONE, ZLIB, GZIP} x {single write, 1-byte reads, 7-byte reads} x {maxAllocation 0, 1 MiB}, so 1584 combinations: 87 truncate, spread over 35 distinct sizes, all of them ZlibWrapper.NONE. ZLIB and GZIP stay correct because their trailer keeps bytes in the input buffer while output is still pending. It also predates #16532: the same sweep on df65997 gives 168. Reachability, on 4.2 035d76e:
So there is nothing to cherry-pick to 4.1, codec-compression and the Decompressor API only start at 4.2, but JdkZlibDecoder does need its own fix, and on all three branches: stop reading needsInput() as "no output left", and never hand inflate(...) a zero-capacity buffer. I can put that up against 4.1 with a regression test if you want it done that way. |
Sorry, something went wrong.
|
@renechoi yes please provide a PR :) Thanks! |
Sorry, something went wrong.
…l [skip ci] Bumps [io.netty:netty-all](https://github.com/netty/netty) from 4.2.16.Final to 4.2.17.Final. Release notes *Sourced from [io.netty:netty-all's releases](https://github.com/netty/netty/releases).* > netty-4.2.17.Final > ------------------ > > What's Changed > -------------- > > * AsciiString.cached(String) should sanitize the provided String ([#13749](https://redirect.github.com/netty/netty/issues/13749)) by [`@vpelikh`](https://github.com/vpelikh) in [netty/netty#17007](https://redirect.github.com/netty/netty/pull/17007) > * Fix deploy workflow by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17071](https://redirect.github.com/netty/netty/pull/17071) > * Fix AsciiString.cached(String) performance regression by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17074](https://redirect.github.com/netty/netty/pull/17074) > * SslHandler: Fix possible buffer leak when an OOME is thrown during allocation by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17059](https://redirect.github.com/netty/netty/pull/17059) > * Avoid leak presence detector in leak profile by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17073](https://redirect.github.com/netty/netty/pull/17073) > * Add HttpContentCompressor constructor with ability to specify desired maxPipelineDepth by [`@reta`](https://github.com/reta) in [netty/netty#17068](https://redirect.github.com/netty/netty/pull/17068) > * IoUring: preserve readPending when rescheduling cancelled reads by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17087](https://redirect.github.com/netty/netty/pull/17087) > * Reject negative maxOrder in PooledByteBufAllocator by [`@coderbruis`](https://github.com/coderbruis) in [netty/netty#17093](https://redirect.github.com/netty/netty/pull/17093) > * Fix AdaptiveByteBuf.\_setLongLE calling checked setLongLE by [`@franz1981`](https://github.com/franz1981) in [netty/netty#17098](https://redirect.github.com/netty/netty/pull/17098) > * Snappy: Guard decoder against invalid chunk lengths by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17099](https://redirect.github.com/netty/netty/pull/17099) > * Fix OCSP Tests by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17114](https://redirect.github.com/netty/netty/pull/17114) > * Update to latest netty-tcnative release by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17056](https://redirect.github.com/netty/netty/pull/17056) > * Use safe decompressor in Lz4FrameDecoder by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17118](https://redirect.github.com/netty/netty/pull/17118) > * Configure TestLens for the PR builds by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17129](https://redirect.github.com/netty/netty/pull/17129) > * IoUring: add SO\_INQ support for Unix domain sockets by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17127](https://redirect.github.com/netty/netty/pull/17127) > * fix(mqtt): drop UNSUBACK reason codes for MQTT 3.x encoding by [`@ChunMengLu`](https://github.com/ChunMengLu) in [netty/netty#17117](https://redirect.github.com/netty/netty/pull/17117) > * Propagate the CI envionment variables through to the docker builds by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17138](https://redirect.github.com/netty/netty/pull/17138) > * Codec-compression: Add decompressor API by [`@yawkat`](https://github.com/yawkat) in [netty/netty#16745](https://redirect.github.com/netty/netty/pull/16745) > * Fix silent failures and optimize error short-circuit in multi-threaded tests by [`@rajan-github`](https://github.com/rajan-github) in [netty/netty#17106](https://redirect.github.com/netty/netty/pull/17106) > * Codec-compression: Add Bzip2Decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17145](https://redirect.github.com/netty/netty/pull/17145) > * Fix buddy cache evicting chunks with live buffers by [`@franz1981`](https://github.com/franz1981) in [netty/netty#17154](https://redirect.github.com/netty/netty/pull/17154) > * Codec-compression: Add Snappy frame decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17153](https://redirect.github.com/netty/netty/pull/17153) > * Codec-compression: Add zlib decompressors by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17155](https://redirect.github.com/netty/netty/pull/17155) > * Codec-compression: Add Zstd decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17152](https://redirect.github.com/netty/netty/pull/17152) > * Codec-compression: Add LZF decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17147](https://redirect.github.com/netty/netty/pull/17147) > * Codec-compression: Add BrotliDecompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17146](https://redirect.github.com/netty/netty/pull/17146) > * Codec-compression: Add Lz4FrameDecompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17148](https://redirect.github.com/netty/netty/pull/17148) > * `HttpObjectEncoder` / `DefaultHttp2FrameWriter`: fix buffer leak when a `Throwable` is thrown during header encoding by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#17089](https://redirect.github.com/netty/netty/pull/17089) > * Fix direct memory OOM on low-core containers by [`@franz1981`](https://github.com/franz1981) in [netty/netty#17166](https://redirect.github.com/netty/netty/pull/17166) > * IoUring: Fix the recvmmsg emulation by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17187](https://redirect.github.com/netty/netty/pull/17187) > * Avoid classloader leak via GlobalEventExecutor terminationFuture failure by [`@seonwooj0810`](https://github.com/seonwooj0810) in [netty/netty#17140](https://redirect.github.com/netty/netty/pull/17140) > * BrotliEncoder: Prevent duplicate close scheduling by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17175](https://redirect.github.com/netty/netty/pull/17175) > * Fix JdkZlibDecompressor losing the tail of highly compressible streams by [`@renechoi`](https://github.com/renechoi) in [netty/netty#17191](https://redirect.github.com/netty/netty/pull/17191) > * Update compress-lzf to 1.2.1 by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17194](https://redirect.github.com/netty/netty/pull/17194) > * Do not write WebSocket handshake response to the tail of the pipeline by [`@el-psy-kongroo-d`](https://github.com/el-psy-kongroo-d) in [netty/netty#17192](https://redirect.github.com/netty/netty/pull/17192) > * `HttpServerCodec`: do not consume the method queue for 1xx interim responses by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#17182](https://redirect.github.com/netty/netty/pull/17182) > * Weakly reference engines from the OpenSSL engine map by [`@bryce-anderson`](https://github.com/bryce-anderson) in [netty/netty#17199](https://redirect.github.com/netty/netty/pull/17199) > * OpenSSL: Allow to obtain used named group via OpenSslSession by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17058](https://redirect.github.com/netty/netty/pull/17058) > * Add `.editorconfig` to enforce consistent coding style by [`@vpelikh`](https://github.com/vpelikh) in [netty/netty#17052](https://redirect.github.com/netty/netty/pull/17052) > * Update surefire plugin to latest version by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17210](https://redirect.github.com/netty/netty/pull/17210) > * Merge changes from forks by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17213](https://redirect.github.com/netty/netty/pull/17213) > > New Contributors > ---------------- > > * [`@vpelikh`](https://github.com/vpelikh) made their first contribution in [netty/netty#17007](https://redirect.github.com/netty/netty/pull/17007) > * [`@ChunMengLu`](https://github.com/ChunMengLu) made their first contribution in [netty/netty#17117](https://redirect.github.com/netty/netty/pull/17117) > * [`@rajan-github`](https://github.com/rajan-github) made their first contribution in [netty/netty#17106](https://redirect.github.com/netty/netty/pull/17106) > * [`@seonwooj0810`](https://github.com/seonwooj0810) made their first contribution in [netty/netty#17140](https://redirect.github.com/netty/netty/pull/17140) > * [`@renechoi`](https://github.com/renechoi) made their first contribution in [netty/netty#17191](https://redirect.github.com/netty/netty/pull/17191) ... (truncated) Commits * [`e0789d3`](netty/netty@e0789d3) [maven-release-plugin] prepare release netty-4.2.17.Final * [`1b5abc6`](netty/netty@1b5abc6) Merge changes from forks ([#17213](https://redirect.github.com/netty/netty/issues/17213)) * [`36fbf57`](netty/netty@36fbf57) Update surefire plugin to latest version ([#17210](https://redirect.github.com/netty/netty/issues/17210)) * [`a96226c`](netty/netty@a96226c) Add `.editorconfig` to enforce consistent coding style ([#17052](https://redirect.github.com/netty/netty/issues/17052)) * [`14a4e6a`](netty/netty@14a4e6a) OpenSSL: Allow to obtain used named group via OpenSslSession ([#17058](https://redirect.github.com/netty/netty/issues/17058)) * [`26255b1`](netty/netty@26255b1) Weakly reference engines from the OpenSSL engine map ([#17199](https://redirect.github.com/netty/netty/issues/17199)) * [`ae41417`](netty/netty@ae41417) `HttpServerCodec`: do not consume the method queue for 1xx interim responses ... * [`41f1db5`](netty/netty@41f1db5) Do not write WebSocket handshake response to the tail of the pipeline ([#17192](https://redirect.github.com/netty/netty/issues/17192)) * [`035d76e`](netty/netty@035d76e) Update compress-lzf to 1.2.1 ([#17194](https://redirect.github.com/netty/netty/issues/17194)) * [`7681aff`](netty/netty@7681aff) Fix JdkZlibDecompressor losing the tail of highly compressible streams ([#17191](https://redirect.github.com/netty/netty/issues/17191)) * Additional commits viewable in [compare view](netty/netty@netty-4.2.16.Final...netty-4.2.17.Final) [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
…ip ci] Bumps `netty.version` from 4.2.16.Final to 4.2.17.Final. Updates `io.netty:netty-transport` from 4.2.16.Final to 4.2.17.Final Release notes *Sourced from [io.netty:netty-transport's releases](https://github.com/netty/netty/releases).* > netty-4.2.17.Final > ------------------ > > What's Changed > -------------- > > * AsciiString.cached(String) should sanitize the provided String ([#13749](https://redirect.github.com/netty/netty/issues/13749)) by [`@vpelikh`](https://github.com/vpelikh) in [netty/netty#17007](https://redirect.github.com/netty/netty/pull/17007) > * Fix deploy workflow by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17071](https://redirect.github.com/netty/netty/pull/17071) > * Fix AsciiString.cached(String) performance regression by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17074](https://redirect.github.com/netty/netty/pull/17074) > * SslHandler: Fix possible buffer leak when an OOME is thrown during allocation by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17059](https://redirect.github.com/netty/netty/pull/17059) > * Avoid leak presence detector in leak profile by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17073](https://redirect.github.com/netty/netty/pull/17073) > * Add HttpContentCompressor constructor with ability to specify desired maxPipelineDepth by [`@reta`](https://github.com/reta) in [netty/netty#17068](https://redirect.github.com/netty/netty/pull/17068) > * IoUring: preserve readPending when rescheduling cancelled reads by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17087](https://redirect.github.com/netty/netty/pull/17087) > * Reject negative maxOrder in PooledByteBufAllocator by [`@coderbruis`](https://github.com/coderbruis) in [netty/netty#17093](https://redirect.github.com/netty/netty/pull/17093) > * Fix AdaptiveByteBuf.\_setLongLE calling checked setLongLE by [`@franz1981`](https://github.com/franz1981) in [netty/netty#17098](https://redirect.github.com/netty/netty/pull/17098) > * Snappy: Guard decoder against invalid chunk lengths by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17099](https://redirect.github.com/netty/netty/pull/17099) > * Fix OCSP Tests by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17114](https://redirect.github.com/netty/netty/pull/17114) > * Update to latest netty-tcnative release by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17056](https://redirect.github.com/netty/netty/pull/17056) > * Use safe decompressor in Lz4FrameDecoder by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17118](https://redirect.github.com/netty/netty/pull/17118) > * Configure TestLens for the PR builds by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17129](https://redirect.github.com/netty/netty/pull/17129) > * IoUring: add SO\_INQ support for Unix domain sockets by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17127](https://redirect.github.com/netty/netty/pull/17127) > * fix(mqtt): drop UNSUBACK reason codes for MQTT 3.x encoding by [`@ChunMengLu`](https://github.com/ChunMengLu) in [netty/netty#17117](https://redirect.github.com/netty/netty/pull/17117) > * Propagate the CI envionment variables through to the docker builds by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17138](https://redirect.github.com/netty/netty/pull/17138) > * Codec-compression: Add decompressor API by [`@yawkat`](https://github.com/yawkat) in [netty/netty#16745](https://redirect.github.com/netty/netty/pull/16745) > * Fix silent failures and optimize error short-circuit in multi-threaded tests by [`@rajan-github`](https://github.com/rajan-github) in [netty/netty#17106](https://redirect.github.com/netty/netty/pull/17106) > * Codec-compression: Add Bzip2Decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17145](https://redirect.github.com/netty/netty/pull/17145) > * Fix buddy cache evicting chunks with live buffers by [`@franz1981`](https://github.com/franz1981) in [netty/netty#17154](https://redirect.github.com/netty/netty/pull/17154) > * Codec-compression: Add Snappy frame decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17153](https://redirect.github.com/netty/netty/pull/17153) > * Codec-compression: Add zlib decompressors by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17155](https://redirect.github.com/netty/netty/pull/17155) > * Codec-compression: Add Zstd decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17152](https://redirect.github.com/netty/netty/pull/17152) > * Codec-compression: Add LZF decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17147](https://redirect.github.com/netty/netty/pull/17147) > * Codec-compression: Add BrotliDecompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17146](https://redirect.github.com/netty/netty/pull/17146) > * Codec-compression: Add Lz4FrameDecompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17148](https://redirect.github.com/netty/netty/pull/17148) > * `HttpObjectEncoder` / `DefaultHttp2FrameWriter`: fix buffer leak when a `Throwable` is thrown during header encoding by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#17089](https://redirect.github.com/netty/netty/pull/17089) > * Fix direct memory OOM on low-core containers by [`@franz1981`](https://github.com/franz1981) in [netty/netty#17166](https://redirect.github.com/netty/netty/pull/17166) > * IoUring: Fix the recvmmsg emulation by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17187](https://redirect.github.com/netty/netty/pull/17187) > * Avoid classloader leak via GlobalEventExecutor terminationFuture failure by [`@seonwooj0810`](https://github.com/seonwooj0810) in [netty/netty#17140](https://redirect.github.com/netty/netty/pull/17140) > * BrotliEncoder: Prevent duplicate close scheduling by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17175](https://redirect.github.com/netty/netty/pull/17175) > * Fix JdkZlibDecompressor losing the tail of highly compressible streams by [`@renechoi`](https://github.com/renechoi) in [netty/netty#17191](https://redirect.github.com/netty/netty/pull/17191) > * Update compress-lzf to 1.2.1 by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17194](https://redirect.github.com/netty/netty/pull/17194) > * Do not write WebSocket handshake response to the tail of the pipeline by [`@el-psy-kongroo-d`](https://github.com/el-psy-kongroo-d) in [netty/netty#17192](https://redirect.github.com/netty/netty/pull/17192) > * `HttpServerCodec`: do not consume the method queue for 1xx interim responses by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#17182](https://redirect.github.com/netty/netty/pull/17182) > * Weakly reference engines from the OpenSSL engine map by [`@bryce-anderson`](https://github.com/bryce-anderson) in [netty/netty#17199](https://redirect.github.com/netty/netty/pull/17199) > * OpenSSL: Allow to obtain used named group via OpenSslSession by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17058](https://redirect.github.com/netty/netty/pull/17058) > * Add `.editorconfig` to enforce consistent coding style by [`@vpelikh`](https://github.com/vpelikh) in [netty/netty#17052](https://redirect.github.com/netty/netty/pull/17052) > * Update surefire plugin to latest version by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17210](https://redirect.github.com/netty/netty/pull/17210) > * Merge changes from forks by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17213](https://redirect.github.com/netty/netty/pull/17213) > > New Contributors > ---------------- > > * [`@vpelikh`](https://github.com/vpelikh) made their first contribution in [netty/netty#17007](https://redirect.github.com/netty/netty/pull/17007) > * [`@ChunMengLu`](https://github.com/ChunMengLu) made their first contribution in [netty/netty#17117](https://redirect.github.com/netty/netty/pull/17117) > * [`@rajan-github`](https://github.com/rajan-github) made their first contribution in [netty/netty#17106](https://redirect.github.com/netty/netty/pull/17106) > * [`@seonwooj0810`](https://github.com/seonwooj0810) made their first contribution in [netty/netty#17140](https://redirect.github.com/netty/netty/pull/17140) > * [`@renechoi`](https://github.com/renechoi) made their first contribution in [netty/netty#17191](https://redirect.github.com/netty/netty/pull/17191) ... (truncated) Commits * [`e0789d3`](netty/netty@e0789d3) [maven-release-plugin] prepare release netty-4.2.17.Final * [`1b5abc6`](netty/netty@1b5abc6) Merge changes from forks ([#17213](https://redirect.github.com/netty/netty/issues/17213)) * [`36fbf57`](netty/netty@36fbf57) Update surefire plugin to latest version ([#17210](https://redirect.github.com/netty/netty/issues/17210)) * [`a96226c`](netty/netty@a96226c) Add `.editorconfig` to enforce consistent coding style ([#17052](https://redirect.github.com/netty/netty/issues/17052)) * [`14a4e6a`](netty/netty@14a4e6a) OpenSSL: Allow to obtain used named group via OpenSslSession ([#17058](https://redirect.github.com/netty/netty/issues/17058)) * [`26255b1`](netty/netty@26255b1) Weakly reference engines from the OpenSSL engine map ([#17199](https://redirect.github.com/netty/netty/issues/17199)) * [`ae41417`](netty/netty@ae41417) `HttpServerCodec`: do not consume the method queue for 1xx interim responses ... * [`41f1db5`](netty/netty@41f1db5) Do not write WebSocket handshake response to the tail of the pipeline ([#17192](https://redirect.github.com/netty/netty/issues/17192)) * [`035d76e`](netty/netty@035d76e) Update compress-lzf to 1.2.1 ([#17194](https://redirect.github.com/netty/netty/issues/17194)) * [`7681aff`](netty/netty@7681aff) Fix JdkZlibDecompressor losing the tail of highly compressible streams ([#17191](https://redirect.github.com/netty/netty/issues/17191)) * Additional commits viewable in [compare view](netty/netty@netty-4.2.16.Final...netty-4.2.17.Final) Updates `io.netty:netty-codec` from 4.2.16.Final to 4.2.17.Final Release notes *Sourced from [io.netty:netty-codec's releases](https://github.com/netty/netty/releases).* > netty-4.2.17.Final > ------------------ > > What's Changed > -------------- > > * AsciiString.cached(String) should sanitize the provided String ([#13749](https://redirect.github.com/netty/netty/issues/13749)) by [`@vpelikh`](https://github.com/vpelikh) in [netty/netty#17007](https://redirect.github.com/netty/netty/pull/17007) > * Fix deploy workflow by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17071](https://redirect.github.com/netty/netty/pull/17071) > * Fix AsciiString.cached(String) performance regression by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17074](https://redirect.github.com/netty/netty/pull/17074) > * SslHandler: Fix possible buffer leak when an OOME is thrown during allocation by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17059](https://redirect.github.com/netty/netty/pull/17059) > * Avoid leak presence detector in leak profile by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17073](https://redirect.github.com/netty/netty/pull/17073) > * Add HttpContentCompressor constructor with ability to specify desired maxPipelineDepth by [`@reta`](https://github.com/reta) in [netty/netty#17068](https://redirect.github.com/netty/netty/pull/17068) > * IoUring: preserve readPending when rescheduling cancelled reads by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17087](https://redirect.github.com/netty/netty/pull/17087) > * Reject negative maxOrder in PooledByteBufAllocator by [`@coderbruis`](https://github.com/coderbruis) in [netty/netty#17093](https://redirect.github.com/netty/netty/pull/17093) > * Fix AdaptiveByteBuf.\_setLongLE calling checked setLongLE by [`@franz1981`](https://github.com/franz1981) in [netty/netty#17098](https://redirect.github.com/netty/netty/pull/17098) > * Snappy: Guard decoder against invalid chunk lengths by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17099](https://redirect.github.com/netty/netty/pull/17099) > * Fix OCSP Tests by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17114](https://redirect.github.com/netty/netty/pull/17114) > * Update to latest netty-tcnative release by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17056](https://redirect.github.com/netty/netty/pull/17056) > * Use safe decompressor in Lz4FrameDecoder by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17118](https://redirect.github.com/netty/netty/pull/17118) > * Configure TestLens for the PR builds by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17129](https://redirect.github.com/netty/netty/pull/17129) > * IoUring: add SO\_INQ support for Unix domain sockets by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17127](https://redirect.github.com/netty/netty/pull/17127) > * fix(mqtt): drop UNSUBACK reason codes for MQTT 3.x encoding by [`@ChunMengLu`](https://github.com/ChunMengLu) in [netty/netty#17117](https://redirect.github.com/netty/netty/pull/17117) > * Propagate the CI envionment variables through to the docker builds by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17138](https://redirect.github.com/netty/netty/pull/17138) > * Codec-compression: Add decompressor API by [`@yawkat`](https://github.com/yawkat) in [netty/netty#16745](https://redirect.github.com/netty/netty/pull/16745) > * Fix silent failures and optimize error short-circuit in multi-threaded tests by [`@rajan-github`](https://github.com/rajan-github) in [netty/netty#17106](https://redirect.github.com/netty/netty/pull/17106) > * Codec-compression: Add Bzip2Decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17145](https://redirect.github.com/netty/netty/pull/17145) > * Fix buddy cache evicting chunks with live buffers by [`@franz1981`](https://github.com/franz1981) in [netty/netty#17154](https://redirect.github.com/netty/netty/pull/17154) > * Codec-compression: Add Snappy frame decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17153](https://redirect.github.com/netty/netty/pull/17153) > * Codec-compression: Add zlib decompressors by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17155](https://redirect.github.com/netty/netty/pull/17155) > * Codec-compression: Add Zstd decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17152](https://redirect.github.com/netty/netty/pull/17152) > * Codec-compression: Add LZF decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17147](https://redirect.github.com/netty/netty/pull/17147) > * Codec-compression: Add BrotliDecompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17146](https://redirect.github.com/netty/netty/pull/17146) > * Codec-compression: Add Lz4FrameDecompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17148](https://redirect.github.com/netty/netty/pull/17148) > * `HttpObjectEncoder` / `DefaultHttp2FrameWriter`: fix buffer leak when a `Throwable` is thrown during header encoding by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#17089](https://redirect.github.com/netty/netty/pull/17089) > * Fix direct memory OOM on low-core containers by [`@franz1981`](https://github.com/franz1981) in [netty/netty#17166](https://redirect.github.com/netty/netty/pull/17166) > * IoUring: Fix the recvmmsg emulation by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17187](https://redirect.github.com/netty/netty/pull/17187) > * Avoid classloader leak via GlobalEventExecutor terminationFuture failure by [`@seonwooj0810`](https://github.com/seonwooj0810) in [netty/netty#17140](https://redirect.github.com/netty/netty/pull/17140) > * BrotliEncoder: Prevent duplicate close scheduling by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17175](https://redirect.github.com/netty/netty/pull/17175) > * Fix JdkZlibDecompressor losing the tail of highly compressible streams by [`@renechoi`](https://github.com/renechoi) in [netty/netty#17191](https://redirect.github.com/netty/netty/pull/17191) > * Update compress-lzf to 1.2.1 by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17194](https://redirect.github.com/netty/netty/pull/17194) > * Do not write WebSocket handshake response to the tail of the pipeline by [`@el-psy-kongroo-d`](https://github.com/el-psy-kongroo-d) in [netty/netty#17192](https://redirect.github.com/netty/netty/pull/17192) > * `HttpServerCodec`: do not consume the method queue for 1xx interim responses by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#17182](https://redirect.github.com/netty/netty/pull/17182) > * Weakly reference engines from the OpenSSL engine map by [`@bryce-anderson`](https://github.com/bryce-anderson) in [netty/netty#17199](https://redirect.github.com/netty/netty/pull/17199) > * OpenSSL: Allow to obtain used named group via OpenSslSession by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17058](https://redirect.github.com/netty/netty/pull/17058) > * Add `.editorconfig` to enforce consistent coding style by [`@vpelikh`](https://github.com/vpelikh) in [netty/netty#17052](https://redirect.github.com/netty/netty/pull/17052) > * Update surefire plugin to latest version by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17210](https://redirect.github.com/netty/netty/pull/17210) > * Merge changes from forks by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17213](https://redirect.github.com/netty/netty/pull/17213) > > New Contributors > ---------------- > > * [`@vpelikh`](https://github.com/vpelikh) made their first contribution in [netty/netty#17007](https://redirect.github.com/netty/netty/pull/17007) > * [`@ChunMengLu`](https://github.com/ChunMengLu) made their first contribution in [netty/netty#17117](https://redirect.github.com/netty/netty/pull/17117) > * [`@rajan-github`](https://github.com/rajan-github) made their first contribution in [netty/netty#17106](https://redirect.github.com/netty/netty/pull/17106) > * [`@seonwooj0810`](https://github.com/seonwooj0810) made their first contribution in [netty/netty#17140](https://redirect.github.com/netty/netty/pull/17140) > * [`@renechoi`](https://github.com/renechoi) made their first contribution in [netty/netty#17191](https://redirect.github.com/netty/netty/pull/17191) ... (truncated) Commits * [`e0789d3`](netty/netty@e0789d3) [maven-release-plugin] prepare release netty-4.2.17.Final * [`1b5abc6`](netty/netty@1b5abc6) Merge changes from forks ([#17213](https://redirect.github.com/netty/netty/issues/17213)) * [`36fbf57`](netty/netty@36fbf57) Update surefire plugin to latest version ([#17210](https://redirect.github.com/netty/netty/issues/17210)) * [`a96226c`](netty/netty@a96226c) Add `.editorconfig` to enforce consistent coding style ([#17052](https://redirect.github.com/netty/netty/issues/17052)) * [`14a4e6a`](netty/netty@14a4e6a) OpenSSL: Allow to obtain used named group via OpenSslSession ([#17058](https://redirect.github.com/netty/netty/issues/17058)) * [`26255b1`](netty/netty@26255b1) Weakly reference engines from the OpenSSL engine map ([#17199](https://redirect.github.com/netty/netty/issues/17199)) * [`ae41417`](netty/netty@ae41417) `HttpServerCodec`: do not consume the method queue for 1xx interim responses ... * [`41f1db5`](netty/netty@41f1db5) Do not write WebSocket handshake response to the tail of the pipeline ([#17192](https://redirect.github.com/netty/netty/issues/17192)) * [`035d76e`](netty/netty@035d76e) Update compress-lzf to 1.2.1 ([#17194](https://redirect.github.com/netty/netty/issues/17194)) * [`7681aff`](netty/netty@7681aff) Fix JdkZlibDecompressor losing the tail of highly compressible streams ([#17191](https://redirect.github.com/netty/netty/issues/17191)) * Additional commits viewable in [compare view](netty/netty@netty-4.2.16.Final...netty-4.2.17.Final) Updates `io.netty:netty-handler` from 4.2.16.Final to 4.2.17.Final Release notes *Sourced from [io.netty:netty-handler's releases](https://github.com/netty/netty/releases).* > netty-4.2.17.Final > ------------------ > > What's Changed > -------------- > > * AsciiString.cached(String) should sanitize the provided String ([#13749](https://redirect.github.com/netty/netty/issues/13749)) by [`@vpelikh`](https://github.com/vpelikh) in [netty/netty#17007](https://redirect.github.com/netty/netty/pull/17007) > * Fix deploy workflow by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17071](https://redirect.github.com/netty/netty/pull/17071) > * Fix AsciiString.cached(String) performance regression by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17074](https://redirect.github.com/netty/netty/pull/17074) > * SslHandler: Fix possible buffer leak when an OOME is thrown during allocation by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17059](https://redirect.github.com/netty/netty/pull/17059) > * Avoid leak presence detector in leak profile by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17073](https://redirect.github.com/netty/netty/pull/17073) > * Add HttpContentCompressor constructor with ability to specify desired maxPipelineDepth by [`@reta`](https://github.com/reta) in [netty/netty#17068](https://redirect.github.com/netty/netty/pull/17068) > * IoUring: preserve readPending when rescheduling cancelled reads by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17087](https://redirect.github.com/netty/netty/pull/17087) > * Reject negative maxOrder in PooledByteBufAllocator by [`@coderbruis`](https://github.com/coderbruis) in [netty/netty#17093](https://redirect.github.com/netty/netty/pull/17093) > * Fix AdaptiveByteBuf.\_setLongLE calling checked setLongLE by [`@franz1981`](https://github.com/franz1981) in [netty/netty#17098](https://redirect.github.com/netty/netty/pull/17098) > * Snappy: Guard decoder against invalid chunk lengths by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17099](https://redirect.github.com/netty/netty/pull/17099) > * Fix OCSP Tests by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17114](https://redirect.github.com/netty/netty/pull/17114) > * Update to latest netty-tcnative release by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17056](https://redirect.github.com/netty/netty/pull/17056) > * Use safe decompressor in Lz4FrameDecoder by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17118](https://redirect.github.com/netty/netty/pull/17118) > * Configure TestLens for the PR builds by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17129](https://redirect.github.com/netty/netty/pull/17129) > * IoUring: add SO\_INQ support for Unix domain sockets by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17127](https://redirect.github.com/netty/netty/pull/17127) > * fix(mqtt): drop UNSUBACK reason codes for MQTT 3.x encoding by [`@ChunMengLu`](https://github.com/ChunMengLu) in [netty/netty#17117](https://redirect.github.com/netty/netty/pull/17117) > * Propagate the CI envionment variables through to the docker builds by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17138](https://redirect.github.com/netty/netty/pull/17138) > * Codec-compression: Add decompressor API by [`@yawkat`](https://github.com/yawkat) in [netty/netty#16745](https://redirect.github.com/netty/netty/pull/16745) > * Fix silent failures and optimize error short-circuit in multi-threaded tests by [`@rajan-github`](https://github.com/rajan-github) in [netty/netty#17106](https://redirect.github.com/netty/netty/pull/17106) > * Codec-compression: Add Bzip2Decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17145](https://redirect.github.com/netty/netty/pull/17145) > * Fix buddy cache evicting chunks with live buffers by [`@franz1981`](https://github.com/franz1981) in [netty/netty#17154](https://redirect.github.com/netty/netty/pull/17154) > * Codec-compression: Add Snappy frame decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17153](https://redirect.github.com/netty/netty/pull/17153) > * Codec-compression: Add zlib decompressors by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17155](https://redirect.github.com/netty/netty/pull/17155) > * Codec-compression: Add Zstd decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17152](https://redirect.github.com/netty/netty/pull/17152) > * Codec-compression: Add LZF decompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17147](https://redirect.github.com/netty/netty/pull/17147) > * Codec-compression: Add BrotliDecompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17146](https://redirect.github.com/netty/netty/pull/17146) > * Codec-compression: Add Lz4FrameDecompressor by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17148](https://redirect.github.com/netty/netty/pull/17148) > * `HttpObjectEncoder` / `DefaultHttp2FrameWriter`: fix buffer leak when a `Throwable` is thrown during header encoding by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#17089](https://redirect.github.com/netty/netty/pull/17089) > * Fix direct memory OOM on low-core containers by [`@franz1981`](https://github.com/franz1981) in [netty/netty#17166](https://redirect.github.com/netty/netty/pull/17166) > * IoUring: Fix the recvmmsg emulation by [`@dreamlike-ocean`](https://github.com/dreamlike-ocean) in [netty/netty#17187](https://redirect.github.com/netty/netty/pull/17187) > * Avoid classloader leak via GlobalEventExecutor terminationFuture failure by [`@seonwooj0810`](https://github.com/seonwooj0810) in [netty/netty#17140](https://redirect.github.com/netty/netty/pull/17140) > * BrotliEncoder: Prevent duplicate close scheduling by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17175](https://redirect.github.com/netty/netty/pull/17175) > * Fix JdkZlibDecompressor losing the tail of highly compressible streams by [`@renechoi`](https://github.com/renechoi) in [netty/netty#17191](https://redirect.github.com/netty/netty/pull/17191) > * Update compress-lzf to 1.2.1 by [`@yawkat`](https://github.com/yawkat) in [netty/netty#17194](https://redirect.github.com/netty/netty/pull/17194) > * Do not write WebSocket handshake response to the tail of the pipeline by [`@el-psy-kongroo-d`](https://github.com/el-psy-kongroo-d) in [netty/netty#17192](https://redirect.github.com/netty/netty/pull/17192) > * `HttpServerCodec`: do not consume the method queue for 1xx interim responses by [`@HwangRock`](https://github.com/HwangRock) in [netty/netty#17182](https://redirect.github.com/netty/netty/pull/17182) > * Weakly reference engines from the OpenSSL engine map by [`@bryce-anderson`](https://github.com/bryce-anderson) in [netty/netty#17199](https://redirect.github.com/netty/netty/pull/17199) > * OpenSSL: Allow to obtain used named group via OpenSslSession by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17058](https://redirect.github.com/netty/netty/pull/17058) > * Add `.editorconfig` to enforce consistent coding style by [`@vpelikh`](https://github.com/vpelikh) in [netty/netty#17052](https://redirect.github.com/netty/netty/pull/17052) > * Update surefire plugin to latest version by [`@normanmaurer`](https://github.com/normanmaurer) in [netty/netty#17210](https://redirect.github.com/netty/netty/pull/17210) > * Merge changes from forks by [`@chrisvest`](https://github.com/chrisvest) in [netty/netty#17213](https://redirect.github.com/netty/netty/pull/17213) > > New Contributors > ---------------- > > * [`@vpelikh`](https://github.com/vpelikh) made their first contribution in [netty/netty#17007](https://redirect.github.com/netty/netty/pull/17007) > * [`@ChunMengLu`](https://github.com/ChunMengLu) made their first contribution in [netty/netty#17117](https://redirect.github.com/netty/netty/pull/17117) > * [`@rajan-github`](https://github.com/rajan-github) made their first contribution in [netty/netty#17106](https://redirect.github.com/netty/netty/pull/17106) > * [`@seonwooj0810`](https://github.com/seonwooj0810) made their first contribution in [netty/netty#17140](https://redirect.github.com/netty/netty/pull/17140) > * [`@renechoi`](https://github.com/renechoi) made their first contribution in [netty/netty#17191](https://redirect.github.com/netty/netty/pull/17191) ... (truncated) Commits * [`e0789d3`](netty/netty@e0789d3) [maven-release-plugin] prepare release netty-4.2.17.Final * [`1b5abc6`](netty/netty@1b5abc6) Merge changes from forks ([#17213](https://redirect.github.com/netty/netty/issues/17213)) * [`36fbf57`](netty/netty@36fbf57) Update surefire plugin to latest version ([#17210](https://redirect.github.com/netty/netty/issues/17210)) * [`a96226c`](netty/netty@a96226c) Add `.editorconfig` to enforce consistent coding style ([#17052](https://redirect.github.com/netty/netty/issues/17052)) * [`14a4e6a`](netty/netty@14a4e6a) OpenSSL: Allow to obtain used named group via OpenSslSession ([#17058](https://redirect.github.com/netty/netty/issues/17058)) * [`26255b1`](netty/netty@26255b1) Weakly reference engines from the OpenSSL engine map ([#17199](https://redirect.github.com/netty/netty/issues/17199)) * [`ae41417`](netty/netty@ae41417) `HttpServerCodec`: do not consume the method queue for 1xx interim responses ... * [`41f1db5`](netty/netty@41f1db5) Do not write WebSocket handshake response to the tail of the pipeline ([#17192](https://redirect.github.com/netty/netty/issues/17192)) * [`035d76e`](netty/netty@035d76e) Update compress-lzf to 1.2.1 ([#17194](https://redirect.github.com/netty/netty/issues/17194)) * [`7681aff`](netty/netty@7681aff) Fix JdkZlibDecompressor losing the tail of highly compressible streams ([#17191](https://redirect.github.com/netty/netty/issues/17191)) * Additional commits viewable in [compare view](netty/netty@netty-4.2.16.Final...netty-4.2.17.Final) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- Dependabot commands and options You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
| Back | FazBrowse Home | New Git URL |
Motivation
JdkZlibDecompressor#processOutput sizes every output buffer as
inflater.getRemaining() << 1. The number of remaining input bytes is only a hint: the inflater
may still hold decoded data that did not fit into the previous output buffer, and by then it may
have pulled the last input bytes into its internal state, so getRemaining() is 0 and the
proposed buffer has zero capacity. inflate(...) cannot make progress with it, and because
status() maps inflater.needsInput() straight to NEED_INPUT, the caller is asked for more
input instead of draining the pending output. At the end of the stream endOfInput() then fails
with Compressed stream ended before the end-of-stream marker, even though the stream was
complete and valid.
Measured with 100000 bytes of 'a' deflated raw (115 bytes, ratio ~870:1), driving the
decompressor exactly as its own tests do:
new JdkZlibDecompressor (ZlibWrapper.NONE): 99848 of 100000 bytes, then DecompressionException: Compressed stream ended before the end-of-stream marker legacy JdkZlibDecoder (ZlibWrapper.NONE): 100000 bytes java.util.zip.Inflater : 100000 bytesReplaying the same buffer sizing against a bare Inflater shows the state that is mis-read:
So the tail is inside the Inflater the whole time and only the buffer sizing keeps it there.
The legacy JdkZlibDecoder is not affected because it inflates into a single buffer that it keeps
expanding, so inflate(...) always has room. ZlibWrapper.ZLIB and ZlibWrapper.GZIP hide the
problem as well, because their trailer keeps getRemaining() above zero until the inflater is
finished; raw deflate has no trailer, which is what permessage-deflate and
Content-Encoding: deflate payloads look like.
Modification
MIN_OUTPUT_BUFFER_SIZE (512 bytes). maxAllocation still caps it exactly as before, so a
configured limit keeps its meaning.
report NEED_OUTPUT from status() so the pending output is drained before more input is
requested. needsInput() only says that all input bytes were consumed, not that all output was
produced. The flag is cleared once a call does not fill the buffer or the inflater is finished,
so this cannot loop.
Result
A valid deflate stream is fully decompressed regardless of its compression ratio.
Verification done: added JdkZlibDecompressorTest#testHighlyCompressibleStreamIsFullyDecompressed.
It runs for all three wrappers, cross-checks the compressed input with the JDK inflater first so a
failure cannot be blamed on the fixture, and then asserts the decompressor returns the same bytes.
It fails on unpatched 4.2 for ZlibWrapper.NONE with the exception above and passes with this
change. ./mvnw -pl codec-compression clean install is green on JDK 21 (499 tests, checkstyle,
forbidden-apis and revapi included).
I also ran the other Decompressor implementations through the same battery (round trip at chunk
sizes 1..64K, addInput ownership on malformed input, close() idempotency, concatenated frames,
and hand-crafted gzip headers covering every FLG combination against GZIPInputStream) and found
no other divergence, so this change is limited to the zlib one.