| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Sorry, something went wrong.
|
Another reason I initially moved the gyp file was that I was hoping to reuse tools/v8_gypfiles/GN-scraper.py but I wasn't able to work out how to reference it so for the moment it has been copied. I'm not sure if this fixes #33019 -- I no longer see the reported errors in zlib when compiling with configure --dest-cpu=x86 but the shared Linux server I've been working on doesn't have multilib so the build fails compiling V8. |
Sorry, something went wrong.
|
fwiw, I'd much rather we keep the relevant gyp files in the deps folder and consistently use a pattern that is more like what we do with openssl, nghttp2, etc... where the source files for the dependency are in a subdirectory (e.g. deps/openssl/openssl and deps/nghttp2/lib) |
Sorry, something went wrong.
|
@jasnell Moved the gyp file back to deps/zlib. |
Sorry, something went wrong.
|
CI: https://ci.nodejs.org/job/node-test-pull-request/30957/ (:yellow_heart:) |
Sorry, something went wrong.
|
Sigh. I ran benchmarks over the weekend and there are notable regressions. AFAICT the build logs (see the second link below) show we're passing in the expected compiler flags/definitions for the optimized files based on the BUILD.gn file so ¯\_(ツ)_/¯. I'm not intending to spend a lot of time on this as it's not high on my list of priorities. https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/576/console 21:29:23 confidence improvement accuracy (*) (**) (***) 21:29:23 zlib/createInflate.js n=100 chunkLen=1024 inputLen=16777216 *** -24.48 % ±5.70% ±7.59% ±9.90% 21:29:23 zlib/creation.js n=500000 options='false' type='BrotliCompress' 0.30 % ±3.77% ±5.02% ±6.53% 21:29:23 zlib/creation.js n=500000 options='false' type='BrotliDecompress' 2.63 % ±3.70% ±4.93% ±6.42% 21:29:23 zlib/creation.js n=500000 options='false' type='Deflate' -0.40 % ±2.62% ±3.48% ±4.53% 21:29:23 zlib/creation.js n=500000 options='false' type='DeflateRaw' -0.14 % ±2.19% ±2.92% ±3.80% 21:29:23 zlib/creation.js n=500000 options='false' type='Gunzip' -0.04 % ±4.19% ±5.58% ±7.26% 21:29:23 zlib/creation.js n=500000 options='false' type='Gzip' -0.17 % ±2.13% ±2.84% ±3.70% 21:29:23 zlib/creation.js n=500000 options='false' type='Inflate' 1.16 % ±4.44% ±5.91% ±7.72% 21:29:23 zlib/creation.js n=500000 options='false' type='InflateRaw' 1.47 % ±4.26% ±5.67% ±7.38% 21:29:23 zlib/creation.js n=500000 options='false' type='Unzip' 1.18 % ±3.50% ±4.66% ±6.07% 21:29:23 zlib/creation.js n=500000 options='true' type='BrotliCompress' 3.92 % ±4.00% ±5.32% ±6.92% 21:29:23 zlib/creation.js n=500000 options='true' type='BrotliDecompress' -0.21 % ±4.90% ±6.53% ±8.52% 21:29:23 zlib/creation.js n=500000 options='true' type='Deflate' 1.21 % ±2.24% ±2.98% ±3.88% 21:29:23 zlib/creation.js n=500000 options='true' type='DeflateRaw' -0.02 % ±2.04% ±2.72% ±3.55% 21:29:23 zlib/creation.js n=500000 options='true' type='Gunzip' -2.52 % ±4.26% ±5.67% ±7.40% 21:29:23 zlib/creation.js n=500000 options='true' type='Gzip' -0.96 % ±1.74% ±2.32% ±3.02% 21:29:23 zlib/creation.js n=500000 options='true' type='Inflate' 1.85 % ±4.05% ±5.38% ±7.01% 21:29:23 zlib/creation.js n=500000 options='true' type='InflateRaw' 1.60 % ±3.67% ±4.89% ±6.36% 21:29:23 zlib/creation.js n=500000 options='true' type='Unzip' -3.73 % ±3.84% ±5.12% ±6.69% 21:29:23 zlib/deflate.js n=400000 inputLen=1024 method='createDeflate' -6.13 % ±11.55% ±15.37% ±20.01% 21:29:23 zlib/deflate.js n=400000 inputLen=1024 method='deflate' 1.87 % ±3.66% ±4.88% ±6.37% 21:29:23 zlib/deflate.js n=400000 inputLen=1024 method='deflateSync' 7.41 % ±13.36% ±17.78% ±23.14% 21:29:23 zlib/inflate.js n=400000 inputLen=1024 method='inflate' 0.51 % ±3.30% ±4.39% ±5.71% 21:29:23 zlib/inflate.js n=400000 inputLen=1024 method='inflateSync' -1.42 % ±5.53% ±7.36% ±9.58% 21:29:23 zlib/pipe.js algorithm='brotli' type='buffer' duration=5 inputLen=1024 * -1.84 % ±1.72% ±2.29% ±2.98% 21:29:23 zlib/pipe.js algorithm='brotli' type='string' duration=5 inputLen=1024 *** -11.85 % ±3.66% ±4.93% ±6.52% 21:29:23 zlib/pipe.js algorithm='gzip' type='buffer' duration=5 inputLen=1024 -3.83 % ±5.93% ±7.91% ±10.35% 21:29:23 zlib/pipe.js algorithm='gzip' type='string' duration=5 inputLen=1024 ** -10.43 % ±6.48% ±8.68% ±11.42% 15:47:40 confidence improvement accuracy (*) (**) (***) 15:47:40 zlib/createInflate.js n=100 chunkLen=1024 inputLen=16777216 *** -28.15 % ±5.64% ±7.51% ±9.77% 15:47:40 zlib/creation.js n=500000 options='false' type='BrotliCompress' 1.01 % ±3.96% ±5.27% ±6.88% 15:47:40 zlib/creation.js n=500000 options='false' type='BrotliDecompress' 0.10 % ±4.39% ±5.85% ±7.63% 15:47:40 zlib/creation.js n=500000 options='false' type='Deflate' -1.23 % ±3.23% ±4.31% ±5.62% 15:47:40 zlib/creation.js n=500000 options='false' type='DeflateRaw' -3.16 % ±3.22% ±4.29% ±5.59% 15:47:40 zlib/creation.js n=500000 options='false' type='Gunzip' -0.08 % ±3.81% ±5.06% ±6.59% 15:47:40 zlib/creation.js n=500000 options='false' type='Gzip' 0.95 % ±3.21% ±4.27% ±5.55% 15:47:40 zlib/creation.js n=500000 options='false' type='Inflate' -2.12 % ±3.64% ±4.85% ±6.31% 15:47:40 zlib/creation.js n=500000 options='false' type='InflateRaw' 2.26 % ±3.70% ±4.93% ±6.42% 15:47:40 zlib/creation.js n=500000 options='false' type='Unzip' -0.70 % ±3.96% ±5.27% ±6.86% 15:47:40 zlib/creation.js n=500000 options='true' type='BrotliCompress' 2.61 % ±3.84% ±5.11% ±6.65% 15:47:40 zlib/creation.js n=500000 options='true' type='BrotliDecompress' 0.12 % ±3.91% ±5.21% ±6.78% 15:47:40 zlib/creation.js n=500000 options='true' type='Deflate' -1.18 % ±2.60% ±3.46% ±4.51% 15:47:40 zlib/creation.js n=500000 options='true' type='DeflateRaw' ** 3.87 % ±2.58% ±3.44% ±4.49% 15:47:40 zlib/creation.js n=500000 options='true' type='Gunzip' 1.88 % ±4.89% ±6.51% ±8.49% 15:47:40 zlib/creation.js n=500000 options='true' type='Gzip' -1.61 % ±3.12% ±4.15% ±5.40% 15:47:40 zlib/creation.js n=500000 options='true' type='Inflate' * -4.44 % ±3.40% ±4.52% ±5.89% 15:47:40 zlib/creation.js n=500000 options='true' type='InflateRaw' 0.67 % ±4.28% ±5.69% ±7.41% 15:47:40 zlib/creation.js n=500000 options='true' type='Unzip' -1.72 % ±4.35% ±5.79% ±7.53% 15:47:40 zlib/deflate.js n=400000 inputLen=1024 method='createDeflate' -4.97 % ±11.58% ±15.42% ±20.07% 15:47:40 zlib/deflate.js n=400000 inputLen=1024 method='deflate' 1.16 % ±4.19% ±5.57% ±7.26% 15:47:40 zlib/deflate.js n=400000 inputLen=1024 method='deflateSync' 9.44 % ±13.05% ±17.36% ±22.59% 15:47:40 zlib/inflate.js n=400000 inputLen=1024 method='inflate' -1.83 % ±2.37% ±3.16% ±4.12% 15:47:40 zlib/inflate.js n=400000 inputLen=1024 method='inflateSync' * -4.78 % ±4.75% ±6.32% ±8.22% 15:47:40 zlib/pipe.js algorithm='brotli' type='buffer' duration=5 inputLen=1024 -0.17 % ±1.68% ±2.24% ±2.92% 15:47:41 zlib/pipe.js algorithm='brotli' type='string' duration=5 inputLen=1024 0.66 % ±3.16% ±4.21% ±5.48% 15:47:41 zlib/pipe.js algorithm='gzip' type='buffer' duration=5 inputLen=1024 * 7.87 % ±6.98% ±9.36% ±12.33% 15:47:41 zlib/pipe.js algorithm='gzip' type='string' duration=5 inputLen=1024 *** -12.12 % ±4.46% ±5.95% ±7.78% |
Sorry, something went wrong.
|
Reportedly this fixes building on 32-bit x86 (#33019 (comment)) (which is an unofficial build platform) but I'm assuming the drop in benchmarks would prevent this landing. I don't have the time to investigate this further, so I've put a help wanted label in case anyone else is interested and has the time to look into the benchmarks. |
Sorry, something went wrong.
|
Another report that this fixes building on 32-bit Linux: nodejs/help#2943 (comment) |
Sorry, something went wrong.
|
I rebased against master and force-pushed in the hope that it will somehow cause the performance regression to go away. Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/670/ UPDATE: Bummer. Still shows the performance degredation.confidence improvement accuracy (*) (**) (***) 21:12:01 zlib/createInflate.js n=100 chunkLen=1024 inputLen=16777216 *** -31.53 % ±6.39% ±8.50% ±11.06% 21:12:01 zlib/creation.js n=500000 options='false' type='BrotliCompress' * -4.50 % ±4.21% ±5.60% ±7.30% 21:12:01 zlib/creation.js n=500000 options='false' type='BrotliDecompress' 3.16 % ±5.05% ±6.72% ±8.75% 21:12:01 zlib/creation.js n=500000 options='false' type='Deflate' -1.66 % ±2.54% ±3.38% ±4.40% 21:12:01 zlib/creation.js n=500000 options='false' type='DeflateRaw' 3.29 % ±3.64% ±4.84% ±6.30% 21:12:01 zlib/creation.js n=500000 options='false' type='Gunzip' -0.61 % ±3.19% ±4.24% ±5.53% 21:12:01 zlib/creation.js n=500000 options='false' type='Gzip' -0.22 % ±2.28% ±3.03% ±3.95% 21:12:01 zlib/creation.js n=500000 options='false' type='Inflate' 2.13 % ±3.24% ±4.32% ±5.64% 21:12:01 zlib/creation.js n=500000 options='false' type='InflateRaw' 0.63 % ±4.03% ±5.36% ±6.99% 21:12:01 zlib/creation.js n=500000 options='false' type='Unzip' 1.14 % ±2.65% ±3.53% ±4.60% 21:12:01 zlib/creation.js n=500000 options='true' type='BrotliCompress' * -5.69 % ±5.03% ±6.69% ±8.72% 21:12:01 zlib/creation.js n=500000 options='true' type='BrotliDecompress' -3.75 % ±3.77% ±5.03% ±6.55% 21:12:01 zlib/creation.js n=500000 options='true' type='Deflate' -1.67 % ±4.66% ±6.20% ±8.07% 21:12:01 zlib/creation.js n=500000 options='true' type='DeflateRaw' -0.05 % ±3.64% ±4.84% ±6.30% 21:12:01 zlib/creation.js n=500000 options='true' type='Gunzip' 1.26 % ±3.93% ±5.23% ±6.80% 21:12:01 zlib/creation.js n=500000 options='true' type='Gzip' -0.04 % ±3.52% ±4.68% ±6.09% 21:12:01 zlib/creation.js n=500000 options='true' type='Inflate' -1.45 % ±3.35% ±4.46% ±5.81% 21:12:01 zlib/creation.js n=500000 options='true' type='InflateRaw' 1.38 % ±3.32% ±4.41% ±5.74% 21:12:01 zlib/creation.js n=500000 options='true' type='Unzip' 1.95 % ±4.54% ±6.04% ±7.86% 21:12:01 zlib/deflate.js n=400000 inputLen=1024 method='createDeflate' ** -13.68 % ±9.53% ±12.69% ±16.52% 21:12:01 zlib/deflate.js n=400000 inputLen=1024 method='deflate' 3.57 % ±5.83% ±7.76% ±10.11% 21:12:01 zlib/deflate.js n=400000 inputLen=1024 method='deflateSync' ** 15.18 % ±9.89% ±13.16% ±17.13% 21:12:01 zlib/inflate.js n=400000 inputLen=1024 method='inflate' 1.97 % ±2.90% ±3.85% ±5.02% 21:12:01 zlib/inflate.js n=400000 inputLen=1024 method='inflateSync' -3.59 % ±4.24% ±5.65% ±7.35% 21:12:01 zlib/pipe.js algorithm='brotli' type='buffer' duration=5 inputLen=1024 -0.95 % ±2.85% ±3.79% ±4.95% 21:12:01 zlib/pipe.js algorithm='brotli' type='string' duration=5 inputLen=1024 0.50 % ±0.75% ±1.01% ±1.34% 21:12:01 zlib/pipe.js algorithm='gzip' type='buffer' duration=5 inputLen=1024 ** -7.60 % ±4.75% ±6.32% ±8.23% 21:12:01 zlib/pipe.js algorithm='gzip' type='string' duration=5 inputLen=1024 *** -9.56 % ±5.09% ±6.80% ±8.91% 21:12:01 21:12:01 Be aware that when doing many comparisons the risk of a false-positive 21:12:01 result increases. In this case there are 28 comparisons, you can thus 21:12:01 expect the following amount of false-positive results: 21:12:01 1.40 false positives, when considering a 5% risk acceptance (*, **, ***), 21:12:01 0.28 false positives, when considering a 1% risk acceptance (**, ***), 21:12:01 0.03 false positives, when considering a 0.1% risk acceptance (***) |
Sorry, something went wrong.
|
Updated zlib to 8cd0fc1 hoping that removes the performance regression. Worth a shot. Benchmark CI: https://ci.nodejs.org/view/Node.js%20benchmark/job/benchmark-node-micro-benchmarks/671/ UPDATE: Performance regression is still there.confidence improvement accuracy (*) (**) (***) 06:15:54 zlib/createInflate.js n=100 chunkLen=1024 inputLen=16777216 *** -32.81 % ±7.32% ±9.74% ±12.68% 06:15:54 zlib/creation.js n=500000 options='false' type='BrotliCompress' 0.07 % ±4.29% ±5.71% ±7.43% 06:15:54 zlib/creation.js n=500000 options='false' type='BrotliDecompress' 1.44 % ±4.67% ±6.22% ±8.10% 06:15:54 zlib/creation.js n=500000 options='false' type='Deflate' -1.44 % ±4.23% ±5.63% ±7.33% 06:15:54 zlib/creation.js n=500000 options='false' type='DeflateRaw' 1.77 % ±3.83% ±5.10% ±6.63% 06:15:54 zlib/creation.js n=500000 options='false' type='Gunzip' -0.83 % ±3.23% ±4.30% ±5.60% 06:15:54 zlib/creation.js n=500000 options='false' type='Gzip' -2.02 % ±4.17% ±5.56% ±7.26% 06:15:54 zlib/creation.js n=500000 options='false' type='Inflate' 1.38 % ±4.58% ±6.09% ±7.94% 06:15:54 zlib/creation.js n=500000 options='false' type='InflateRaw' 0.34 % ±3.16% ±4.22% ±5.50% 06:15:54 zlib/creation.js n=500000 options='false' type='Unzip' -0.40 % ±3.89% ±5.17% ±6.73% 06:15:54 zlib/creation.js n=500000 options='true' type='BrotliCompress' 0.99 % ±4.43% ±5.89% ±7.68% 06:15:54 zlib/creation.js n=500000 options='true' type='BrotliDecompress' 2.25 % ±4.75% ±6.32% ±8.22% 06:15:54 zlib/creation.js n=500000 options='true' type='Deflate' 0.09 % ±4.08% ±5.44% ±7.09% 06:15:54 zlib/creation.js n=500000 options='true' type='DeflateRaw' 0.01 % ±3.40% ±4.52% ±5.89% 06:15:54 zlib/creation.js n=500000 options='true' type='Gunzip' 2.89 % ±3.80% ±5.06% ±6.59% 06:15:54 zlib/creation.js n=500000 options='true' type='Gzip' 0.82 % ±3.95% ±5.26% ±6.85% 06:15:54 zlib/creation.js n=500000 options='true' type='Inflate' * 4.02 % ±3.84% ±5.12% ±6.67% 06:15:54 zlib/creation.js n=500000 options='true' type='InflateRaw' -2.01 % ±4.08% ±5.43% ±7.07% 06:15:54 zlib/creation.js n=500000 options='true' type='Unzip' 0.61 % ±3.86% ±5.15% ±6.70% 06:15:54 zlib/deflate.js n=400000 inputLen=1024 method='createDeflate' * -10.74 % ±10.28% ±13.69% ±17.84% 06:15:54 zlib/deflate.js n=400000 inputLen=1024 method='deflate' -1.57 % ±6.50% ±8.65% ±11.26% 06:15:54 zlib/deflate.js n=400000 inputLen=1024 method='deflateSync' ** -15.64 % ±9.36% ±12.46% ±16.22% 06:15:54 zlib/inflate.js n=400000 inputLen=1024 method='inflate' 0.51 % ±4.24% ±5.65% ±7.36% 06:15:54 zlib/inflate.js n=400000 inputLen=1024 method='inflateSync' * -5.03 % ±3.86% ±5.14% ±6.69% 06:15:54 zlib/pipe.js algorithm='brotli' type='buffer' duration=5 inputLen=1024 1.34 % ±3.25% ±4.33% ±5.65% 06:15:54 zlib/pipe.js algorithm='brotli' type='string' duration=5 inputLen=1024 -0.00 % ±0.94% ±1.25% ±1.63% 06:15:54 zlib/pipe.js algorithm='gzip' type='buffer' duration=5 inputLen=1024 ** -7.52 % ±5.36% ±7.14% ±9.31% 06:15:54 zlib/pipe.js algorithm='gzip' type='string' duration=5 inputLen=1024 *** -7.60 % ±4.10% ±5.45% ±7.09% 06:15:54 06:15:54 Be aware that when doing many comparisons the risk of a false-positive 06:15:54 result increases. In this case there are 28 comparisons, you can thus 06:15:54 expect the following amount of false-positive results: 06:15:54 1.40 false positives, when considering a 5% risk acceptance (*, **, ***), 06:15:54 0.28 false positives, when considering a 1% risk acceptance (**, ***), 06:15:54 0.03 false positives, when considering a 0.1% risk acceptance (***) |
Sorry, something went wrong.
There was a problem hiding this comment.
Do we need to update/run license-builder.sh for this? And might this be better living in tools?
Sorry, something went wrong.
There was a problem hiding this comment.
¯\_(ツ)_/¯. It's a copy of the one in tools/v8_gypfiles/GN-scraper.py which we don't currently have in tools/license-builder.sh.
If I remember correctly I tried to reference the existing script/put the script in tools but couldn't get the gyp file to call it successfully via <!@pymod_do_main(.... Maybe someone else in @nodejs/gyp or @nodejs/python can advise.
Sorry, something went wrong.
|
I'm guessing there's a simple fix for the compilation failure with quic enabled on Windows. @nodejs/quic 2020-10-17T10:55:35.3764632Z D:\a\node\node\deps\zlib\cpu_features.c(52,1): fatal error C1189: #error: cpu_features.c CPU feature detection in not defined for your platform [D:\a\node\node\deps\zlib\zlib.vcxproj] |
Sorry, something went wrong.
Restructure the zlib.gyp file based on the upstream gn file, breaking out the files with optimizations that need additional compiler flags. Use a copy of the GN-scraper.py script to reduce the amount of hand editing when the zlib dependency is updated.
|
Did a quick rebase. |
Sorry, something went wrong.
|
FWIW it appears the createInflate benchmark is the only one that shows a consistent performance regression: zlib/createInflate.js n=100 chunkLen=1024 inputLen=16777216 *** -25.68 % ±8.19% ±10.90% ±14.19% |
Sorry, something went wrong.
| 'type': 'static_library', | ||
| 'conditions': [ | ||
| ['OS!="win" or llvm_version!="0.0"', { | ||
| 'cflags': [ |
There was a problem hiding this comment.
Is this getting used at all? The latest test-macOS / test-macOS (pull_request) logs don't seem to include the cflags for this target in https://github.com/nodejs/node/pull/33044/checks?check_run_id=3008235129#step:5:1336:
cc -o /Users/runner/work/node/node/out/Release/obj.target/zlib_crc32_simd/deps/zlib/crc32_simd.o ../deps/zlib/crc32_simd.c '-DV8_DEPRECATION_WARNINGS' '-DV8_IMMINENT_DEPRECATION_WARNINGS' '-D_GLIBCXX_USE_CXX11_ABI=1' '-D_DARWIN_USE_64_BIT_INODE=1' '-DOPENSSL_NO_PINSHARED' '-DOPENSSL_THREADS' '-DCRC32_SIMD_SSE42_PCLMUL' '-DZLIB_IMPLEMENTATION' -I../deps/zlib -O3 -gdwarf-2 -mmacosx-version-min=10.13 -arch x86_64 -Wall -Wendif-labels -W -Wno-unused-parameter -fno-strict-aliasing -MMD -MF /Users/runner/work/node/node/out/Release/.deps//Users/runner/work/node/node/out/Release/obj.target/zlib_crc32_simd/deps/zlib/crc32_simd.o.d.raw -c
Sorry, something went wrong.
|
To follow this ticket :) |
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Restructure the gyp file based on the upstream gn file, breaking out
the files with optimizations that need additional compiler flags.
Use a copy of the GN-scraper.py script to reduce the amount of hand
editing when the zlib dependency is updated.
Refs: #32553
Refs: #33019
Fixes: #32856
cc @nodejs/build-files @nodejs/zlib
Checklist