test: increase RAM requirement for intensive tests - #7772
Conversation
Sorry, something went wrong.
test-fs-read-buffer-tostring-fail and test-fs-readfile-tostring-fail have been timing out on Raspberry Pi 3 devices on the continuous integration server. These devices have 1 Gb of RAM and the tests are memory intensive. Previous checks for memory intensive tests used a 512 Mb cut-off, but that was probably instituted when we only had Pi 1 devices. Consequently, this change increases the threshold for memory-intensive tests to 1 Gb and adds that threshold to test-fs-readfile-tostring-fail.
|
Sample CI failures this change is trying to address: not ok 45 parallel/test-fs-read-buffer-tostring-fail # TIMEOUT # (node:5468) DeprecationWarning: fs.read's legacy String interface is deprecated. Use the Buffer API as mentioned in the documentation instead. --- duration_ms: 120.147 not ok 48 parallel/test-fs-readfile-tostring-fail # TIMEOUT --- duration_ms: 120.179 |
Sorry, something went wrong.
Sorry, something went wrong.
|
LGTM |
Sorry, something went wrong.
|
Hmm, what tests on which hardware are we ignoring after this? (I'm not comfortable about this w/o having that knowledge better available.) |
Sorry, something went wrong.
|
Changes LGTM, but I'd also like to know what tests are now skipped. |
Sorry, something went wrong.
|
Just clicked through everything in the CI results and the only thing this causes to skip the tests are the Pi 2 and Pi 3 devices. (Pi 1 devices were already skipping the tests.) I'll rerun CI so I can confirm that it doesn't affect FreeBSD (build failure) and the plinux hosts (were hung earlier today). |
Sorry, something went wrong.
Sorry, something went wrong.
|
Here are the tests affected (but again, only on Pi devices): $ grep -lR enoughTestMem test | grep -v test/common.js test/addons/stringbytes-external-exceed-max/test-stringbytes-external-at-max.js test/addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-ascii.js test/addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-base64.js test/addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-binary.js test/addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-hex.js test/addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-utf8.js test/addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-2.js test/addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max.js test/parallel/test-fs-read-buffer-tostring-fail.js $ This change also adds test/parallel/test-fs-readfile-tostring-fail.js to that list. |
Sorry, something went wrong.
|
Confirmed that FreeBSD and plinux are not affected by this either. So it's just the Pi devices. |
Sorry, something went wrong.
|
LGTM |
Sorry, something went wrong.
|
LGTM |
Sorry, something went wrong.
|
LGTM |
Sorry, something went wrong.
test-fs-read-buffer-tostring-fail and test-fs-readfile-tostring-fail have been timing out on Raspberry Pi 3 devices on the continuous integration server. These devices have 1 Gb of RAM and the tests are memory intensive. Previous checks for memory intensive tests used a 512 Mb cut-off, but that was probably instituted when we only had Pi 1 devices. Consequently, this change increases the threshold for memory-intensive tests to 1 Gb and adds that threshold to test-fs-readfile-tostring-fail. PR-URL: nodejs#7772 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
|
@Trott Did the other tests not timeout on RPi3? If so, we should find a better solution. |
Sorry, something went wrong.
test-fs-read-buffer-tostring-fail and test-fs-readfile-tostring-fail have been timing out on Raspberry Pi 3 devices on the continuous integration server. These devices have 1 Gb of RAM and the tests are memory intensive. Previous checks for memory intensive tests used a 512 Mb cut-off, but that was probably instituted when we only had Pi 1 devices. Consequently, this change increases the threshold for memory-intensive tests to 1 Gb and adds that threshold to test-fs-readfile-tostring-fail. PR-URL: #7772 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
|
@Fishrock123 The tests were already being skipped usually but not always on Pi2 and Pi3. I've only noticed this now and I don't have an explanation for that unless os.totalmem() is unreliable. For example, https://ci.nodejs.org/job/node-test-binary-arm/2703/ from July 1 (before this change). Pi 1: ok 22 addons/stringbytes-external-exceed-max/test-stringbytes-external-at-max # skip intensive toString tests due to memory confinements --- duration_ms: 1.753 ... ok 23 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max # skip intensive toString tests due to memory confinements --- duration_ms: 1.743 ... ok 24 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-ascii # skip intensive toString tests due to memory confinements --- duration_ms: 1.747 ... ok 25 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-base64 # skip intensive toString tests due to memory confinements --- duration_ms: 1.748 ... ok 26 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-binary # skip intensive toString tests due to memory confinements --- duration_ms: 1.854 ... ok 27 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-hex # skip intensive toString tests due to memory confinements --- duration_ms: 1.748 ... ok 28 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-utf8 # skip intensive toString tests due to memory confinements --- duration_ms: 1.750 ... ok 29 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-2 # skip intensive toString tests due to memory confinements --- duration_ms: 1.747 OK, that makes sense. It skips all those tests on the Pi 1 which has only 512 Mb or something like that. Let's check the Pi 2 run: ok 22 addons/stringbytes-external-exceed-max/test-stringbytes-external-at-max --- duration_ms: 2.135 ... ok 23 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max # skip intensive toString tests due to memory confinements --- duration_ms: 1.31 ... ok 24 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-ascii # skip intensive toString tests due to memory confinements --- duration_ms: 1.35 ... ok 25 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-base64 # skip intensive toString tests due to memory confinements --- duration_ms: 1.35 ... ok 26 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-binary --- duration_ms: 4.342 ... ok 27 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-hex --- duration_ms: 8.453 ... ok 28 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-1-utf8 --- duration_ms: 1.29 ... ok 29 addons/stringbytes-external-exceed-max/test-stringbytes-external-exceed-max-by-2 --- duration_ms: 2.32 Uh....OK, it skips some of those tests but not all of them. So os.totalmem() is returning different values on different instantiations of node processes on the same hardware? ???? ANYWAY, if someone wants to come up with a more nuanced way to handle these resource-intensive tests, I'm open to it. The trade-off will be complexity. The benefit of the existing methodology is simplicity. (I did think about making it a function and having each test pass the RAM threshold in bytes as an argument, but at that point, you start injecting different magic numbers into different tests. Blech. At least this way, there's only one magic number and it's in common.js only. Trade off for the simplicity is a lack of flexibility. I'm OK with that, personally.) |
Sorry, something went wrong.
test-fs-read-buffer-tostring-fail and test-fs-readfile-tostring-fail have been timing out on Raspberry Pi 3 devices on the continuous integration server. These devices have 1 Gb of RAM and the tests are memory intensive. Previous checks for memory intensive tests used a 512 Mb cut-off, but that was probably instituted when we only had Pi 1 devices. Consequently, this change increases the threshold for memory-intensive tests to 1 Gb and adds that threshold to test-fs-readfile-tostring-fail. PR-URL: #7772 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
test-fs-read-buffer-tostring-fail and test-fs-readfile-tostring-fail have been timing out on Raspberry Pi 3 devices on the continuous integration server. These devices have 1 Gb of RAM and the tests are memory intensive. Previous checks for memory intensive tests used a 512 Mb cut-off, but that was probably instituted when we only had Pi 1 devices. Consequently, this change increases the threshold for memory-intensive tests to 1 Gb and adds that threshold to test-fs-readfile-tostring-fail. PR-URL: #7772 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
test-fs-read-buffer-tostring-fail and test-fs-readfile-tostring-fail have been timing out on Raspberry Pi 3 devices on the continuous integration server. These devices have 1 Gb of RAM and the tests are memory intensive. Previous checks for memory intensive tests used a 512 Mb cut-off, but that was probably instituted when we only had Pi 1 devices. Consequently, this change increases the threshold for memory-intensive tests to 1 Gb and adds that threshold to test-fs-readfile-tostring-fail. PR-URL: #7772 Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl> Reviewed-By: Colin Ihrig <cjihrig@gmail.com> Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com> Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Checklist
Affected core subsystem(s)
test fs
Description of change
test-fs-read-buffer-tostring-fail and test-fs-readfile-tostring-fail
have been timing out on Raspberry Pi 3 devices on the continuous
integration server. These devices have 1 Gb of RAM and the tests are
memory intensive. Previous checks for memory intensive tests used a 512
Mb cut-off, but that was probably instituted when we only had Pi 1
devices.
Consequently, this change increases the threshold for memory-intensive
tests to 1 Gb and adds that threshold to test-fs-readfile-tostring-fail.