FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

perf(browser): improve browser parallelisation by sheremet-va · Pull Request #7665 · vitest-dev/vitest · GitHub

/ vitest Public

perf(browser): improve browser parallelisation - #7665

Merged
sheremet-va merged 28 commits into
vitest-dev:mainfrom
sheremet-va:perf/browser-parallelisation
Apr 17, 2025
ConversationCommits (28)ChecksFiles changed
Merged

perf(browser): improve browser parallelisation#7665
sheremet-va merged 28 commits into
vitest-dev:mainfrom
sheremet-va:perf/browser-parallelisation

Conversation

sheremet-va commented Mar 12, 2025 •
edited
Loading

Copy link
Copy Markdown
Member

Description

Fixes #7616
Fixes #6834

This made tests run 2 as fast as before in PW provider for me

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • It's really useful if your PR references an issue where it is discussed ahead of time. If the feature is substantial or introduces breaking changes without a discussion, PR might be closed.
  • Ideally, include a test that fails without this PR but passes with it.
  • Please, don't make changes to pnpm-lock.yaml unless you introduce a new test example.

Tests

  • Run the tests with pnpm test:ci.

Documentation

  • If you introduce new functionality, document it. You can run documentation with pnpm run docs command.

Changesets

  • Changes in changelog are generated from PR name. Please, make sure that it explains your changes in an understandable manner. Please, prefix changeset messages with feat:, fix:, perf:, docs:, or chore:.

netlify Bot commented Mar 12, 2025 •
edited
Loading

Copy link
Copy Markdown

✅ Deploy Preview for vitest-dev ready!

Name Link
🔨 Latest commit c58a06e
🔍 Latest deploy log https://app.netlify.com/sites/vitest-dev/deploys/67eab87debd281000854d2b0
😎 Deploy Preview https://deploy-preview-7665--vitest-dev.netlify.app
📱 Preview on mobile Toggle QR Code...



Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

pkg-pr-new Bot commented Mar 13, 2025 •
edited
Loading

Copy link
Copy Markdown
@vitest/browser

npm i https://pkg.pr.new/@vitest/browser@7665

@vitest/coverage-istanbul

npm i https://pkg.pr.new/@vitest/coverage-istanbul@7665

@vitest/coverage-v8

npm i https://pkg.pr.new/@vitest/coverage-v8@7665

@vitest/expect

npm i https://pkg.pr.new/@vitest/expect@7665

@vitest/mocker

npm i https://pkg.pr.new/@vitest/mocker@7665

@vitest/pretty-format

npm i https://pkg.pr.new/@vitest/pretty-format@7665

@vitest/runner

npm i https://pkg.pr.new/@vitest/runner@7665

@vitest/spy

npm i https://pkg.pr.new/@vitest/spy@7665

@vitest/snapshot

npm i https://pkg.pr.new/@vitest/snapshot@7665

@vitest/ui

npm i https://pkg.pr.new/@vitest/ui@7665

@vitest/utils

npm i https://pkg.pr.new/@vitest/utils@7665

vite-node

npm i https://pkg.pr.new/vite-node@7665

vitest

npm i https://pkg.pr.new/vitest@7665

@vitest/web-worker

npm i https://pkg.pr.new/@vitest/web-worker@7665

@vitest/ws-client

npm i https://pkg.pr.new/@vitest/ws-client@7665

commit: c58a06e

sheremet-va commented Mar 13, 2025

Copy link
Copy Markdown
Member Author

@nstepien can you check if this improves things for you? watch mode, no-isolation and debugging don't work properly now, but I think just running with --run should work

nstepien commented Mar 13, 2025 •
edited
Loading

Copy link
Copy Markdown
Contributor

With our unchanged config:

npx vitest run

 RUN  v3.0.8 D:/repo

4:04:53 PM [vite] (client) Re-optimizing dependencies because lockfile has changed
⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯

Vitest caught 1 unhandled error during the test run.
This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ Unhandled Error ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Error: [vitest-api]: Timeout calling "createTesters"
 ❯ Object.onTimeoutError node_modules/@vitest/browser/dist/index.js:3010:17
 ❯ Timeout._onTimeout node_modules/@vitest/browser/dist/index.js:2715:41
 ❯ listOnTimeout node:internal/timers:614:17
 ❯ process.processTimers node:internal/timers:549:7

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯


 Test Files   (232)
      Tests  53 passed (65)
     Errors  1 error
   Start at  16:04:53
   Duration  74.49s (transform 0ms, setup 1.57s, collect 2.46s, tests 0ms, environment 0ms, prepare 1.07s)

At first I though it was breaking the default reporter, but it's actually only running one test suite at a time.
Not sure if I need to configure anything differently, but this doesn't work well.

nstepien commented Mar 13, 2025

Copy link
Copy Markdown
Contributor

Another run in a subset of our codebase:

npx vitest run src/camp

 RUN  v3.0.8 D:/repo

 ✓  chromium  src/....test.tsx (13 tests) 13214ms
 ✓  chromium  src/....test.tsx (6 tests) 8096ms
 ✓  chromium  src/....test.tsx (11 tests) 7908ms
 ✓  chromium  src/....test.tsx (3 tests) 2447ms
 ✓  chromium  src/....test.tsx (3 tests) 1469ms
                                                                                              
 Test Files  5 passed (5)                                                                     
      Tests  36 passed (36)                                                                   
   Start at  16:16:19                                                                         
   Duration  46.19s (transform 0ms, setup 2.45s, collect 1.50s, tests 33.13s, environment 0ms, prepare 528ms)

it runs to completion, but it still only runs one test suite at a time.

sheremet-va commented Mar 13, 2025

Copy link
Copy Markdown
Member Author

it runs to completion, but it still only runs one test suite at a time.

This was a mistake on my part. Last minute if broke the parallelisation because of isolate: false

At first I though it was breaking the default reporter, but it's actually only running one test suite at a time.

Does it print any tests running? It seems like there is a timeout for createTesters - there shouldn't be one

nstepien commented Mar 13, 2025

Copy link
Copy Markdown
Contributor

Does it print any tests running?

It does, seems like the default reporter behaves as expected.

nstepien commented Mar 13, 2025

Copy link
Copy Markdown
Contributor

The latest build works better, but I still get the Timeout calling "createTesters" error before I can confirm whether parallelization has been fixed.

nstepien commented Mar 14, 2025

Copy link
Copy Markdown
Contributor

The latest builds work great now 👍

sheremet-va commented Mar 14, 2025

Copy link
Copy Markdown
Member Author

The latest builds work great now 👍

Is it any faster than before?

nstepien commented Mar 14, 2025

Copy link
Copy Markdown
Contributor

Absolutely, yes

sheremet-va commented Mar 14, 2025

Copy link
Copy Markdown
Member Author

Absolutely, yes

Can you share the numbers?

nstepien commented Mar 14, 2025

Copy link
Copy Markdown
Contributor

In our CI, before:

Duration  800.08s (transform 0ms, setup 679.36s, collect 877.16s, tests 1486.22s, environment 0ms, prepare 207.40s)

After, with maxWorkers: 8 because tests somehow become flaky with increased parallelization, we need to fix our tests:

Duration  360.87s (transform 0ms, setup 364.73s, collect 483.01s, tests 1519.42s, environment 0ms, prepare 64.00s)

Once we fix our flaky tests it'll be faster still.

sheremet-va added the p3-significant High priority enhancement (priority) label Mar 17, 2025

sheremet-va commented Mar 26, 2025

Copy link
Copy Markdown
Member Author

With this change, firefox now fails consistently on Windows 😞 I am also getting weird time outs in different tests

I think we should wait for #7377 to be fixed first before releasing this

sheremet-va mentioned this pull request Mar 31, 2025
Closed
6 tasks
sheremet-va marked this pull request as ready for review April 17, 2025 11:22
sheremet-va merged commit 816a5c5 into vitest-dev:main Apr 17, 2025
sheremet-va deleted the perf/browser-parallelisation branch April 17, 2025 11:22
This was referenced Apr 17, 2025
Closed
Closed
schogges pushed a commit to kumahq/kuma-gui that referenced this pull request Apr 23, 2025
chore(deps-dev): Bump vitest from 3.1.1 to 3.1.2 (#3781)
ecaab5d 
v3.1.2
   🐞 Bug Fixes
Add global chai variable in vitest/globals (fix: #7474)  -  by @​Jay-Karia in vitest-dev/vitest#7771 and vitest-dev/vitest#7474 (d9297)
Prevent modifying test.exclude when same object passed in coverage.exclude  -  by @​AriPerkkio in vitest-dev/vitest#7774 (c3751)
Fix already hoisted mock  -  by @​hi-ogawa in vitest-dev/vitest#7815 (773b1)
Fix test.scoped inheritance  -  by @​hi-ogawa in vitest-dev/vitest#7814 (db6c3)
Remove pointer-events-none after resizing the left panel  -  by @​alexprudhomme in vitest-dev/vitest#7811 (a7e77)
Default to run mode when stdin is not a TTY  -  by @​kentonv, @​hi-ogawa and @​sheremet-va in vitest-dev/vitest#7673 (6358f)
Use happy-dom/jsdom types for envionmentOptions  -  by @​hi-ogawa in vitest-dev/vitest#7795 (67430)
browser:
Fix transform error before browser server initialization  -  by @​hi-ogawa in vitest-dev/vitest#7783 (5f762)
Fix mocking from outside of root  -  by @​hi-ogawa in vitest-dev/vitest#7789 (03f55)
Scale iframe for non ui case  -  by @​hi-ogawa in vitest-dev/vitest#6512 (c3374)
coverage:
await profiler calls  -  by @​AriPerkkio in vitest-dev/vitest#7763 (795a6)
Expose profiling timers  -  by @​AriPerkkio in vitest-dev/vitest#7820 (5652b)
deps:
Update all non-major dependencies  -  in vitest-dev/vitest#7765 (7c3df)
Update all non-major dependencies  -  in vitest-dev/vitest#7831 (15701)
runner:
Correctly call test hooks and teardown functions  -  by @​sheremet-va in vitest-dev/vitest#7775 (3c00c)
Show stacktrace on test timeout error  -  by @​hi-ogawa in vitest-dev/vitest#7799 (df33b)
ui:
Load panel sizes from storage on initial load  -  by @​userquin in vitest-dev/vitest#7265 (6555d)
vite-node:
Named export should overwrite export all  -  by @​hi-ogawa in vitest-dev/vitest#7846 (5ba0d)
Add ERR_MODULE_NOT_FOUND code error if module cannot be loaded  -  by @​sheremet-va in vitest-dev/vitest#7776 (f9eac)
   🏎 Performance
browser: Improve browser parallelisation  -  by @​sheremet-va in vitest-dev/vitest#7665 (816a5)
renovate Bot mentioned this pull request Apr 24, 2025
Merged
1 task
nstepien mentioned this pull request Apr 24, 2025
Open
6 tasks
This was referenced Apr 24, 2025
Merged
Open
This was referenced May 26, 2025
Merged
Open
Closed
This was referenced Jun 3, 2025
Closed
Closed

mleimer commented Jul 3, 2025 •
edited
Loading

Copy link
Copy Markdown

I understand that this fix shall run multiple test files in parallel in browser-mode. However, I am unable to achieve that in a non-headless mode, i.e. only one test file is running at a time.

Repository
https://github.com/mleimer/vitest-browser-parallelism

Do I misunderstand anything or what am I missing out?

When setting browser.headless: true, the test files indeed run in parallel. Is there anyway to achieve the same without using headless?

sheremet-va commented Jul 3, 2025

Copy link
Copy Markdown
Member Author

I understand that this fix shall run multiple test files in parallel in browser-mode. However, I am unable to achieve that in a non-headless mode, i.e. only one test file is running at a time.

Repository https://github.com/mleimer/vitest-browser-parallelism

Do I misunderstand anything or what am I missing out?

When setting browser.headless: true, the test files indeed run in parallel. Is there anyway to achieve the same without using headless?

Tests run in parallel only in headless mode.

renovate Bot mentioned this pull request Aug 12, 2025
Open
1 task
This was referenced Aug 16, 2026
Merged
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Show hidden characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

cr-tracked p3-significant High priority enhancement (priority) upstream

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Running browser tests gradually decreases parallelism Browser tests don't run in parallel with browser.enabled = true

3 participants

Back | FazBrowse Home | New Git URL