🔎 Search Terms
memory, oom, resolveObjectTypeMembers, getReducedType
🙁 Actual behavior
we have a ~37k file TS program at work and checking it on 7.0 needs ~13 GiB of heap single threaded (~16 GiB peak), and about double that with the default 4 checkers. people keep running out of memory, esp with the editor open too
i profiled it and there are two spots where the checker builds a ton of stuff it never uses:
- looking up one property on an instantiated class/interface (expect(x).toBe, arr.map etc) instantiates every member of the type plus everything it inherits. on our program thats ~15.7M member symbols and only ~3M of them ever get used. isWeakType, getSingleSignature and friends also build the whole table just to count stuff
- getReducedType builds the combined property for every name in an intersection just to check if it reduces to never. only a handful of names can actually do that (shared across constituents with a literal type, or private)
🙂 Expected behavior
only build what's needed. i have changes for both that give the same output as main. full go test suite passes (also with multiple checkers and -race) and i added tests for the edge cases i hit along the way
typescript-benchmarking tsc scenarios with both changes, 4 checkers, median of 3:
|
heap |
peak rss |
check time |
| vscode |
6.22 → 5.65 GiB (−9%) |
9.75 → 8.97 GiB (−8%) |
same |
| mui-docs |
4.75 → 2.72 GiB (−43%) |
6.80 → 4.03 GiB (−41%) |
5.89 → 3.86s (−35%) |
| xstate-main |
0.55 → 0.49 GiB (−10%) |
0.67 → 0.61 GiB (−8%) |
same |
| webpack |
0.94 → 0.84 GiB (−11%) |
1.15 → 1.00 GiB (−13%) |
0.51 → 0.48s (−6%) |
| Compiler |
−4% |
−6% |
same |
| Compiler-Unions |
−8% |
−2% |
same |
single threaded looks about the same (vscode −8% heap, mui-docs −35%). the intersection change is basically all of the mui-docs win and the member change is the ~8-10% on everything else. on our own program heap goes 13.1 → 10.4 GiB
output matched main on every run except TS6059 on mui-docs, but that one flakes on main too (#64477)
opened #64475 (members) and #64476 (intersections) as drafts. #64475 overlaps with #64372, and a lot of the complexity in it is there to keep the half built members behavior that #64372 removes. so if that lands first ill rebase and it should get a lot simpler
Additional information about the issue
--generateTrace couldnt handle a program this size (the types dump went past 160 GB before i killed it) so i found these with go heap profiles and some counters hacked into the checker
fwiw i used claude code for a lot of the digging and the code. ive reviewed it and ill be the one following up on review
🔎 Search Terms
memory, oom, resolveObjectTypeMembers, getReducedType
🙁 Actual behavior
we have a ~37k file TS program at work and checking it on 7.0 needs ~13 GiB of heap single threaded (~16 GiB peak), and about double that with the default 4 checkers. people keep running out of memory, esp with the editor open too
i profiled it and there are two spots where the checker builds a ton of stuff it never uses:
🙂 Expected behavior
only build what's needed. i have changes for both that give the same output as main. full go test suite passes (also with multiple checkers and -race) and i added tests for the edge cases i hit along the way
typescript-benchmarking tsc scenarios with both changes, 4 checkers, median of 3:
single threaded looks about the same (vscode −8% heap, mui-docs −35%). the intersection change is basically all of the mui-docs win and the member change is the ~8-10% on everything else. on our own program heap goes 13.1 → 10.4 GiB
output matched main on every run except TS6059 on mui-docs, but that one flakes on main too (#64477)
opened #64475 (members) and #64476 (intersections) as drafts. #64475 overlaps with #64372, and a lot of the complexity in it is there to keep the half built members behavior that #64372 removes. so if that lands first ill rebase and it should get a lot simpler
Additional information about the issue
--generateTrace couldnt handle a program this size (the types dump went past 160 GB before i killed it) so i found these with go heap profiles and some counters hacked into the checker
fwiw i used claude code for a lot of the digging and the code. ive reviewed it and ill be the one following up on review