Skip to content

Cache complex union and intersection relations #37910

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jan 5, 2021

Conversation

ahejlsberg
Copy link
Member

We've discussed caching union and intersection relations before, but had concerns about the increased memory usage. This PR introduces caching for complex union and intersection relations, specifically those than have a combined number of constituents of at least 4. This ideally strikes a balance between time and memory.

This PR improves check time for our "compiler with unions" code base by about 12%.

@ahejlsberg
Copy link
Member Author

@typescript-bot perf test this

@typescript-bot
Copy link
Collaborator

typescript-bot commented Apr 11, 2020

Heya @ahejlsberg, I've started to run the perf test suite on this PR at 1cd5b0a. You can monitor the build here.

Update: The results are in!

@ahejlsberg ahejlsberg changed the title Cache complex union intersection relations Cache complex union and intersection relations Apr 11, 2020
@typescript-bot
Copy link
Collaborator

@ahejlsberg
The results of the perf run you requested are in!

Here they are:

Comparison Report - master..37910

Metric master 37910 Delta Best Worst
Angular - node (v10.16.3, x64)
Memory used 327,719k (± 0.03%) 327,218k (± 0.02%) -501k (- 0.15%) 327,025k 327,403k
Parse Time 1.64s (± 0.35%) 1.64s (± 0.55%) +0.00s (+ 0.12%) 1.62s 1.66s
Bind Time 0.90s (± 0.85%) 0.89s (± 1.00%) -0.00s (- 0.11%) 0.87s 0.91s
Check Time 4.76s (± 0.36%) 4.78s (± 0.55%) +0.02s (+ 0.38%) 4.73s 4.85s
Emit Time 5.34s (± 0.63%) 5.35s (± 0.70%) +0.02s (+ 0.32%) 5.26s 5.44s
Total Time 12.63s (± 0.30%) 12.67s (± 0.50%) +0.03s (+ 0.28%) 12.49s 12.80s
Monaco - node (v10.16.3, x64)
Memory used 327,070k (± 0.02%) 327,107k (± 0.02%) +37k (+ 0.01%) 327,005k 327,311k
Parse Time 1.27s (± 0.66%) 1.27s (± 0.63%) -0.00s (- 0.08%) 1.25s 1.28s
Bind Time 0.77s (± 0.47%) 0.78s (± 0.57%) +0.00s (+ 0.52%) 0.77s 0.79s
Check Time 4.77s (± 0.36%) 4.77s (± 0.40%) 0.00s ( 0.00%) 4.73s 4.83s
Emit Time 2.92s (± 0.57%) 2.93s (± 0.78%) +0.01s (+ 0.24%) 2.89s 2.98s
Total Time 9.73s (± 0.28%) 9.74s (± 0.29%) +0.01s (+ 0.13%) 9.69s 9.80s
TFS - node (v10.16.3, x64)
Memory used 292,114k (± 0.04%) 292,063k (± 0.02%) -51k (- 0.02%) 291,925k 292,252k
Parse Time 0.96s (± 0.60%) 0.96s (± 1.04%) +0.00s (+ 0.31%) 0.94s 0.99s
Bind Time 0.74s (± 0.46%) 0.74s (± 1.00%) 0.00s ( 0.00%) 0.72s 0.76s
Check Time 4.30s (± 0.32%) 4.33s (± 0.75%) +0.03s (+ 0.58%) 4.23s 4.42s
Emit Time 3.06s (± 1.15%) 3.06s (± 0.65%) -0.00s (- 0.00%) 3.02s 3.10s
Total Time 9.06s (± 0.46%) 9.09s (± 0.50%) +0.03s (+ 0.35%) 9.02s 9.24s
material-ui - node (v10.16.3, x64)
Memory used 451,089k (± 0.01%) 454,746k (± 0.02%) +3,657k (+ 0.81%) 454,561k 455,059k
Parse Time 1.79s (± 0.66%) 1.78s (± 0.49%) -0.01s (- 0.61%) 1.76s 1.80s
Bind Time 0.68s (± 0.72%) 0.68s (± 0.85%) -0.00s (- 0.15%) 0.67s 0.70s
Check Time 12.63s (± 0.44%) 12.60s (± 0.56%) -0.03s (- 0.25%) 12.47s 12.74s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 15.10s (± 0.39%) 15.06s (± 0.50%) -0.04s (- 0.28%) 14.94s 15.23s
Angular - node (v12.1.0, x64)
Memory used 303,246k (± 0.03%) 302,805k (± 0.02%) -442k (- 0.15%) 302,660k 303,044k
Parse Time 1.59s (± 0.84%) 1.58s (± 0.60%) -0.01s (- 0.56%) 1.57s 1.61s
Bind Time 0.88s (± 0.83%) 0.88s (± 0.78%) -0.01s (- 0.79%) 0.86s 0.89s
Check Time 4.68s (± 0.55%) 4.64s (± 0.30%) -0.04s (- 0.83%) 4.62s 4.67s
Emit Time 5.53s (± 0.86%) 5.48s (± 0.57%) -0.05s (- 0.89%) 5.43s 5.55s
Total Time 12.69s (± 0.54%) 12.58s (± 0.24%) -0.11s (- 0.86%) 12.51s 12.66s
Monaco - node (v12.1.0, x64)
Memory used 307,049k (± 0.02%) 307,087k (± 0.01%) +38k (+ 0.01%) 306,967k 307,143k
Parse Time 1.22s (± 0.39%) 1.23s (± 0.49%) +0.00s (+ 0.41%) 1.21s 1.24s
Bind Time 0.75s (± 0.80%) 0.74s (± 0.92%) -0.00s (- 0.00%) 0.73s 0.76s
Check Time 4.59s (± 0.34%) 4.61s (± 0.60%) +0.01s (+ 0.33%) 4.54s 4.66s
Emit Time 2.98s (± 0.88%) 2.99s (± 1.07%) +0.02s (+ 0.54%) 2.92s 3.06s
Total Time 9.54s (± 0.34%) 9.57s (± 0.51%) +0.04s (+ 0.37%) 9.47s 9.68s
TFS - node (v12.1.0, x64)
Memory used 274,341k (± 0.01%) 274,362k (± 0.01%) +21k (+ 0.01%) 274,255k 274,457k
Parse Time 0.94s (± 0.85%) 0.94s (± 1.01%) +0.01s (+ 0.75%) 0.92s 0.97s
Bind Time 0.70s (± 0.95%) 0.71s (± 0.99%) +0.01s (+ 0.85%) 0.70s 0.72s
Check Time 4.22s (± 0.87%) 4.24s (± 0.51%) +0.02s (+ 0.36%) 4.21s 4.31s
Emit Time 3.10s (± 0.79%) 3.10s (± 0.84%) +0.01s (+ 0.19%) 3.05s 3.17s
Total Time 8.96s (± 0.48%) 9.00s (± 0.46%) +0.04s (+ 0.40%) 8.91s 9.10s
material-ui - node (v12.1.0, x64)
Memory used 428,383k (± 0.01%) 431,384k (± 0.01%) +3,001k (+ 0.70%) 431,265k 431,552k
Parse Time 1.75s (± 0.47%) 1.76s (± 0.58%) +0.00s (+ 0.23%) 1.74s 1.79s
Bind Time 0.64s (± 1.07%) 0.64s (± 1.25%) +0.00s (+ 0.31%) 0.62s 0.66s
Check Time 11.27s (± 0.41%) 11.24s (± 0.48%) -0.03s (- 0.28%) 11.15s 11.39s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 13.66s (± 0.33%) 13.63s (± 0.43%) -0.03s (- 0.18%) 13.56s 13.79s
Angular - node (v8.9.0, x64)
Memory used 322,585k (± 0.01%) 322,169k (± 0.01%) -416k (- 0.13%) 322,059k 322,256k
Parse Time 2.12s (± 0.37%) 2.13s (± 0.46%) +0.01s (+ 0.52%) 2.11s 2.16s
Bind Time 0.92s (± 0.88%) 0.93s (± 0.75%) +0.01s (+ 0.98%) 0.91s 0.94s
Check Time 5.47s (± 0.90%) 5.43s (± 2.06%) -0.04s (- 0.70%) 5.21s 5.58s
Emit Time 6.19s (± 1.85%) 6.35s (± 1.62%) +0.16s (+ 2.57%) 6.13s 6.61s
Total Time 14.69s (± 0.71%) 14.84s (± 0.72%) +0.14s (+ 0.97%) 14.47s 14.96s
Monaco - node (v8.9.0, x64)
Memory used 325,536k (± 0.02%) 325,583k (± 0.02%) +47k (+ 0.01%) 325,505k 325,786k
Parse Time 1.55s (± 0.49%) 1.56s (± 0.45%) +0.01s (+ 0.97%) 1.55s 1.58s
Bind Time 0.89s (± 0.91%) 0.91s (± 1.09%) +0.02s (+ 1.68%) 0.89s 0.93s
Check Time 5.36s (± 0.25%) 5.43s (± 0.51%) +0.07s (+ 1.34%) 5.38s 5.51s
Emit Time 3.52s (± 0.49%) 3.50s (± 0.48%) -0.01s (- 0.40%) 3.47s 3.54s
Total Time 11.32s (± 0.15%) 11.41s (± 0.31%) +0.09s (+ 0.79%) 11.34s 11.49s
TFS - node (v8.9.0, x64)
Memory used 291,502k (± 0.02%) 291,542k (± 0.01%) +40k (+ 0.01%) 291,472k 291,607k
Parse Time 1.26s (± 0.44%) 1.26s (± 0.56%) -0.00s (- 0.08%) 1.25s 1.28s
Bind Time 0.74s (± 0.50%) 0.75s (± 0.77%) +0.01s (+ 0.67%) 0.74s 0.76s
Check Time 5.06s (± 1.70%) 5.06s (± 1.68%) +0.00s (+ 0.04%) 4.85s 5.19s
Emit Time 3.17s (± 1.99%) 3.20s (± 2.89%) +0.03s (+ 1.01%) 3.06s 3.41s
Total Time 10.23s (± 0.40%) 10.27s (± 0.35%) +0.04s (+ 0.36%) 10.15s 10.32s
material-ui - node (v8.9.0, x64)
Memory used 453,525k (± 0.01%) 457,197k (± 0.01%) +3,672k (+ 0.81%) 457,111k 457,322k
Parse Time 2.11s (± 0.39%) 2.13s (± 0.69%) +0.02s (+ 1.04%) 2.11s 2.16s
Bind Time 0.82s (± 0.92%) 0.81s (± 1.03%) -0.02s (- 2.30%) 0.79s 0.83s
Check Time 16.67s (± 0.66%) 16.61s (± 0.83%) -0.07s (- 0.40%) 16.23s 16.86s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 19.60s (± 0.56%) 19.54s (± 0.72%) -0.06s (- 0.32%) 19.15s 19.80s
Angular - node (v8.9.0, x86)
Memory used 185,686k (± 0.03%) 185,483k (± 0.02%) -203k (- 0.11%) 185,421k 185,583k
Parse Time 2.06s (± 0.55%) 2.07s (± 0.77%) +0.01s (+ 0.34%) 2.05s 2.12s
Bind Time 1.07s (± 0.63%) 1.07s (± 0.62%) -0.00s (- 0.37%) 1.05s 1.08s
Check Time 4.99s (± 0.53%) 5.02s (± 0.55%) +0.03s (+ 0.56%) 4.97s 5.09s
Emit Time 6.02s (± 0.93%) 6.07s (± 0.62%) +0.05s (+ 0.75%) 6.00s 6.17s
Total Time 14.15s (± 0.55%) 14.22s (± 0.40%) +0.08s (+ 0.55%) 14.10s 14.35s
Monaco - node (v8.9.0, x86)
Memory used 185,381k (± 0.03%) 185,378k (± 0.02%) -4k (- 0.00%) 185,311k 185,447k
Parse Time 1.60s (± 0.54%) 1.61s (± 0.99%) +0.00s (+ 0.31%) 1.57s 1.65s
Bind Time 0.77s (± 1.12%) 0.77s (± 1.04%) -0.00s (- 0.00%) 0.76s 0.79s
Check Time 5.42s (± 0.41%) 5.48s (± 0.58%) +0.06s (+ 1.09%) 5.42s 5.56s
Emit Time 2.86s (± 0.90%) 2.90s (± 1.03%) +0.03s (+ 1.15%) 2.85s 2.98s
Total Time 10.66s (± 0.44%) 10.76s (± 0.65%) +0.10s (+ 0.90%) 10.66s 10.99s
TFS - node (v8.9.0, x86)
Memory used 166,920k (± 0.02%) 166,908k (± 0.03%) -12k (- 0.01%) 166,797k 167,020k
Parse Time 1.29s (± 0.54%) 1.29s (± 0.73%) +0.00s (+ 0.08%) 1.27s 1.32s
Bind Time 0.71s (± 0.63%) 0.72s (± 1.42%) +0.01s (+ 1.26%) 0.71s 0.75s
Check Time 4.65s (± 0.37%) 4.65s (± 0.59%) +0.00s (+ 0.06%) 4.59s 4.71s
Emit Time 2.97s (± 0.74%) 2.98s (± 0.93%) +0.01s (+ 0.44%) 2.89s 3.03s
Total Time 9.62s (± 0.30%) 9.64s (± 0.51%) +0.02s (+ 0.25%) 9.52s 9.74s
material-ui - node (v8.9.0, x86)
Memory used 256,799k (± 0.01%) 258,774k (± 0.01%) +1,975k (+ 0.77%) 258,677k 258,844k
Parse Time 2.20s (± 0.83%) 2.19s (± 0.48%) -0.01s (- 0.41%) 2.16s 2.21s
Bind Time 0.69s (± 0.81%) 0.69s (± 1.43%) 0.00s ( 0.00%) 0.67s 0.72s
Check Time 15.30s (± 0.73%) 15.22s (± 0.62%) -0.08s (- 0.51%) 14.98s 15.36s
Emit Time 0.00s (± 0.00%) 0.00s (± 0.00%) 0.00s ( NaN%) 0.00s 0.00s
Total Time 18.18s (± 0.63%) 18.10s (± 0.54%) -0.08s (- 0.47%) 17.86s 18.27s
System
Machine Namets-ci-ubuntu
Platformlinux 4.4.0-166-generic
Architecturex64
Available Memory16 GB
Available Memory1 GB
CPUs4 × Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
Hosts
  • node (v10.16.3, x64)
  • node (v12.1.0, x64)
  • node (v8.9.0, x64)
  • node (v8.9.0, x86)
Scenarios
  • Angular - node (v10.16.3, x64)
  • Angular - node (v12.1.0, x64)
  • Angular - node (v8.9.0, x64)
  • Angular - node (v8.9.0, x86)
  • Monaco - node (v10.16.3, x64)
  • Monaco - node (v12.1.0, x64)
  • Monaco - node (v8.9.0, x64)
  • Monaco - node (v8.9.0, x86)
  • TFS - node (v10.16.3, x64)
  • TFS - node (v12.1.0, x64)
  • TFS - node (v8.9.0, x64)
  • TFS - node (v8.9.0, x86)
  • material-ui - node (v10.16.3, x64)
  • material-ui - node (v12.1.0, x64)
  • material-ui - node (v8.9.0, x64)
  • material-ui - node (v8.9.0, x86)
Benchmark Name Iterations
Current 37910 10
Baseline master 10

@ahejlsberg
Copy link
Member Author

Not much effect on our perf test suites, so not clear how valuable this is. But it definitely has a pretty positive effect on the compiler with unions code base.

@ahejlsberg
Copy link
Member Author

@weswigham @RyanCavanaugh FYI

@DanielRosenwasser
Copy link
Member

I'm surprised there's not a more pronounced effect on Material UI but I guess it makes sense

@sandersn sandersn added the For Uncommitted Bug PR for untriaged, rejected, closed or missing bug label May 5, 2020
Copy link
Member

@weswigham weswigham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't remember if we chatted about this back in April, but I'm still assigned, so 🤷. It'd be nice to enable the compiler with unions perf test (IIRC it was disabled because it caused an older version of the compiler to crash, but that shouldn't be an issue right now - cc @rbuckton - you know if/how we can turn it on?) to show the beneficial effect this actually has, but this seems OK, implementation-wise.

@weswigham
Copy link
Member

@rbuckton know if we can go about enabling that perf test at some point (and get some historical data for it)?

@ahejlsberg we going to merge this at some point? Should it have a milestone? It looks like it needs a merge for a baseline update.

# Conflicts:
#	tests/baselines/reference/classPropertyErrorOnNameOnly.errors.txt
@ahejlsberg ahejlsberg merged commit e1fda83 into master Jan 5, 2021
@ahejlsberg ahejlsberg deleted the cacheComplexUnionIntersectionRelations branch January 5, 2021 01:15
Zzzen pushed a commit to Zzzen/TypeScript that referenced this pull request Jan 16, 2021
* Cache complex union/intersection relations

* Accept new baselines

* Accept new baselines
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Author: Team For Uncommitted Bug PR for untriaged, rejected, closed or missing bug
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

5 participants