Skip to content

Investigate and polish perf in OpenAPI pipeline #56829

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
captainsafia opened this issue Jul 16, 2024 · 1 comment
Open

Investigate and polish perf in OpenAPI pipeline #56829

captainsafia opened this issue Jul 16, 2024 · 1 comment
Assignees
Labels
area-minimal Includes minimal APIs, endpoint filters, parameter binding, request delegate generator etc area-mvc Includes: MVC, Actions and Controllers, Localization, CORS, most templates feature-openapi Perf

Comments

@captainsafia
Copy link
Member

We introduced microbenchmarks to the new OpenAPI-generation implementation in preview4. Since then, we've introduced a few more features to the implementation that have perf impacts (more transformers, $ref support, recursive schema transformers). This issue tracks doing a perf analysis on the implementation after all these changes are in to assess impact and trade-offs of making improvements here.

Here's the baseline for numbers run against 28acf62 for our current set of benchmarks.

Method EndpointCount Mean Error StdDev Op/s Gen 0 Gen 1 Gen 2 Allocated
GenerateDocument 10 366.4 μs 5.36 μs 5.74 μs 2,729.52 3.9063 - - 694 KB
GenerateDocument 100 3,365.4 μs 29.89 μs 27.96 μs 297.14 46.8750 15.6250 - 6,295 KB
GenerateDocument 1000 37,951.0 μs 228.55 μs 202.61 μs 26.35 200.0000 - - 62,314 KB
Method ElementCount Mean Error StdDev Op/s Gen 0 Gen 1 Gen 2 Allocated
OpenApiSchema_GetHashCode 1 260.1 ns 2.38 ns 2.11 ns 3,844,780.4 - - - -
OpenApiSchema_GetHashCode 10 260.1 ns 3.44 ns 3.05 ns 3,844,440.9 - - - -
OpenApiSchema_GetHashCode 100 262.2 ns 0.95 ns 0.84 ns 3,813,767.2 - - - -
Method TransformerCount Mean Error StdDev Op/s Gen 0 Gen 1 Gen 2 Allocated
ActivatedOperationTransformer 10 2.466 μs 0.0377 μs 0.0477 μs 405,514.3 0.0534 0.0114 - 7 KB
OperationTransformerAsDelegate 10 2.561 μs 0.0195 μs 0.0173 μs 390,494.5 0.0496 0.0114 - 6 KB
ActivatedDocumentTransformer 10 2.462 μs 0.0369 μs 0.0288 μs 406,209.9 0.0534 0.0114 - 7 KB
DocumentTransformerAsDelegate 10 2.399 μs 0.0463 μs 0.0455 μs 416,841.0 0.0496 0.0114 - 6 KB
ActivatedSchemaTransformer 10 31.617 μs 0.4569 μs 0.4274 μs 31,628.4 0.3662 - - 56 KB
SchemaTransformerAsDelegate 10 30.972 μs 0.6155 μs 0.6320 μs 32,287.4 0.3662 - - 55 KB
ActivatedOperationTransformer 100 4.082 μs 0.0557 μs 0.0494 μs 244,954.8 0.0687 0.0153 - 9 KB
OperationTransformerAsDelegate 100 3.250 μs 0.0595 μs 0.0497 μs 307,655.7 0.0458 0.0076 - 6 KB
ActivatedDocumentTransformer 100 4.149 μs 0.0428 μs 0.0358 μs 241,048.5 0.0687 0.0153 - 9 KB
DocumentTransformerAsDelegate 100 3.241 μs 0.0258 μs 0.0215 μs 308,574.1 0.0496 0.0114 - 6 KB
ActivatedSchemaTransformer 100 38.633 μs 0.2470 μs 0.2190 μs 25,884.8 0.4883 - - 66 KB
SchemaTransformerAsDelegate 100 35.940 μs 0.4064 μs 0.3802 μs 27,824.2 0.4883 - - 62 KB
ActivatedOperationTransformer 1000 34.187 μs 0.4856 μs 0.4543 μs 29,251.0 0.2441 - - 30 KB
OperationTransformerAsDelegate 1000 11.128 μs 0.0788 μs 0.0658 μs 89,863.8 0.0458 0.0153 - 6 KB
ActivatedDocumentTransformer 1000 36.648 μs 0.6589 μs 0.8332 μs 27,286.5 0.2441 - - 30 KB
DocumentTransformerAsDelegate 1000 12.282 μs 0.0807 μs 0.0674 μs 81,422.7 0.0458 0.0153 - 6 KB
ActivatedSchemaTransformer 1000 145.059 μs 1.1951 μs 0.9980 μs 6,893.7 1.2207 0.2441 - 164 KB
SchemaTransformerAsDelegate 1000 72.551 μs 0.4466 μs 0.3959 μs 13,783.4 0.7324 - - 117 KB
@captainsafia captainsafia added Perf area-mvc Includes: MVC, Actions and Controllers, Localization, CORS, most templates feature-openapi area-minimal Includes minimal APIs, endpoint filters, parameter binding, request delegate generator etc labels Jul 16, 2024
@captainsafia captainsafia self-assigned this Aug 13, 2024
@captainsafia
Copy link
Member Author

Parking this in .NET 10 planning since I anticipate any further perf optimizations will ship as part of that release.

One thing I am keen to do is examine whether we are allocating too many OpenApiSchema objects as we construct the model

@captainsafia captainsafia added this to the .NET 10 Planning milestone Aug 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-minimal Includes minimal APIs, endpoint filters, parameter binding, request delegate generator etc area-mvc Includes: MVC, Actions and Controllers, Localization, CORS, most templates feature-openapi Perf
Projects
None yet
Development

No branches or pull requests

1 participant