Add infrastructure support for microbenchmarking #18242
Labels
:Delivery/Build
Build or test infrastructure
>enhancement
Team:Delivery
Meta label for Delivery team
v5.0.0-alpha4
Comments
danielmitterdorfer
added
>enhancement
:Delivery/Build
danielmitterdorfer
added a commit
to danielmitterdorfer/elasticsearch
that referenced
this issue
Jun 15, 2016
With this commit we add a benchmarks project that contains the necessary build infrastructure and an example benchmark. It is added as a separate project to avoid interfering with the regular build too much (especially sanity checks) and to keep the microbenchmarks isolated. Microbenchmarks are generated with `gradle :benchmarks:jmhJar` and can be run with ` gradle :benchmarks:jmh`. We intentionally do not use the [jmh-gradle-plugin](https://github.com/melix/jmh-gradle-plugin) as it causes all sorts of problems (dependencies are not properly excluded, not all JMH parameters can be set) and it adds another abstraction layer that is not needed. Closes elastic#18242
danielmitterdorfer
added a commit
that referenced
this issue
Jun 15, 2016
With this commit we add a benchmarks project that contains the necessary build infrastructure and an example benchmark. It is added as a separate project to avoid interfering with the regular build too much (especially sanity checks) and to keep the microbenchmarks isolated. Microbenchmarks are generated with `gradle :benchmarks:jmhJar` and can be run with ` gradle :benchmarks:jmh`. We intentionally do not use the [jmh-gradle-plugin](https://github.com/melix/jmh-gradle-plugin) as it causes all sorts of problems (dependencies are not properly excluded, not all JMH parameters can be set) and it adds another abstraction layer that is not needed. Closes #18242
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Rationale
We have recently removed a few benchmarks from Elasticsearch core (see #15356). Although most of our benchmarking should be on the macro-level (i.e. running against a real Elasticsearch cluster) we're sometimes interested in benchmarking at a more narrow level. As we don't need a fully running cluster in that case, it makes sense to base them on JMH as it is the de-facto standard for microbenchmarking on the JVM.
Scope
The scope of this ticket is just to add the necessary infrastructure to our build scripts (and maybe provide one or two example benchmarks). We will not recreate the former benchmark suite in this ticket (nor should we aim to recreate the whole suite again).
Next Steps
After this is set up, we should run the benchmarks also regularly on Jenkins (on a dedicated quiet machine for benchmarks) and add proper reporting, e.g. with the JMH Benchmark Jenkins Plugin to spot performance regressions.
The text was updated successfully, but these errors were encountered: