Remove benchmark package #15356
Remove benchmark package #15356
Conversation
|
+1 but I know this can be controversial so I would suggest to wait a bit before merging |
|
I've pondered the same thing myself many times during some refactoring efforts; I'm in favor of this. LGTM. |
|
I am +1 on this change. What I would ask is that we capture the intent of some of these benchmarks into a meta issue in in our perf testing infra to make sure we end up having coverage for those and testing them in a continuous manner. Things like indexing perf for json doc with "just" string and a number, many aliases tests, ... (/cc @danielmitterdorfer) |
|
Funny, I have also came across the benchmark classes and felt the same way. So +1 too. I'll create a meta issue over in the Rally repo and reference this ticket so we don't forget it. Thanks for bringing it to my attention @kimchy. |
OK let's merge now :) |
Tons of ancient "benchmarks" exist in elasticsearch. These are main methods that do some kind of construction of ES classes and time various things. The problem with these is they are not maintained, and not run. Refactorings that touch anything that is common in these classes is very painful. Going through these, almost all would simply not work in 2.x without modifications (because they do not set path.home). This change removes the entire benchmark package. If someone needs to run a benchmark like this, they can look at history for examples if necessary (although these examples are often not realistic and should just start real elasticsearch processes in a shell script). Longer term, we should make this easier to do by having the build support adding real benchmarks which can be run in jenkins (so we know they actually run, instead of doing refactorings with pure guesswork as to whether the benchmark would run correctly).
rjernst
force-pushed
the
clean_benchmarks
branch
from
9a29660 to
81d6363
Compare
|
@danielmitterdorfer I have removed two more benchmark classes in d14ae5f |
Tons of ancient "benchmarks" exist in elasticsearch. These are main
methods that do some kind of construction of ES classes and time various
things. The problem with these is they are not maintained, and not run.
Refactorings that touch anything that is common in these classes is very
painful. Going through these, almost all would simply not work in 2.x
without modifications (because they do not set path.home).
This change removes the entire benchmark package. If someone needs to
run a benchmark like this, they can look at history for examples if
necessary (although these examples are often not realistic and should
just start real elasticsearch processes in a shell script). Longer term,
we should make this easier to do by having the build support adding real
benchmarks which can be run in jenkins (so we know they actually run,
instead of doing refactorings with pure guesswork as to whether the
benchmark would run correctly).