-
Notifications
You must be signed in to change notification settings - Fork 25.2k
[ML] Implement AucRoc metric for classification #60502
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cd6d20a
to
2b1f9f3
Compare
3f5f0c9
to
9c70df7
Compare
95c85aa
to
1dc8a0d
Compare
f2c19b3
to
a366fbb
Compare
…fail when the nested field is absent; Add validation that requires that either all the nested fields are specified or none of them are;
…ssification.AucRoc
…edXContents after merge
fe85e5c
to
f99bad1
Compare
run elasticsearch-ci/packaging-sample-windows |
I'll discuss the necessary changes and push them in a separate PR as this one has grown big already. |
przemekwitek
added a commit
to przemekwitek/elasticsearch
that referenced
this pull request
Sep 30, 2020
przemekwitek
added a commit
that referenced
this pull request
Sep 30, 2020
61 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR implements AucRoc metric for evaluating classification.
It achieves it the following way:
top_classes_field
) to theClassification
evaluation objectAucRoc
) underclassification
packageEvaluationMetric::getRequiredFields
) by which each metric may request fields it needs for its calculationIt also introduces a few refactorings:
EvaluationFields
object (reused by all the evaluation types)EvaluationMetric::aggs
now accepts the newEvaluationFields
object instead of the individual field namesclassification.AucRoc
andoutlierdetection.AucRoc
classes using a base class (AbstractAucRoc
)The newly added fields are never indexed (persisted) so they don't require mappings.
Relates #62160