Skip to content

Scoring support in ES|QL #116599

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tteofili opened this issue Nov 11, 2024 · 2 comments
Open

Scoring support in ES|QL #116599

tteofili opened this issue Nov 11, 2024 · 2 comments
Assignees
Labels
:Analytics/ES|QL AKA ESQL >enhancement Meta Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)

Comments

@tteofili
Copy link
Contributor

tteofili commented Nov 11, 2024

This meta-issue covers the work needed to enable scoring when using running full-text queries in ES|QL.

High Level Vision: score is just a regular column, no magic

Scoring is enabled through a METADATA field, _score. For example,

FROM employees METADATA _score
  | WHERE title:"test"
  | SORT _score
  • Scoring is explicit, opt-in ( via METADATA )
  • By adding METADATA _score a score column is added to the table
  • The _score column is “dynamic”, potentially any command could alter it
  • WHERE .. MATCH will affect the score (in the same as an ES query would)
  • No implicit sorting on score. Sorting can be achieved through existing commands, e.g. a | SORT _score
### Tasks
- [x] enable scoring with METADATA _score: #113120
- [ ] https://github.com/elastic/elasticsearch/issues/120082
- [x] clarify valid _score overrides
- [x] make score collector parameters configurable/inferrable
- [x] clarify scoring behavior with subqueries
- [ ] test for scoring consistency across _search and ES|QL with METADATA _score
- [x] clarify valid scenarios for _score manipulation
- [ ] refactor EsQueryExect.Sort to allow non-FieldAttribute backed Sorts (e.g., _score)
- [ ] https://github.com/elastic/elasticsearch/issues/117641
- [ ] test how scoring works with aggregation commands
- [ ] make it possible to explain scores (EXPLAIN api)
- [ ] clarify behavior when a _score field exists in a targeted index
- [ ] https://github.com/elastic/elasticsearch/issues/118460
- [ ] https://github.com/elastic/elasticsearch/issues/120940
- [ ] https://github.com/elastic/elasticsearch/pull/121793
- [ ] https://github.com/elastic/elasticsearch/pull/124540
@elasticsearchmachine elasticsearchmachine added the needs:triage Requires assignment of a team area label label Nov 11, 2024
@tteofili tteofili self-assigned this Nov 11, 2024
@tteofili tteofili added >enhancement :Search/Search Search-related issues that do not fall into other categories labels Nov 11, 2024
@elasticsearchmachine elasticsearchmachine added the Team:Search Meta label for search team label Nov 11, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@elasticsearchmachine elasticsearchmachine removed the needs:triage Requires assignment of a team area label label Nov 11, 2024
@tteofili tteofili added needs:triage Requires assignment of a team area label :Analytics/ES|QL AKA ESQL and removed :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team labels Nov 11, 2024
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Nov 11, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL >enhancement Meta Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo)
Projects
None yet
Development

No branches or pull requests

3 participants