Skip to content

Allow rescorer with field collapsing #107779

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 12 commits into from
Apr 29, 2024
Merged

Allow rescorer with field collapsing #107779

merged 12 commits into from
Apr 29, 2024

Conversation

jimczi
Copy link
Contributor

@jimczi jimczi commented Apr 23, 2024

This change adds the support for rescoring collapsed documents.
The rescoring is applied on the top document per group on each shard.

Closes #27243

jimczi added 2 commits April 23, 2024 16:27
@jimczi
Allow rescorer with field collapsing
66f4754 
This change adds the support for rescoring collapsed documents.
The rescoring is applied on the top document per group on each shard.

Closes elastic#27243
@jimczi
fix compileTest
4fa5392
@jimczi jimczi added >feature :Search/Search Search-related issues that do not fall into other categories v8.15.0 labels Apr 23, 2024
@github-actions GitHub Actions
Copy link
Contributor

Documentation preview:

@elasticsearchmachine elasticsearchmachine added the Team:Search Meta label for search team label Apr 23, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@jimczi jimczi requested a review from a team as a code owner April 23, 2024 16:46
benwtrent
benwtrent reviewed Apr 23, 2024
View reviewed changes
Copy link
Member

@benwtrent benwtrent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very exciting!

benwtrent
benwtrent approved these changes Apr 24, 2024
View reviewed changes
Copy link
Member

@benwtrent benwtrent left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic.

Do we think we could also add rescore support for inner hits in the future?

I realize inner-hits are optional for collapse, so we can discuss this later. But it would be good to provide rescoring for inner hits so that their scores & order reflect their collapsed fields. This may require an API change for the inner-hits configuration to accept a "rescore" parameter or something. I honestly don't know.

@jimczi
Copy link
Contributor Author

jimczi commented Apr 24, 2024

Do we think we could also add rescore support for inner hits in the future

That's a good question. Adding the rescoring for inner_hits should be trivial since we can copy the rescorer into the expand query. I wanted to add that initially but then realise that users may want to apply a different window_size when rescoring inside each group. The original window_size might be too big to apply on every group since that would be applied on each top docs request per group. I left it for a follow up and added a warning in the doc for now.

kderusso
kderusso approved these changes Apr 24, 2024
View reviewed changes
Copy link
Member

@kderusso kderusso left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - thanks for taking care of this!

@jimczi jimczi merged commit 4380cd1 into elastic:main Apr 29, 2024
14 checks passed
@jimczi jimczi deleted the collapse_with_rescore branch April 29, 2024 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@mark-vieira mark-vieira mark-vieira approved these changes

@benwtrent benwtrent benwtrent approved these changes

@kderusso kderusso kderusso approved these changes

Assignees
No one assigned
Labels
>feature :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team v8.15.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

support field collapsing + rescore
5 participants
@jimczi @elasticsearchmachine @mark-vieira @benwtrent @kderusso