Allow reloading of search time analyzers #42888
Conversation
Currently changing resources (like dictionaries, synonym files etc...) of search time analyzers is only possible by closing an index, changing the underlying resource (e.g. synonym files) and then re-opening the index for the change to take effect. This PR adds a new API endpoint that allows triggering reloading of certain analysis resources (currently token filters) that will then pick up changes in underlying file resources. To achieve this we introduce a new type of custom analyzer (ReloadableCustomAnalyzer) that uses a ReuseStrategy that allows swapping out analysis components. Custom analyzers that contain filters that are markes as "updateable" will automatically choose this implementation. This PR also adds this capability to `synonym` token filters for use in search time analyzers. Relates to elastic#29051
cbuescher
added
>enhancement
:Search Relevance/Analysis
|
Pinging @elastic/es-search |
server/src/main/java/org/elasticsearch/index/analysis/ReloadableCustomAnalyzer.java
Show resolved
Hide resolved
...es/analysis-common/src/test/java/org/elasticsearch/analysis/common/SynonymAnalyzerTests.java
Show resolved
Hide resolved
|
@jimczi @romseygeek I added a commit to make both CustomAnalyzer and ReloadableCustomanalyzer implement a common interface that allows getting the AnalyzerComponents in a uniform way. I also added a unit test for ReloadableCustomanalyzer that tests that changes made via reloading become visible to several concurrently running threads. Let me know what you think about those changes and if they address your previous comments. |
+1, the change looks good to me. Any idea why the tests are failing ? |
|
@elasticmachine run elasticsearch-ci/1 |
|
@jimczi tests are green now, I had to undo a change I did in IndexAnalyzers because it interfered with the (somewhat weird) way we use temporary |
There was a problem hiding this comment.
I left some minor comments but the change looks good to me. I am also curious to know what @jpountz and @romseygeek think.
server/src/main/java/org/elasticsearch/index/analysis/ReloadableCustomAnalyzer.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/action/admin/indices/analyze/TransportAnalyzeAction.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/index/analysis/IndexAnalyzers.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/action/admin/indices/analyze/TransportAnalyzeAction.java
Outdated
Show resolved
Hide resolved
|
I'm a bit under water with various stuff, please don't wait for me to merge it. |
|
@jpountz sure I'll ask for your review before finally merging the feature branch to master. |
Currently changing resources (like dictionaries, synonym files etc...) of search
time analyzers is only possible by closing an index, changing the underlying
resource (e.g. synonym files) and then re-opening the index for the change to
take effect.
This PR adds a new API endpoint that allows triggering reloading of certain
analysis resources (currently token filters) that will then pick up changes in
underlying file resources. To achieve this we introduce a new type of custom
analyzer (ReloadableCustomAnalyzer) that uses a ReuseStrategy that allows
swapping out analysis components. Custom analyzers that contain filters that are
markes as "updateable" will automatically choose this implementation. This PR
also adds this capability to synonym token filters for use in search time
analyzers.
Relates to #29051