Skip to content

Commit 0a90962

Browse files
dnhatndnhatn
committed
Merge branch '6.x' into ccr-6.x
* 6.x: Watcher: Make settings reloadable (#31746) [Rollup] Histo group config should support scaled_floats (#32048) lazy snapshot repository initialization (#31606) Add secure setting for watcher email password (#31620) Watcher: cleanup ensureWatchExists use (#31926) Add second level of field collapsing (#31808) Added lenient flag for synonym token filter (#31484) (#31970) Test: Fix a second case of bad watch creation [Rollup] Use composite's missing_bucket (#31402) Docs: Restyled cloud link in getting started Docs: Change formatting of Cloud options Re-instate link in StringFunctionUtils javadocs Correct spelling of AnalysisPlugin#requriesAnalysisSettings (#32025) Fix problematic chars in javadoc [ML] Move open job failure explanation out of root cause (#31925) [ML] Switch ML native QA tests to use a 3 node cluster (#32011)
2 parents 7fde7a0 + 6e1b187 commit 0a90962

File tree

97 files changed

+1879
-699
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

97 files changed

+1879
-699
lines changed

‎docs/reference/analysis/tokenfilters/synonym-graph-tokenfilter.asciidoc

+43-1
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,49 @@ PUT /test_index
5050
The above configures a `search_synonyms` filter, with a path of
5151
`analysis/synonym.txt` (relative to the `config` location). The
5252
`search_synonyms` analyzer is then configured with the filter.
53-
Additional settings are: `expand` (defaults to `true`).
53+
54+
Additional settings are:
55+
56+
* `expand` (defaults to `true`).
57+
* `lenient` (defaults to `false`). If `true` ignores exceptions while parsing the synonym configuration. It is important
58+
to note that only those synonym rules which cannot get parsed are ignored. For instance consider the following request:
59+
60+
[source,js]
61+
--------------------------------------------------
62+
PUT /test_index
63+
{
64+
"settings": {
65+
"index" : {
66+
"analysis" : {
67+
"analyzer" : {
68+
"synonym" : {
69+
"tokenizer" : "standard",
70+
"filter" : ["my_stop", "synonym_graph"]
71+
}
72+
},
73+
"filter" : {
74+
"my_stop": {
75+
"type" : "stop",
76+
"stopwords": ["bar"]
77+
},
78+
"synonym_graph" : {
79+
"type" : "synonym_graph",
80+
"lenient": true,
81+
"synonyms" : ["foo, bar => baz"]
82+
}
83+
}
84+
}
85+
}
86+
}
87+
}
88+
--------------------------------------------------
89+
// CONSOLE
90+
With the above request the word `bar` gets skipped but a mapping `foo => baz` is still added. However, if the mapping
91+
being added was "foo, baz => bar" nothing would get added to the synonym list. This is because the target word for the
92+
mapping is itself eliminated because it was a stop word. Similarly, if the mapping was "bar, foo, baz" and `expand` was
93+
set to `false` no mapping would get added as when `expand=false` the target mapping is the first word. However, if
94+
`expand=true` then the mappings added would be equivalent to `foo, baz => foo, baz` i.e, all mappings other than the
95+
stop word.
5496

5597
[float]
5698
==== `tokenizer` and `ignore_case` are deprecated

‎docs/reference/analysis/tokenfilters/synonym-tokenfilter.asciidoc

+45-2
Original file line numberDiff line numberDiff line change
@@ -33,12 +33,55 @@ PUT /test_index
3333

3434
The above configures a `synonym` filter, with a path of
3535
`analysis/synonym.txt` (relative to the `config` location). The
36-
`synonym` analyzer is then configured with the filter. Additional
37-
settings is: `expand` (defaults to `true`).
36+
`synonym` analyzer is then configured with the filter.
3837

3938
This filter tokenize synonyms with whatever tokenizer and token filters
4039
appear before it in the chain.
4140

41+
Additional settings are:
42+
43+
* `expand` (defaults to `true`).
44+
* `lenient` (defaults to `false`). If `true` ignores exceptions while parsing the synonym configuration. It is important
45+
to note that only those synonym rules which cannot get parsed are ignored. For instance consider the following request:
46+
47+
[source,js]
48+
--------------------------------------------------
49+
PUT /test_index
50+
{
51+
"settings": {
52+
"index" : {
53+
"analysis" : {
54+
"analyzer" : {
55+
"synonym" : {
56+
"tokenizer" : "standard",
57+
"filter" : ["my_stop", "synonym"]
58+
}
59+
},
60+
"filter" : {
61+
"my_stop": {
62+
"type" : "stop",
63+
"stopwords": ["bar"]
64+
},
65+
"synonym" : {
66+
"type" : "synonym",
67+
"lenient": true,
68+
"synonyms" : ["foo, bar => baz"]
69+
}
70+
}
71+
}
72+
}
73+
}
74+
}
75+
--------------------------------------------------
76+
// CONSOLE
77+
With the above request the word `bar` gets skipped but a mapping `foo => baz` is still added. However, if the mapping
78+
being added was "foo, baz => bar" nothing would get added to the synonym list. This is because the target word for the
79+
mapping is itself eliminated because it was a stop word. Similarly, if the mapping was "bar, foo, baz" and `expand` was
80+
set to `false` no mapping would get added as when `expand=false` the target mapping is the first word. However, if
81+
`expand=true` then the mappings added would be equivalent to `foo, baz => foo, baz` i.e, all mappings other than the
82+
stop word.
83+
84+
4285
[float]
4386
==== `tokenizer` and `ignore_case` are deprecated
4487

‎docs/reference/getting-started.asciidoc

+3
Original file line numberDiff line numberDiff line change
@@ -104,10 +104,13 @@ With that out of the way, let's get started with the fun part...
104104

105105
== Installation
106106

107+
[TIP]
108+
==============
107109
You can skip installation completely by using our hosted
108110
Elasticsearch Service on https://www.elastic.co/cloud[Elastic Cloud], which is
109111
available on AWS and GCP. You can
110112
https://www.elastic.co/cloud/elasticsearch-service/signup[try out the hosted service] for free.
113+
==============
111114

112115
Elasticsearch requires at least Java 8. Specifically as of this writing, it is recommended that you use the Oracle JDK version {jdk}. Java installation varies from platform to platform so we won't go into those details here. Oracle's recommended installation documentation can be found on http://docs.oracle.com/javase/8/docs/technotes/guides/install/install_overview.html[Oracle's website]. Suffice to say, before you install Elasticsearch, please check your Java version first by running (and then install/upgrade accordingly if needed):
113116

‎docs/reference/search/request/collapse.asciidoc

+102
Original file line numberDiff line numberDiff line change
@@ -116,3 +116,105 @@ The default is based on the number of data nodes and the default search thread p
116116

117117
WARNING: `collapse` cannot be used in conjunction with <<search-request-scroll, scroll>>,
118118
<<search-request-rescore, rescore>> or <<search-request-search-after, search after>>.
119+
120+
==== Second level of collapsing
121+
122+
Second level of collapsing is also supported and is applied to `inner_hits`.
123+
For example, the following request finds the top scored tweets for
124+
each country, and within each country finds the top scored tweets
125+
for each user.
126+
127+
[source,js]
128+
--------------------------------------------------
129+
GET /twitter/_search
130+
{
131+
"query": {
132+
"match": {
133+
"message": "elasticsearch"
134+
}
135+
},
136+
"collapse" : {
137+
"field" : "country",
138+
"inner_hits" : {
139+
"name": "by_location",
140+
"collapse" : {"field" : "user"},
141+
"size": 3
142+
}
143+
}
144+
}
145+
--------------------------------------------------
146+
// NOTCONSOLE
147+
148+
149+
Response:
150+
[source,js]
151+
--------------------------------------------------
152+
{
153+
...
154+
"hits": [
155+
{
156+
"_index": "twitter",
157+
"_type": "_doc",
158+
"_id": "9",
159+
"_score": ...,
160+
"_source": {...},
161+
"fields": {"country": ["UK"]},
162+
"inner_hits":{
163+
"by_location": {
164+
"hits": {
165+
...,
166+
"hits": [
167+
{
168+
...
169+
"fields": {"user" : ["user124"]}
170+
},
171+
{
172+
...
173+
"fields": {"user" : ["user589"]}
174+
},
175+
{
176+
...
177+
"fields": {"user" : ["user001"]}
178+
}
179+
]
180+
}
181+
}
182+
}
183+
},
184+
{
185+
"_index": "twitter",
186+
"_type": "_doc",
187+
"_id": "1",
188+
"_score": ..,
189+
"_source": {...},
190+
"fields": {"country": ["Canada"]},
191+
"inner_hits":{
192+
"by_location": {
193+
"hits": {
194+
...,
195+
"hits": [
196+
{
197+
...
198+
"fields": {"user" : ["user444"]}
199+
},
200+
{
201+
...
202+
"fields": {"user" : ["user1111"]}
203+
},
204+
{
205+
...
206+
"fields": {"user" : ["user999"]}
207+
}
208+
]
209+
}
210+
}
211+
}
212+
213+
},
214+
....
215+
]
216+
}
217+
--------------------------------------------------
218+
// NOTCONSOLE
219+
220+
NOTE: Second level of of collapsing doesn't allow `inner_hits`.

‎docs/reference/setup/install.asciidoc

+5
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,16 @@
11
[[install-elasticsearch]]
22
== Installing Elasticsearch
33

4+
[float]
5+
=== Hosted Elasticsearch
46
Elasticsearch can be run on your own hardware or using our hosted
57
Elasticsearch Service on https://www.elastic.co/cloud[Elastic Cloud], which is
68
available on AWS and GCP. You can
79
https://www.elastic.co/cloud/elasticsearch-service/signup[try out the hosted service] for free.
810

11+
[float]
12+
=== Installing Elasticsearch Yourself
13+
914
Elasticsearch is provided in the following package formats:
1015

1116
[horizontal]

‎modules/analysis-common/src/main/java/org/elasticsearch/analysis/common/CommonAnalysisPlugin.java

+13-13
Original file line numberDiff line numberDiff line change
@@ -135,7 +135,7 @@
135135
import java.util.Map;
136136
import java.util.TreeMap;
137137

138-
import static org.elasticsearch.plugins.AnalysisPlugin.requriesAnalysisSettings;
138+
import static org.elasticsearch.plugins.AnalysisPlugin.requiresAnalysisSettings;
139139

140140
public class CommonAnalysisPlugin extends Plugin implements AnalysisPlugin {
141141

@@ -201,11 +201,11 @@ public Map<String, AnalysisProvider<TokenFilterFactory>> getTokenFilters() {
201201
filters.put("cjk_width", CJKWidthFilterFactory::new);
202202
filters.put("classic", ClassicFilterFactory::new);
203203
filters.put("czech_stem", CzechStemTokenFilterFactory::new);
204-
filters.put("common_grams", requriesAnalysisSettings(CommonGramsTokenFilterFactory::new));
204+
filters.put("common_grams", requiresAnalysisSettings(CommonGramsTokenFilterFactory::new));
205205
filters.put("decimal_digit", DecimalDigitFilterFactory::new);
206206
filters.put("delimited_payload_filter", LegacyDelimitedPayloadTokenFilterFactory::new);
207207
filters.put("delimited_payload", DelimitedPayloadTokenFilterFactory::new);
208-
filters.put("dictionary_decompounder", requriesAnalysisSettings(DictionaryCompoundWordTokenFilterFactory::new));
208+
filters.put("dictionary_decompounder", requiresAnalysisSettings(DictionaryCompoundWordTokenFilterFactory::new));
209209
filters.put("dutch_stem", DutchStemTokenFilterFactory::new);
210210
filters.put("edge_ngram", EdgeNGramTokenFilterFactory::new);
211211
filters.put("edgeNGram", EdgeNGramTokenFilterFactory::new);
@@ -216,11 +216,11 @@ public Map<String, AnalysisProvider<TokenFilterFactory>> getTokenFilters() {
216216
filters.put("german_normalization", GermanNormalizationFilterFactory::new);
217217
filters.put("german_stem", GermanStemTokenFilterFactory::new);
218218
filters.put("hindi_normalization", HindiNormalizationFilterFactory::new);
219-
filters.put("hyphenation_decompounder", requriesAnalysisSettings(HyphenationCompoundWordTokenFilterFactory::new));
219+
filters.put("hyphenation_decompounder", requiresAnalysisSettings(HyphenationCompoundWordTokenFilterFactory::new));
220220
filters.put("indic_normalization", IndicNormalizationFilterFactory::new);
221-
filters.put("keep", requriesAnalysisSettings(KeepWordFilterFactory::new));
222-
filters.put("keep_types", requriesAnalysisSettings(KeepTypesFilterFactory::new));
223-
filters.put("keyword_marker", requriesAnalysisSettings(KeywordMarkerTokenFilterFactory::new));
221+
filters.put("keep", requiresAnalysisSettings(KeepWordFilterFactory::new));
222+
filters.put("keep_types", requiresAnalysisSettings(KeepTypesFilterFactory::new));
223+
filters.put("keyword_marker", requiresAnalysisSettings(KeywordMarkerTokenFilterFactory::new));
224224
filters.put("kstem", KStemTokenFilterFactory::new);
225225
filters.put("length", LengthTokenFilterFactory::new);
226226
filters.put("limit", LimitTokenCountFilterFactory::new);
@@ -229,8 +229,8 @@ public Map<String, AnalysisProvider<TokenFilterFactory>> getTokenFilters() {
229229
filters.put("multiplexer", MultiplexerTokenFilterFactory::new);
230230
filters.put("ngram", NGramTokenFilterFactory::new);
231231
filters.put("nGram", NGramTokenFilterFactory::new);
232-
filters.put("pattern_capture", requriesAnalysisSettings(PatternCaptureGroupTokenFilterFactory::new));
233-
filters.put("pattern_replace", requriesAnalysisSettings(PatternReplaceTokenFilterFactory::new));
232+
filters.put("pattern_capture", requiresAnalysisSettings(PatternCaptureGroupTokenFilterFactory::new));
233+
filters.put("pattern_replace", requiresAnalysisSettings(PatternReplaceTokenFilterFactory::new));
234234
filters.put("persian_normalization", PersianNormalizationFilterFactory::new);
235235
filters.put("porter_stem", PorterStemTokenFilterFactory::new);
236236
filters.put("remove_duplicates", RemoveDuplicatesTokenFilterFactory::new);
@@ -241,10 +241,10 @@ public Map<String, AnalysisProvider<TokenFilterFactory>> getTokenFilters() {
241241
filters.put("serbian_normalization", SerbianNormalizationFilterFactory::new);
242242
filters.put("snowball", SnowballTokenFilterFactory::new);
243243
filters.put("sorani_normalization", SoraniNormalizationFilterFactory::new);
244-
filters.put("stemmer_override", requriesAnalysisSettings(StemmerOverrideTokenFilterFactory::new));
244+
filters.put("stemmer_override", requiresAnalysisSettings(StemmerOverrideTokenFilterFactory::new));
245245
filters.put("stemmer", StemmerTokenFilterFactory::new);
246246
filters.put("trim", TrimTokenFilterFactory::new);
247-
filters.put("truncate", requriesAnalysisSettings(TruncateTokenFilterFactory::new));
247+
filters.put("truncate", requiresAnalysisSettings(TruncateTokenFilterFactory::new));
248248
filters.put("unique", UniqueTokenFilterFactory::new);
249249
filters.put("uppercase", UpperCaseTokenFilterFactory::new);
250250
filters.put("word_delimiter_graph", WordDelimiterGraphTokenFilterFactory::new);
@@ -256,8 +256,8 @@ public Map<String, AnalysisProvider<TokenFilterFactory>> getTokenFilters() {
256256
public Map<String, AnalysisProvider<CharFilterFactory>> getCharFilters() {
257257
Map<String, AnalysisProvider<CharFilterFactory>> filters = new TreeMap<>();
258258
filters.put("html_strip", HtmlStripCharFilterFactory::new);
259-
filters.put("pattern_replace", requriesAnalysisSettings(PatternReplaceCharFilterFactory::new));
260-
filters.put("mapping", requriesAnalysisSettings(MappingCharFilterFactory::new));
259+
filters.put("pattern_replace", requiresAnalysisSettings(PatternReplaceCharFilterFactory::new));
260+
filters.put("mapping", requiresAnalysisSettings(MappingCharFilterFactory::new));
261261
return filters;
262262
}
263263

‎modules/rank-eval/src/test/java/org/elasticsearch/index/rankeval/ExpectedReciprocalRankTests.java

+2-2
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ public void testProbabilityOfRelevance() {
6969
* 4 | 1 | 0.03125 | 0.078125 | 0.00244140625 |
7070
* }</pre>
7171
*
72-
* err => sum of last column
72+
* err = sum of last column
7373
*/
7474
public void testERRAt() {
7575
List<RatedDocument> rated = new ArrayList<>();
@@ -94,7 +94,7 @@ public void testERRAt() {
9494
* 4 | 1 | 0.03125 | 0.125 | 0.00390625 |
9595
* }</pre>
9696
*
97-
* err => sum of last column
97+
* err = sum of last column
9898
*/
9999
public void testERRMissingRatings() {
100100
List<RatedDocument> rated = new ArrayList<>();

0 commit comments

Comments
 (0)