Skip to content

Commit 90cdc65

Browse files
jasontedorjasontedor
committed
Merge remote-tracking branch 'elastic/master' into optimize-warning-head-de-duplication
* elastic/master: (24 commits) [TEST] Mute MlMappingsUpgradeIT testMappingsUpgrade Streamline skip_unavailable handling (elastic#37672) Only bootstrap and elect node in current voting configuration (elastic#37712) Ensure either success or failure path for SearchOperationListener is called (elastic#37467) Target only specific index in update settings test Add a note how to benchmark Elasticsearch Don't use Groovy's `withDefault` (elastic#37726) Adapt SyncedFlushService (elastic#37691) Mute FilterAggregatorTests#testRandom Switch mapping/aggregations over to java time (elastic#36363) [ML] Update ML results mappings on process start (elastic#37706) Modify removal_of_types.asciidoc (elastic#37648) Fix edge case in PutMappingRequestTests (elastic#37665) Use new bulk API endpoint in the docs (elastic#37698) Expose sequence number and primary terms in search responses (elastic#37639) Remove LicenseServiceClusterNotRecoveredTests (elastic#37528) Migrate SpecificMasterNodesIT to Zen2 (elastic#37532) Fix MetaStateFormat tests Use plain text instead of latexmath Fix a typo in a warning message in TestFixturesPlugin (elastic#37631) ...
2 parents 6e1ecec + 6a5d9d9 commit 90cdc65

File tree

284 files changed

+4331
-3453
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

284 files changed

+4331
-3453
lines changed

‎TESTING.asciidoc

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -631,3 +631,13 @@ inside `/etc/hosts`, e.g.:
631631
255.255.255.255 broadcasthost
632632
::1 localhost ElasticMBP.local`
633633
....
634+
635+
== Benchmarking
636+
637+
For changes that might affect the performance characteristics of Elasticsearch
638+
you should also run macrobenchmarks. We maintain a macrobenchmarking tool
639+
called https://github.com/elastic/rally[Rally]
640+
which you can use to measure the performance impact. It comes with a set of
641+
default benchmarks that we also
642+
https://elasticsearch-benchmarks.elastic.co/[run every night]. To get started,
643+
please see https://esrally.readthedocs.io/en/stable/[Rally's documentation].

‎benchmarks/src/main/java/org/elasticsearch/benchmark/time/DateFormatterBenchmark.java

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,4 +55,3 @@ public TemporalAccessor parseJodaDate() {
5555
return jodaFormatter.parse("1234567890");
5656
}
5757
}
58-

‎buildSrc/src/main/groovy/org/elasticsearch/gradle/BuildPlugin.groovy

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -392,7 +392,7 @@ class BuildPlugin implements Plugin<Project> {
392392
static void requireJavaHome(Task task, int version) {
393393
Project rootProject = task.project.rootProject // use root project for global accounting
394394
if (rootProject.hasProperty('requiredJavaVersions') == false) {
395-
rootProject.rootProject.ext.requiredJavaVersions = [:].withDefault{key -> return []}
395+
rootProject.rootProject.ext.requiredJavaVersions = [:]
396396
rootProject.gradle.taskGraph.whenReady { TaskExecutionGraph taskGraph ->
397397
List<String> messages = []
398398
for (entry in rootProject.requiredJavaVersions) {
@@ -415,7 +415,7 @@ class BuildPlugin implements Plugin<Project> {
415415
throw new GradleException("JAVA${version}_HOME required to run task:\n${task}")
416416
}
417417
} else {
418-
rootProject.requiredJavaVersions.get(version).add(task)
418+
rootProject.requiredJavaVersions.getOrDefault(version, []).add(task)
419419
}
420420
}
421421

‎buildSrc/src/main/java/org/elasticsearch/gradle/testfixtures/TestFixturesPlugin.java

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ public void apply(Project project) {
9696
if (dockerComposeSupported(project) == false) {
9797
project.getLogger().warn(
9898
"Tests for {} require docker-compose at /usr/local/bin/docker-compose or /usr/bin/docker-compose " +
99-
"but none could not be found so these will be skipped", project.getPath()
99+
"but none could be found so these will be skipped", project.getPath()
100100
);
101101
tasks.withType(getTaskClass("com.carrotsearch.gradle.junit4.RandomizedTestingTask"), task ->
102102
task.setEnabled(false)

‎client/rest-high-level/src/test/java/org/elasticsearch/client/IndicesClientIT.java

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1177,7 +1177,7 @@ public void testIndexPutSettings() throws IOException {
11771177
createIndex(index, Settings.EMPTY);
11781178

11791179
assertThat(dynamicSetting.getDefault(Settings.EMPTY), not(dynamicSettingValue));
1180-
UpdateSettingsRequest dynamicSettingRequest = new UpdateSettingsRequest();
1180+
UpdateSettingsRequest dynamicSettingRequest = new UpdateSettingsRequest(index);
11811181
dynamicSettingRequest.settings(Settings.builder().put(dynamicSettingKey, dynamicSettingValue).build());
11821182
AcknowledgedResponse response = execute(dynamicSettingRequest, highLevelClient().indices()::putSettings,
11831183
highLevelClient().indices()::putSettingsAsync);
@@ -1187,7 +1187,7 @@ public void testIndexPutSettings() throws IOException {
11871187
assertThat(indexSettingsAsMap.get(dynamicSettingKey), equalTo(String.valueOf(dynamicSettingValue)));
11881188

11891189
assertThat(staticSetting.getDefault(Settings.EMPTY), not(staticSettingValue));
1190-
UpdateSettingsRequest staticSettingRequest = new UpdateSettingsRequest();
1190+
UpdateSettingsRequest staticSettingRequest = new UpdateSettingsRequest(index);
11911191
staticSettingRequest.settings(Settings.builder().put(staticSettingKey, staticSettingValue).build());
11921192
ElasticsearchException exception = expectThrows(ElasticsearchException.class, () -> execute(staticSettingRequest,
11931193
highLevelClient().indices()::putSettings, highLevelClient().indices()::putSettingsAsync));
@@ -1207,7 +1207,7 @@ public void testIndexPutSettings() throws IOException {
12071207
assertThat(indexSettingsAsMap.get(staticSettingKey), equalTo(staticSettingValue));
12081208

12091209
assertThat(unmodifiableSetting.getDefault(Settings.EMPTY), not(unmodifiableSettingValue));
1210-
UpdateSettingsRequest unmodifiableSettingRequest = new UpdateSettingsRequest();
1210+
UpdateSettingsRequest unmodifiableSettingRequest = new UpdateSettingsRequest(index);
12111211
unmodifiableSettingRequest.settings(Settings.builder().put(unmodifiableSettingKey, unmodifiableSettingValue).build());
12121212
exception = expectThrows(ElasticsearchException.class, () -> execute(unmodifiableSettingRequest,
12131213
highLevelClient().indices()::putSettings, highLevelClient().indices()::putSettingsAsync));

‎client/rest-high-level/src/test/java/org/elasticsearch/client/indices/PutMappingRequestTests.java

Lines changed: 15 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -19,15 +19,14 @@
1919

2020
package org.elasticsearch.client.indices;
2121

22-
import org.apache.lucene.util.LuceneTestCase.AwaitsFix;
2322
import org.elasticsearch.common.xcontent.XContentBuilder;
2423
import org.elasticsearch.common.xcontent.XContentParser;
2524
import org.elasticsearch.index.RandomCreateIndexGenerator;
2625
import org.elasticsearch.test.AbstractXContentTestCase;
2726

2827
import java.io.IOException;
28+
import java.util.Map;
2929

30-
@AwaitsFix(bugUrl = "https://github.com/elastic/elasticsearch/issues/37654")
3130
public class PutMappingRequestTests extends AbstractXContentTestCase<PutMappingRequest> {
3231

3332
@Override
@@ -47,7 +46,10 @@ protected PutMappingRequest createTestInstance() {
4746
@Override
4847
protected PutMappingRequest doParseInstance(XContentParser parser) throws IOException {
4948
PutMappingRequest request = new PutMappingRequest();
50-
request.source(parser.map());
49+
Map<String, Object> map = parser.map();
50+
if (map.isEmpty() == false) {
51+
request.source(map);
52+
}
5153
return request;
5254
}
5355

@@ -58,11 +60,16 @@ protected boolean supportsUnknownFields() {
5860

5961
@Override
6062
protected void assertEqualInstances(PutMappingRequest expected, PutMappingRequest actual) {
61-
try (XContentParser expectedJson = createParser(expected.xContentType().xContent(), expected.source());
62-
XContentParser actualJson = createParser(actual.xContentType().xContent(), actual.source())) {
63-
assertEquals(expectedJson.mapOrdered(), actualJson.mapOrdered());
64-
} catch (IOException e) {
65-
throw new RuntimeException(e);
63+
if (actual.source() != null) {
64+
try (XContentParser expectedJson = createParser(expected.xContentType().xContent(), expected.source());
65+
XContentParser actualJson = createParser(actual.xContentType().xContent(), actual.source())) {
66+
assertEquals(expectedJson.mapOrdered(), actualJson.mapOrdered());
67+
} catch (IOException e) {
68+
throw new RuntimeException(e);
69+
}
70+
} else {
71+
// if the original `source` is null, the parsed source should be so too
72+
assertNull(expected.source());
6673
}
6774
}
6875
}

‎docs/reference/aggregations/bucket/adjacency-matrix-aggregation.asciidoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ Example:
3030

3131
[source,js]
3232
--------------------------------------------------
33-
PUT /emails/_doc/_bulk?refresh
33+
PUT /emails/_bulk?refresh
3434
{ "index" : { "_id" : 1 } }
3535
{ "accounts" : ["hillary", "sidney"]}
3636
{ "index" : { "_id" : 2 } }

‎docs/reference/aggregations/bucket/filters-aggregation.asciidoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ Example:
99

1010
[source,js]
1111
--------------------------------------------------
12-
PUT /logs/_doc/_bulk?refresh
12+
PUT /logs/_bulk?refresh
1313
{ "index" : { "_id" : 1 } }
1414
{ "body" : "warning: page could not be rendered" }
1515
{ "index" : { "_id" : 2 } }

‎docs/reference/aggregations/metrics/scripted-metric-aggregation.asciidoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -147,7 +147,7 @@ Imagine a situation where you index the following documents into an index with 2
147147

148148
[source,js]
149149
--------------------------------------------------
150-
PUT /transactions/_doc/_bulk?refresh
150+
PUT /transactions/_bulk?refresh
151151
{"index":{"_id":1}}
152152
{"type": "sale","amount": 80}
153153
{"index":{"_id":2}}

‎docs/reference/aggregations/metrics/tophits-aggregation.asciidoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@ The top_hits aggregation returns regular search hits, because of this many per h
2525
* <<search-request-script-fields,Script fields>>
2626
* <<search-request-docvalue-fields,Doc value fields>>
2727
* <<search-request-version,Include versions>>
28+
* <<search-request-seq-no-primary-term,Include Sequence Numbers and Primary Terms>>
2829

2930
==== Example
3031

‎docs/reference/docs/bulk.asciidoc

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ newlines. Example:
5757
[source,js]
5858
--------------------------------------------------
5959
$ cat requests
60-
{ "index" : { "_index" : "test", "_type" : "_doc", "_id" : "1" } }
60+
{ "index" : { "_index" : "test", "_id" : "1" } }
6161
{ "field1" : "value1" }
6262
$ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
6363
{"took":7, "errors": false, "items":[{"index":{"_index":"test","_type":"_doc","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}
@@ -72,12 +72,12 @@ example of a correct sequence of bulk commands:
7272
[source,js]
7373
--------------------------------------------------
7474
POST _bulk
75-
{ "index" : { "_index" : "test", "_type" : "_doc", "_id" : "1" } }
75+
{ "index" : { "_index" : "test", "_id" : "1" } }
7676
{ "field1" : "value1" }
77-
{ "delete" : { "_index" : "test", "_type" : "_doc", "_id" : "2" } }
78-
{ "create" : { "_index" : "test", "_type" : "_doc", "_id" : "3" } }
77+
{ "delete" : { "_index" : "test", "_id" : "2" } }
78+
{ "create" : { "_index" : "test", "_id" : "3" } }
7979
{ "field1" : "value3" }
80-
{ "update" : {"_id" : "1", "_type" : "_doc", "_index" : "test"} }
80+
{ "update" : {"_id" : "1", "_index" : "test"} }
8181
{ "doc" : {"field2" : "value2"} }
8282
--------------------------------------------------
8383
// CONSOLE
@@ -265,15 +265,15 @@ the options. Example with update actions:
265265
[source,js]
266266
--------------------------------------------------
267267
POST _bulk
268-
{ "update" : {"_id" : "1", "_type" : "_doc", "_index" : "index1", "retry_on_conflict" : 3} }
268+
{ "update" : {"_id" : "1", "_index" : "index1", "retry_on_conflict" : 3} }
269269
{ "doc" : {"field" : "value"} }
270-
{ "update" : { "_id" : "0", "_type" : "_doc", "_index" : "index1", "retry_on_conflict" : 3} }
270+
{ "update" : { "_id" : "0", "_index" : "index1", "retry_on_conflict" : 3} }
271271
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
272-
{ "update" : {"_id" : "2", "_type" : "_doc", "_index" : "index1", "retry_on_conflict" : 3} }
272+
{ "update" : {"_id" : "2", "_index" : "index1", "retry_on_conflict" : 3} }
273273
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
274-
{ "update" : {"_id" : "3", "_type" : "_doc", "_index" : "index1", "_source" : true} }
274+
{ "update" : {"_id" : "3", "_index" : "index1", "_source" : true} }
275275
{ "doc" : {"field" : "value"} }
276-
{ "update" : {"_id" : "4", "_type" : "_doc", "_index" : "index1"} }
276+
{ "update" : {"_id" : "4", "_index" : "index1"} }
277277
{ "doc" : {"field" : "value"}, "_source": true}
278278
--------------------------------------------------
279279
// CONSOLE

‎docs/reference/docs/concurrency-control.asciidoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,7 @@ returns:
8787

8888

8989
Note: The <<search-search,Search API>> can return the `_seq_no` and `_primary_term`
90-
for each search hit by requesting the `_seq_no` and `_primary_term` <<search-request-docvalue-fields,Doc Value Fields>>.
90+
for each search hit by setting <<search-request-seq-no-primary-term,`seq_no_primary_term` parameter>>.
9191

9292
The sequence number and the primary term uniquely identify a change. By noting down
9393
the sequence number and primary term returned, you can make sure to only change the

‎docs/reference/getting-started.asciidoc

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -633,7 +633,7 @@ As a quick example, the following call indexes two documents (ID 1 - John Doe an
633633

634634
[source,js]
635635
--------------------------------------------------
636-
POST /customer/_doc/_bulk?pretty
636+
POST /customer/_bulk?pretty
637637
{"index":{"_id":"1"}}
638638
{"name": "John Doe" }
639639
{"index":{"_id":"2"}}
@@ -645,7 +645,7 @@ This example updates the first document (ID of 1) and then deletes the second do
645645

646646
[source,sh]
647647
--------------------------------------------------
648-
POST /customer/_doc/_bulk?pretty
648+
POST /customer/_bulk?pretty
649649
{"update":{"_id":"1"}}
650650
{"doc": { "name": "John Doe becomes Jane Doe" } }
651651
{"delete":{"_id":"2"}}
@@ -692,7 +692,7 @@ You can download the sample dataset (accounts.json) from https://github.com/elas
692692

693693
[source,sh]
694694
--------------------------------------------------
695-
curl -H "Content-Type: application/json" -XPOST "localhost:9200/bank/_doc/_bulk?pretty&refresh" --data-binary "@accounts.json"
695+
curl -H "Content-Type: application/json" -XPOST "localhost:9200/bank/_bulk?pretty&refresh" --data-binary "@accounts.json"
696696
curl "localhost:9200/_cat/indices?v"
697697
--------------------------------------------------
698698
// NOTCONSOLE

‎docs/reference/mapping/removal_of_types.asciidoc

Lines changed: 17 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -112,7 +112,7 @@ have looked something like this:
112112

113113
[source,js]
114114
----
115-
PUT twitter?include_type_name=true
115+
PUT twitter
116116
{
117117
"mappings": {
118118
"user": {
@@ -157,16 +157,16 @@ GET twitter/tweet/_search
157157
----
158158
// NOTCONSOLE
159159

160-
You could achieve the same thing by adding a custom `type` field as follows:
160+
You can achieve the same thing by adding a custom `type` field as follows:
161161

162162
[source,js]
163163
----
164-
PUT twitter?include_type_name=true
164+
PUT twitter?include_type_name=true <1>
165165
{
166166
"mappings": {
167167
"_doc": {
168168
"properties": {
169-
"type": { "type": "keyword" }, <1>
169+
"type": { "type": "keyword" }, <2>
170170
"name": { "type": "text" },
171171
"user_name": { "type": "keyword" },
172172
"email": { "type": "keyword" },
@@ -204,15 +204,17 @@ GET twitter/_search
204204
},
205205
"filter": {
206206
"match": {
207-
"type": "tweet" <1>
207+
"type": "tweet" <2>
208208
}
209209
}
210210
}
211211
}
212212
}
213213
----
214214
// NOTCONSOLE
215-
<1> The explicit `type` field takes the place of the implicit `_type` field.
215+
<1> Use `include_type_name=true` in case need to use the "old" syntax including the "_doc" object like
216+
in this example
217+
<2> The explicit `type` field takes the place of the implicit `_type` field.
216218

217219
[float]
218220
==== Parent/Child without mapping types
@@ -299,7 +301,7 @@ This first example splits our `twitter` index into a `tweets` index and a
299301

300302
[source,js]
301303
----
302-
PUT users?include_type_name=true
304+
PUT users
303305
{
304306
"settings": {
305307
"index.mapping.single_type": true
@@ -321,7 +323,7 @@ PUT users?include_type_name=true
321323
}
322324
}
323325
324-
PUT tweets?include_type_name=true
326+
PUT tweets
325327
{
326328
"settings": {
327329
"index.mapping.single_type": true
@@ -376,7 +378,7 @@ documents of different types which have conflicting IDs:
376378

377379
[source,js]
378380
----
379-
PUT new_twitter?include_type_name=true
381+
PUT new_twitter
380382
{
381383
"mappings": {
382384
"_doc": {
@@ -427,10 +429,12 @@ POST _reindex
427429
[float]
428430
=== Use `include_type_name=false` to prepare for upgrade to 8.0
429431

430-
Index creation, mappings and document APIs support the `include_type_name`
431-
option. When set to `false`, this option enables the behavior that will become
432-
default in 8.0 when types are removed. See some examples of interactions with
433-
Elasticsearch with this option turned off:
432+
Index creation and mapping APIs support a new `include_type_name` url parameter
433+
starting with version 6.7. It will default to `true` in version 6.7, default to
434+
`false` in version 7.0 and will be removed in version 8.0. When set to `true`,
435+
this parameter enables the pre-7.0 behavior of using type names in the API.
436+
437+
See some examples of interactions with Elasticsearch with this option turned off:
434438

435439
[float]
436440
==== Index creation

‎docs/reference/query-dsl/script-score-query.asciidoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ rewriting equivalent functions of your own, as these functions try
5353
to be the most efficient by using the internal mechanisms.
5454

5555
===== rational
56-
latexmath:[rational(value,k) = value/(k + value)]
56+
`rational(value,k) = value/(k + value)`
5757

5858
[source,js]
5959
--------------------------------------------------
@@ -64,7 +64,7 @@ latexmath:[rational(value,k) = value/(k + value)]
6464
// NOTCONSOLE
6565

6666
===== sigmoid
67-
latexmath:[sigmoid(value, k, a) = value^a/ (k^a + value^a)]
67+
`sigmoid(value, k, a) = value^a/ (k^a + value^a)`
6868

6969
[source,js]
7070
--------------------------------------------------

‎docs/reference/search/request-body.asciidoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -213,7 +213,7 @@ include::request/preference.asciidoc[]
213213

214214
include::request/explain.asciidoc[]
215215

216-
include::request/version.asciidoc[]
216+
include::request/version-and-seq-no.asciidoc[]
217217

218218
include::request/index-boost.asciidoc[]
219219

‎docs/reference/search/request/inner-hits.asciidoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@ Inner hits also supports the following per document features:
7676
* <<search-request-script-fields,Script fields>>
7777
* <<search-request-docvalue-fields,Doc value fields>>
7878
* <<search-request-version,Include versions>>
79+
* <<search-request-seq-no-primary-term,Include Sequence Numbers and Primary Terms>>
7980

8081
[[nested-inner-hits]]
8182
==== Nested inner hits

0 commit comments

Comments
 (0)