Skip to content

Commit 4d36917

Browse files
tlrxDaveCTurnerywelschdakroneandreidan
authored
Merge feature/searchable-snapshots branch into 7.x (#54803) (#54825)
This is a backport of #54803 for 7.x. This pull request cherry picks the squashed commit from #54803 with the additional commits: 6f50c92 which adjusts master code to 7.x a114549 to mute a failing ILM test (#54818) 48cbca1 and 50186b2 that cleans up and fixes the previous test aae12bb that adds a missing feature flag (#54861) 6f330e3 that adds missing serialization bits (#54864) bf72c02 that adjust the version in YAML tests a51955f that adds some plumbing for the transport client used in integration tests Co-authored-by: David Turner <[email protected]> Co-authored-by: Yannick Welsch <[email protected]> Co-authored-by: Lee Hinman <[email protected]> Co-authored-by: Andrei Dan <[email protected]>
1 parent 8f2ddae commit 4d36917

File tree

163 files changed

+14154
-496
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

163 files changed

+14154
-496
lines changed

‎benchmarks/src/main/java/org/elasticsearch/benchmark/routing/allocation/Allocators.java

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -47,17 +47,21 @@ private static class NoopGatewayAllocator extends GatewayAllocator {
4747
public static final NoopGatewayAllocator INSTANCE = new NoopGatewayAllocator();
4848

4949
@Override
50-
public void applyStartedShards(RoutingAllocation allocation, List<ShardRouting> startedShards) {
50+
public void applyStartedShards(List<ShardRouting> startedShards, RoutingAllocation allocation) {
5151
// noop
5252
}
5353

5454
@Override
55-
public void applyFailedShards(RoutingAllocation allocation, List<FailedShard> failedShards) {
55+
public void applyFailedShards(List<FailedShard> failedShards, RoutingAllocation allocation) {
5656
// noop
5757
}
5858

5959
@Override
60-
public void allocateUnassigned(RoutingAllocation allocation) {
60+
public void allocateUnassigned(
61+
ShardRouting shardRouting,
62+
RoutingAllocation allocation,
63+
UnassignedAllocationHandler unassignedAllocationHandler
64+
) {
6165
// noop
6266
}
6367
}

‎docs/build.gradle

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,7 @@ testClusters.integTest {
5454
if (BuildParams.isSnapshotBuild() == false) {
5555
systemProperty 'es.autoscaling_feature_flag_registered', 'true'
5656
systemProperty 'es.eql_feature_flag_registered', 'true'
57+
systemProperty 'es.searchable_snapshots_feature_enabled', 'true'
5758
}
5859
setting 'xpack.autoscaling.enabled', 'true'
5960
setting 'xpack.eql.enabled', 'true'

‎docs/reference/ilm/apis/get-lifecycle.asciidoc

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,9 @@ If the request succeeds, the body of the response contains the policy definition
100100
"delete": {
101101
"min_age": "30d",
102102
"actions": {
103-
"delete": {}
103+
"delete": {
104+
"delete_searchable_snapshot": true
105+
}
104106
}
105107
}
106108
}

‎docs/reference/ilm/policy-definitions.asciidoc

Lines changed: 47 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -112,6 +112,7 @@ policy definition.
112112
- <<ilm-unfollow-action,Unfollow>>
113113
- <<ilm-allocate-action,Allocate>>
114114
- <<ilm-freeze-action,Freeze>>
115+
- <<ilm-searchable-snapshot-action, Searchable Snapshot>>
115116
* Delete
116117
- <<ilm-wait-for-snapshot-action,Wait For Snapshot>>
117118
- <<ilm-delete-action,Delete>>
@@ -266,7 +267,15 @@ Phases allowed: delete.
266267

267268
The Delete Action does just that, it deletes the index.
268269

269-
This action does not have any options associated with it.
270+
[[ilm-delete-action-options]]
271+
.Delete
272+
[options="header"]
273+
|======
274+
| Name | Required | Default | Description
275+
| `delete_searchable_snapshot` | no | true | Deletes the searchable snapshot created in the cold phase, if a
276+
snapshot was created (ie. if the <<ilm-searchable-snapshot-action,
277+
Searchable Snapshot Action>> was used in the cold phase)
278+
|======
270279

271280
[source,console]
272281
--------------------------------------------------
@@ -580,6 +589,43 @@ The above example illustrates a policy which attempts to delete an
580589
index one day after the index has been rolled over. It does not
581590
delete the index one day after it has been created.
582591

592+
[[ilm-searchable-snapshot-action]]
593+
==== Searchable Snapshot
594+
595+
Phases allowed: cold.
596+
597+
This action takes a snapshot of the managed index in the configured repository
598+
and mounts it as a searchable snapshot.
599+
600+
[[ilm-searchable-snapshot-options]]
601+
.Searchable Snapshot Options
602+
[options="header"]
603+
|======
604+
| Name | Required | Default | Description
605+
| `snapshot_repository` | yes | - | Repository used to store the snapshot created by this action. The snapshot will be,
606+
by default, deleted by the <<ilm-delete-action, Delete Action>> in the delete phase, if
607+
configured, but the user can configure the <<ilm-delete-action, Delete Action>> to keep
608+
the snapshot using the `delete_searchable_snapshot` option.
609+
610+
|======
611+
612+
[source,console]
613+
--------------------------------------------------
614+
PUT _ilm/policy/my_policy
615+
{
616+
"policy": {
617+
"phases": {
618+
"cold": {
619+
"actions": {
620+
"searchable_snapshot" : {
621+
"snapshot_repository" : "backing_repo"
622+
}
623+
}
624+
}
625+
}
626+
}
627+
}
628+
--------------------------------------------------
583629
[[ilm-set-priority-action]]
584630
==== Set Priority
585631

‎docs/reference/ilm/update-lifecycle-policy.asciidoc

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,9 @@ with its version bumped to 2.
113113
"delete": {
114114
"min_age": "10d",
115115
"actions": {
116-
"delete": {}
116+
"delete": {
117+
"delete_searchable_snapshot": true
118+
}
117119
}
118120
}
119121
}

‎docs/reference/rest-api/index.asciidoc

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -31,6 +31,7 @@ endif::[]
3131
* <<indices-reload-analyzers,Reload Search Analyzers API>>
3232
* <<rollup-apis,Rollup APIs>>
3333
* <<search, Search APIs>>
34+
* <<searchable-snapshots-apis, Searchable snapshots APIs>>
3435
* <<security-api,Security APIs>>
3536
* <<snapshot-lifecycle-management-api,Snapshot lifecycle management APIs>>
3637
* <<transform-apis,{transform-cap} APIs>>
@@ -59,6 +60,7 @@ include::{es-repo-dir}/migration/migration.asciidoc[]
5960
include::{es-repo-dir}/indices/apis/reload-analyzers.asciidoc[]
6061
include::{es-repo-dir}/rollup/rollup-api.asciidoc[]
6162
include::{es-repo-dir}/search.asciidoc[]
63+
include::{es-repo-dir}/searchable-snapshots/apis/searchable-snapshots-apis.asciidoc[]
6264
include::{xes-repo-dir}/rest-api/security.asciidoc[]
6365
include::{es-repo-dir}/slm/apis/slm-api.asciidoc[]
6466
include::{es-repo-dir}/transform/apis/index.asciidoc[]

‎docs/reference/searchable-snapshots/apis/clear-cache.asciidoc

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
[role="xpack"]
2+
[testenv="basic"]
3+
[[searchable-snapshots-api-clear-cache]]
4+
=== Clear cache API
5+
++++
6+
<titleabbrev>Clear cache</titleabbrev>
7+
++++
8+
9+
experimental[]
10+
11+
Clear the cache of searchable snapshots.
12+
13+
[[searchable-snapshots-api-clear-cache-request]]
14+
==== {api-request-title}
15+
16+
`POST /_searchable_snapshots/cache/clear`
17+
18+
`POST /<index>/_searchable_snapshots/cache/clear`
19+
20+
[[searchable-snapshots-api-clear-cache-prereqs]]
21+
==== {api-prereq-title}
22+
23+
If the {es} {security-features} are enabled, you must have the
24+
`manage` cluster privilege and the `manage` index privilege
25+
for any included indices to use this API.
26+
For more information, see <<security-privileges>>.
27+
28+
[[searchable-snapshots-api-clear-cache-desc]]
29+
==== {api-description-title}
30+
31+
32+
[[searchable-snapshots-api-clear-cache-path-params]]
33+
==== {api-path-parms-title}
34+
35+
`<index>`::
36+
(Optional, string)
37+
A comma-separated list of index names for which the
38+
searchable snapshots cache must be cleared.
39+
40+
41+
[[searchable-snapshots-api-clear-cache-example]]
42+
==== {api-examples-title}
43+
////
44+
[source,console]
45+
-----------------------------------
46+
PUT /docs
47+
{
48+
"settings" : {
49+
"index.number_of_shards" : 1,
50+
"index.number_of_replicas" : 0
51+
}
52+
}
53+
54+
PUT /_snapshot/my_repository/my_snapshot?wait_for_completion=true
55+
{
56+
"include_global_state": false,
57+
"indices": "docs"
58+
}
59+
60+
DELETE /docs
61+
62+
POST /_snapshot/my_repository/my_snapshot/_mount?wait_for_completion=true
63+
{
64+
"index": "docs"
65+
}
66+
-----------------------------------
67+
// TEST[setup:setup-repository]
68+
////
69+
70+
Clears the cache of the index `docs`:
71+
72+
[source,console]
73+
--------------------------------------------------
74+
POST /docs/_searchable_snapshots/cache/clear
75+
--------------------------------------------------
76+
// TEST[continued]

‎docs/reference/searchable-snapshots/apis/get-stats.asciidoc

Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
[role="xpack"]
2+
[testenv="basic"]
3+
[[searchable-snapshots-api-stats]]
4+
=== Searchable snapshot statistics API
5+
++++
6+
<titleabbrev>Searchable snapshot statistics</titleabbrev>
7+
++++
8+
9+
experimental[]
10+
11+
Retrieve various statistics about searchable snapshots.
12+
13+
[[searchable-snapshots-api-stats-request]]
14+
==== {api-request-title}
15+
16+
`GET /_searchable_snapshots/stats`
17+
18+
`GET /<index>/_searchable_snapshots/stats`
19+
20+
[[searchable-snapshots-api-stats-prereqs]]
21+
==== {api-prereq-title}
22+
23+
If the {es} {security-features} are enabled, you must have the
24+
`manage` cluster privilege and the `manage` index privilege
25+
for any included indices to use this API.
26+
For more information, see <<security-privileges>>.
27+
28+
[[searchable-snapshots-api-stats-desc]]
29+
==== {api-description-title}
30+
31+
32+
[[searchable-snapshots-api-stats-path-params]]
33+
==== {api-path-parms-title}
34+
35+
`<index>`::
36+
(Optional, string)
37+
A comma-separated list of index names for which the
38+
statistics must be retrieved.
39+
40+
41+
[[searchable-snapshots-api-stats-example]]
42+
==== {api-examples-title}
43+
////
44+
[source,console]
45+
-----------------------------------
46+
PUT /docs
47+
{
48+
"settings" : {
49+
"index.number_of_shards" : 1,
50+
"index.number_of_replicas" : 0
51+
}
52+
}
53+
54+
PUT /_snapshot/my_repository/my_snapshot?wait_for_completion=true
55+
{
56+
"include_global_state": false,
57+
"indices": "docs"
58+
}
59+
60+
DELETE /docs
61+
62+
POST /_snapshot/my_repository/my_snapshot/_mount?wait_for_completion=true
63+
{
64+
"index": "docs"
65+
}
66+
-----------------------------------
67+
// TEST[setup:setup-repository]
68+
////
69+
70+
Retrieves the statistics of the index `docs`:
71+
72+
[source,console]
73+
--------------------------------------------------
74+
GET /docs/_searchable_snapshots/stats
75+
--------------------------------------------------
76+
// TEST[continued]

0 commit comments

Comments
 (0)