Skip to content

[7.x] Add the frozen tier node role and ILM phase (#68605) #68612

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 5, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
- match:
$body: |
/ #ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
^ ((\d{1,3}\.){3}\d{1,3} \s+ \d+ \s+ \d* \s+ (-)?\d* \s+ ((-)?\d*(\.\d+)?)? \s+ ((-)?\d*(\.\d+)?)?\s+ ((-)?\d*(\.\d+)?)? \s+ (-|[cdhilmrstvw]{1,11}) \s+ [-*x] \s+ (\S+\s?)+ \n)+ $/
^ ((\d{1,3}\.){3}\d{1,3} \s+ \d+ \s+ \d* \s+ (-)?\d* \s+ ((-)?\d*(\.\d+)?)? \s+ ((-)?\d*(\.\d+)?)?\s+ ((-)?\d*(\.\d+)?)? \s+ (-|[cdfhilmrstvw]{1,11}) \s+ [-*x] \s+ (\S+\s?)+ \n)+ $/

- do:
cat.nodes:
Expand All @@ -16,7 +16,7 @@
- match:
$body: |
/^ ip \s+ heap\.percent \s+ ram\.percent \s+ cpu \s+ load_1m \s+ load_5m \s+ load_15m \s+ node\.role \s+ master \s+ name \n
((\d{1,3}\.){3}\d{1,3} \s+ \d+ \s+ \d* \s+ (-)?\d* \s+ ((-)?\d*(\.\d+)?)? \s+ ((-)?\d*(\.\d+)?)? \s+ ((-)?\d*(\.\d+)?)? \s+ (-|[cdhilmrstvw]{1,11}) \s+ [-*x] \s+ (\S+\s?)+ \n)+ $/
((\d{1,3}\.){3}\d{1,3} \s+ \d+ \s+ \d* \s+ (-)?\d* \s+ ((-)?\d*(\.\d+)?)? \s+ ((-)?\d*(\.\d+)?)? \s+ ((-)?\d*(\.\d+)?)? \s+ (-|[cdfhilmrstvw]{1,11}) \s+ [-*x] \s+ (\S+\s?)+ \n)+ $/

- do:
cat.nodes:
Expand Down
16 changes: 14 additions & 2 deletions docs/reference/datatiers.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,8 @@ typically share the same hardware profile:
and hold your most recent, most-frequently-accessed data.
* <<warm-tier, Warm tier>> nodes hold time series data that is accessed less-frequently
and rarely needs to be updated.
* <<cold-tier, Cold tier>> nodes hold time series data that is accessed occasionally and not normally updated.
* <<cold-tier, Cold tier>> nodes hold time series data that is accessed infrequently and not normally updated.
* <<frozen-tier, Frozen tier>> nodes hold time series data that is accessed rarely and never updated.

When you index documents directly to a specific index, they remain on content tier nodes indefinitely.

Expand Down Expand Up @@ -74,12 +75,23 @@ For resiliency, indices in the warm tier should be configured to use one or more
=== Cold tier

Once data is no longer being updated, it can move from the warm tier to the cold tier where it
stays for the rest of its life.
stays while being queried infrequently.
The cold tier is still a responsive query tier, but data in the cold tier is not normally updated.
As data transitions into the cold tier it can be compressed and shrunken.
For resiliency, indices in the cold tier can rely on
<<ilm-searchable-snapshot, searchable snapshots>>, eliminating the need for replicas.

[discrete]
[[frozen-tier]]
=== Frozen tier

Once data is no longer being queried, or being queried rarely, it may move from the cold tier
to the frozen tier where it stays for the rest of its life.
The frozen tier is a less responsive query tier than the cold tier, and data in the frozen tier is
not normally updated. As data transitions into the frozen tier it can be compressed and shrunken.
For resiliency, indices in the frozen tier can rely on <<ilm-searchable-snapshot, searchable
snapshots>>, eliminating the need for replicas or even a local copy.

[discrete]
[[data-tier-allocation]]
=== Data tier index allocation
Expand Down
13 changes: 13 additions & 0 deletions docs/reference/glossary.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -197,6 +197,19 @@ Frozen indices use a memory-efficient shard implementation and throttle searches
Searching a frozen index is lower overhead than re-opening a closed index to enable searching.
// end::frozen-index-def[]

[[glossary-frozen-phase]] frozen phase ::
// tag::frozen-phase-def[]
The fourth possible phase in the <<glossary-index-lifecycle,index lifecycle>>.
In the frozen phase, an index is no longer updated and queried rarely.
The information still needs to be searchable, but it’s okay if those queries are extremely slow.
// end::frozen-phase-def[]

[[glossary-frozen-tier]] frozen tier::
// tag::frozen-tier-def[]
A <<glossary-data-tier, data tier>> that contains nodes that hold time series data
that is accessed rarely and not normally updated.
// end::frozen-tier-def[]

[[glossary-hidden-index]] hidden index ::
// tag::hidden-index-def[]
An index that is excluded by default when you access indices using a wildcard expression.
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/ilm/actions/_ilm-action-template.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ docs/reference/ilm/actions.asciidoc
[[ilm-sample]]
=== Sample

Phases allowed: hot, warm, cold, delete.
Phases allowed: hot, warm, cold, frozen, delete.

////
INTRO
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/ilm/actions/ilm-allocate.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
[[ilm-allocate]]
=== Allocate

Phases allowed: warm, cold.
Phases allowed: warm, cold, frozen.

Updates the index settings to change which nodes are allowed to host the index shards
and change the number of replicas.
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/ilm/actions/ilm-delete.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Permanently removes the index.
`delete_searchable_snapshot`::
beta:[]
(Optional, Boolean)
Deletes the searchable snapshot created in the cold phase.
Deletes the searchable snapshot created in a previous phase.
Defaults to `true`.
This option is applicable when the <<ilm-searchable-snapshot,searchable
snapshot>> action is used in the cold phase.
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/ilm/actions/ilm-freeze.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
[[ilm-freeze]]
=== Freeze

Phases allowed: cold.
Phases allowed: cold, frozen.

<<frozen-indices, Freezes>> an index to minimize its memory footprint.

Expand Down
11 changes: 9 additions & 2 deletions docs/reference/ilm/actions/ilm-migrate.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
[[ilm-migrate]]
=== Migrate

Phases allowed: warm, cold.
Phases allowed: warm, cold, frozen.

Moves the index to the <<data-tiers, data tier>> that corresponds
to the current phase by updating the <<tier-preference-allocation-filter, `index.routing.allocation.include._tier_preference`>>
index setting.
{ilm-init} automatically injects the migrate action in the warm and cold
{ilm-init} automatically injects the migrate action in the warm, cold, and frozen
phases if no allocation options are specified with the <<ilm-allocate, allocate>> action.
If you specify an allocate action that only modifies the number of index
replicas, {ilm-init} reduces the number of replicas before migrating the index.
Expand All @@ -25,6 +25,13 @@ to `data_cold,data_warm,data_hot`. This moves the index to nodes in the
<<cold-tier, cold tier>>. If there are no nodes in the cold tier, it falls back to the
<<warm-tier, warm>> tier, or the <<hot-tier, hot>> tier if there are no warm nodes available.

In the frozen phase, the `migrate` action sets
<<tier-preference-allocation-filter, `index.routing.allocation.include._tier_preference`>>
to `data_frozen,data_cold,data_warm,data_hot`. This moves the index to nodes in the
<<frozen-tier, frozen tier>>. If there are no nodes in the frozen tier, it falls back to the
<<cold-tier, cold>> tier, then the <<warm-tier, warm>> tier, then finally the <<hot-tier, hot>>
tier.

The migrate action is not allowed in the hot phase.
The initial index allocation is performed <<data-tier-allocation, automatically>>,
and can be configured manually or via <<index-templates, index templates>>.
Expand Down
4 changes: 2 additions & 2 deletions docs/reference/ilm/actions/ilm-searchable-snapshot.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

beta::[]

Phases allowed: hot, cold.
Phases allowed: hot, cold, frozen.

Takes a snapshot of the managed index in the configured repository
and mounts it as a searchable snapshot.
Expand All @@ -17,7 +17,7 @@ will reject the policy.

IMPORTANT: If the `searchable_snapshot` action is used in the `hot` phase the
subsequent phases cannot define any of the `shrink`, `forcemerge`, `freeze` or
`searchable_snapshot` (also available in the cold phase) actions.
`searchable_snapshot` (also available in the cold and frozen phases) actions.

[NOTE]
This action cannot be performed on a data stream's write index. Attempts to do
Expand Down
4 changes: 2 additions & 2 deletions docs/reference/ilm/actions/ilm-set-priority.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
[[ilm-set-priority]]
=== Set priority

Phases allowed: hot, warm, cold.
Phases allowed: hot, warm, cold, frozen.

Sets the <<recovery-prioritization, priority>> of the index as
soon as the policy enters the hot, warm, or cold phase.
soon as the policy enters the hot, warm, cold, or frozen phase.
Higher priority indices are recovered before indices with lower priorities following a node restart.

Generally, indexes in the hot phase should have the highest value and
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/ilm/actions/ilm-unfollow.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
[[ilm-unfollow]]
=== Unfollow

Phases allowed: hot, warm, cold.
Phases allowed: hot, warm, cold, frozen.

Converts a {ref}/ccr-apis.html[{ccr-init}] follower index into a regular index.
This enables the shrink, rollover, and searchable snapshot actions
Expand Down
16 changes: 12 additions & 4 deletions docs/reference/ilm/ilm-index-lifecycle.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,14 @@
<titleabbrev>Index lifecycle</titleabbrev>
++++

{ilm-init} defines four index lifecycle _phases_:
{ilm-init} defines five index lifecycle _phases_:

* **Hot**: The index is actively being updated and queried.
* **Warm**: The index is no longer being updated but is still being queried.
* **Cold**: The index is no longer being updated and is seldom queried. The
information still needs to be searchable, but it's okay if those queries are
slower.
* **Cold**: The index is no longer being updated and is queried infrequently. The information still
needs to be searchable, but it's okay if those queries are slower.
* **Frozen**: The index is no longer being updated and is queried rarely. The information still
needs to be searchable, but it's okay if those queries are extremely slow.
* **Delete**: The index is no longer needed and can safely be removed.

An index's _lifecycle policy_ specifies which phases
Expand Down Expand Up @@ -107,6 +108,13 @@ ifdef::permanently-unreleased-branch[]
- <<ilm-rollup,Rollup>>
endif::[]
- <<ilm-searchable-snapshot, Searchable Snapshot>>
* Frozen
- <<ilm-set-priority,Set Priority>>
- <<ilm-unfollow,Unfollow>>
- <<ilm-allocate,Allocate>>
- <<ilm-migrate,Migrate>>
- <<ilm-freeze,Freeze>>
- <<ilm-searchable-snapshot, Searchable Snapshot>>
* Delete
- <<ilm-wait-for-snapshot,Wait For Snapshot>>
- <<ilm-delete,Delete>>
4 changes: 2 additions & 2 deletions docs/reference/ilm/ilm-tutorial.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,8 @@ or the {ilm-init} APIs.
=== Create a lifecycle policy

A lifecycle policy specifies the phases in the index lifecycle
and the actions to perform in each phase. A lifecycle can have up to four phases:
`hot`, `warm`, `cold`, and `delete`.
and the actions to perform in each phase. A lifecycle can have up to five phases:
`hot`, `warm`, `cold`, `frozen`, and `delete`.

For example, you might define a `timeseries_policy` that has two phases:

Expand Down
5 changes: 3 additions & 2 deletions docs/reference/modules/indices/recovery.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,9 @@ You can view a list of in-progress and completed recoveries using the
(<<cluster-update-settings,Dynamic>>) Limits total inbound and outbound
recovery traffic for each node. Applies to both peer recoveries as well
as snapshot recoveries (i.e., restores from a snapshot). Defaults to `40mb`
unless the node is a <<cold-tier, dedicated cold node>> in which case the
default relates to the total memory available to the node:
unless the node is a dedicated <<cold-tier, cold>> or
<<frozen-tier, frozen>> node, in which case the default relates to the
total memory available to the node:

.Recovery Rate for Cold Nodes
[options="header"]
Expand Down
12 changes: 12 additions & 0 deletions docs/reference/rest-api/usage.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -323,6 +323,18 @@ GET /_xpack/usage
"primary_shard_size_median_bytes" : 0,
"primary_shard_size_mad_bytes" : 0
},
"data_frozen" : {
"node_count" : 1,
"index_count" : 0,
"total_shard_count" : 0,
"primary_shard_count" : 0,
"doc_count" : 0,
"total_size_bytes" : 0,
"primary_size_bytes" : 0,
"primary_shard_size_avg_bytes" : 0,
"primary_shard_size_median_bytes" : 0,
"primary_shard_size_mad_bytes" : 0
},
"data_cold" : {
"node_count" : 0,
"index_count" : 0,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
- match:
$body: |
/ #ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name
^ ((\d{1,3}\.){3}\d{1,3} \s+ \d+ \s+ \d* \s+ (-)?\d* \s+ ((-)?\d*(\.\d+)?)? \s+ ((-)?\d*(\.\d+)?)?\s+ ((-)?\d*(\.\d+)?)? \s+ (-|[cdhilmrstvw]{1,11}) \s+ [-*x] \s+ (\S+\s?)+ \n)+ $/
^ ((\d{1,3}\.){3}\d{1,3} \s+ \d+ \s+ \d* \s+ (-)?\d* \s+ ((-)?\d*(\.\d+)?)? \s+ ((-)?\d*(\.\d+)?)?\s+ ((-)?\d*(\.\d+)?)? \s+ (-|[cdfhilmrstvw]{1,11}) \s+ [-*x] \s+ (\S+\s?)+ \n)+ $/

- do:
cat.nodes:
Expand All @@ -16,7 +16,7 @@
- match:
$body: |
/^ ip \s+ heap\.percent \s+ ram\.percent \s+ cpu \s+ load_1m \s+ load_5m \s+ load_15m \s+ node\.role \s+ master \s+ name \n
((\d{1,3}\.){3}\d{1,3} \s+ \d+ \s+ \d* \s+ (-)?\d* \s+ ((-)?\d*(\.\d+)?)? \s+ ((-)?\d*(\.\d+)?)? \s+ ((-)?\d*(\.\d+)?)? \s+ (-|[cdhilmrstvw]{1,11}) \s+ [-*x] \s+ (\S+\s?)+ \n)+ $/
((\d{1,3}\.){3}\d{1,3} \s+ \d+ \s+ \d* \s+ (-)?\d* \s+ ((-)?\d*(\.\d+)?)? \s+ ((-)?\d*(\.\d+)?)? \s+ ((-)?\d*(\.\d+)?)? \s+ (-|[cdfhilmrstvw]{1,11}) \s+ [-*x] \s+ (\S+\s?)+ \n)+ $/

- do:
cat.nodes:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,8 @@ public class RecoverySettings {
// if the node is not a data node, this value doesn't matter, use the default
return defaultMaxBytesPerSec.getStringRep();
}
if ((dataRoles.size() > 1 || dataRoles.get(0).roleName().equals("data_cold") == false) ||
roles.contains(DiscoveryNodeRole.MASTER_ROLE)) {
// if the node is not a dedicated cold node, use the default
if (dataRoles.stream().allMatch(dn -> dn.roleName().equals("data_cold") || dn.roleName().equals("data_frozen")) == false) {
// the node is not a dedicated cold and/or frozen node, use the default
return defaultMaxBytesPerSec.getStringRep();
}
/*
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@
body:
# Notice that adding new default roles requires extending this list
roles: ["master",
"data", "data_content", "data_hot", "data_warm", "data_cold",
"data", "data_content", "data_hot", "data_warm", "data_cold", "data_frozen",
"ingest", "ml", "transform", "remote_cluster_client"]

- match: { "acknowledged": true }
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,8 @@ public List<DiscoveryNodeRole> roles() {
DataTier.DATA_CONTENT_NODE_ROLE,
DataTier.DATA_HOT_NODE_ROLE,
DataTier.DATA_WARM_NODE_ROLE,
DataTier.DATA_COLD_NODE_ROLE
DataTier.DATA_COLD_NODE_ROLE,
DataTier.DATA_FROZEN_NODE_ROLE
);
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,10 @@ public class DataTier {
public static final String DATA_HOT = "data_hot";
public static final String DATA_WARM = "data_warm";
public static final String DATA_COLD = "data_cold";
public static final String DATA_FROZEN = "data_frozen";

public static final Set<String> ALL_DATA_TIERS = new HashSet<>(Arrays.asList(DATA_CONTENT, DATA_HOT, DATA_WARM, DATA_COLD));
public static final Set<String> ALL_DATA_TIERS =
new HashSet<>(Arrays.asList(DATA_CONTENT, DATA_HOT, DATA_WARM, DATA_COLD, DATA_FROZEN));

/**
* Returns true if the given tier name is a valid tier
Expand All @@ -47,7 +49,8 @@ public static boolean validTierName(String tierName) {
return DATA_CONTENT.equals(tierName) ||
DATA_HOT.equals(tierName) ||
DATA_WARM.equals(tierName) ||
DATA_COLD.equals(tierName);
DATA_COLD.equals(tierName) ||
DATA_FROZEN.equals(tierName);
}

/**
Expand Down Expand Up @@ -167,6 +170,27 @@ public DiscoveryNodeRole getCompatibilityRole(Version nodeVersion) {
}
};

public static DiscoveryNodeRole DATA_FROZEN_NODE_ROLE = new DiscoveryNodeRole("data_frozen", "f", true) {
@Override
public boolean isEnabledByDefault(final Settings settings) {
return DiscoveryNode.hasRole(settings, DiscoveryNodeRole.DATA_ROLE);
}

@Override
public Setting<Boolean> legacySetting() {
// we do not register these settings, they're not intended to be used externally, only for proper defaults
return Setting.boolSetting(
"node.data_frozen",
settings ->
// Don't use DiscoveryNode#isDataNode(Settings) here, as it is called before all plugins are initialized
Boolean.toString(DiscoveryNode.hasRole(settings, DiscoveryNodeRole.DATA_ROLE)),
Setting.Property.Deprecated,
Setting.Property.NodeScope
);
}

};

public static boolean isContentNode(DiscoveryNode discoveryNode) {
return discoveryNode.getRoles().contains(DATA_CONTENT_NODE_ROLE) || discoveryNode.getRoles().contains(DiscoveryNodeRole.DATA_ROLE);
}
Expand All @@ -183,6 +207,10 @@ public static boolean isColdNode(DiscoveryNode discoveryNode) {
return discoveryNode.getRoles().contains(DATA_COLD_NODE_ROLE) || discoveryNode.getRoles().contains(DiscoveryNodeRole.DATA_ROLE);
}

public static boolean isFrozenNode(DiscoveryNode discoveryNode) {
return discoveryNode.getRoles().contains(DATA_FROZEN_NODE_ROLE) || discoveryNode.getRoles().contains(DiscoveryNodeRole.DATA_ROLE);
}

/**
* This setting provider injects the setting allocating all newly created indices with
* {@code index.routing.allocation.include._tier: "data_hot"} unless the user overrides the
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -437,7 +437,8 @@ public Set<DiscoveryNodeRole> getRoles() {
DataTier.DATA_CONTENT_NODE_ROLE,
DataTier.DATA_HOT_NODE_ROLE,
DataTier.DATA_WARM_NODE_ROLE,
DataTier.DATA_COLD_NODE_ROLE));
DataTier.DATA_COLD_NODE_ROLE,
DataTier.DATA_FROZEN_NODE_ROLE));
}

@Override
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,8 +34,8 @@ public class MigrateAction implements LifecycleAction {
public static final ParseField ENABLED_FIELD = new ParseField("enabled");

// Represents an ordered list of data tiers from cold to hot (or slow to fast)
private static final List<String> COLD_TO_HOT_TIERS = org.elasticsearch.common.collect.List.of(
DataTier.DATA_COLD, DataTier.DATA_WARM, DataTier.DATA_HOT
private static final List<String> FROZEN_TO_HOT_TIERS = org.elasticsearch.common.collect.List.of(
DataTier.DATA_FROZEN, DataTier.DATA_COLD, DataTier.DATA_WARM, DataTier.DATA_HOT
);

private static final ConstructingObjectParser<MigrateAction, Void> PARSER = new ConstructingObjectParser<>(NAME,
Expand Down Expand Up @@ -115,11 +115,11 @@ public List<Step> toSteps(Client client, String phase, StepKey nextStepKey) {
* This is usually used in conjunction with {@link DataTierAllocationDecider#INDEX_ROUTING_PREFER_SETTING}
*/
static String getPreferredTiersConfiguration(String targetTier) {
int indexOfTargetTier = COLD_TO_HOT_TIERS.indexOf(targetTier);
int indexOfTargetTier = FROZEN_TO_HOT_TIERS.indexOf(targetTier);
if (indexOfTargetTier == -1) {
throw new IllegalArgumentException("invalid data tier [" + targetTier + "]");
}
return COLD_TO_HOT_TIERS.stream().skip(indexOfTargetTier).collect(Collectors.joining(","));
return FROZEN_TO_HOT_TIERS.stream().skip(indexOfTargetTier).collect(Collectors.joining(","));
}

@Override
Expand Down
Loading