{index} syntax no supported by rollover_alias in 8.11.1

[Ewemek]

Elasticsearch Version

8.11.1

Installed Plugins

No response

Java Version

bundled

OS Version

Linux fr-btc-01 5.11.0-25-generic #27-Ubuntu SMP Fri Jul 9 23:06:29 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux

Problem Description

I'm trying to upgrade from 7.11.1, without migrating or anything.
So I'm recreating my ILM (using kibana) from scratch.
My index name are: filebeat-8.11.1-%{+YYYY.MM.dd}
My ILM has no problem whatsoever, but when I edit my index template so it uses this ILM with those settings:

{
  ...              
  "template": {
    "settings": {
      ...     
      "index.lifecycle.rollover_alias": "{index}-000001"    
    }
  }
  ...
  "aliases": {
    "{index}-000001": {
      "is_write_index": true
    }
  }
  ...
}

But my index's ILM execution keeps failing: illegal_argument_exception: index.lifecycle.rollover_alias [{index}-000001] does not point to index [filebeat-8.11.0-2023.11.29]

Those settings worked in 7.11.1 but I have the impression that the rollover_alias setting is not interpreting the {index} anymore in 8.11.1.

Steps to Reproduce

Create an ILM, and use it in an index template, and use this settings on the index template:

{
...
"template": {
"settings": {
...
"index.lifecycle.rollover_alias": "{index}-000001"
}
}
...
"aliases": {
"{index}-000001": {
"is_write_index": true
}
}
...
}

Logs (if relevant)

illegal_argument_exception: index.lifecycle.rollover_alias [{index}-000001] does not point to index [filebeat-8.11.0-2023.11.29]`

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[parkertimmins]

Hi, thanks for reporting this behavior. You are correct that {index} is not being interpolated in index.lifecycle.rollover_alias, but this behavior also existed in version 7.11.1. We do not have plans to add interpolation to index.lifecycle.rollover_alias at this time, so we're going to close this ticket as not-planned right now.

I have verified that this behavior existed in 7.11.1 using the following example.

PUT _ilm/policy/my-ilm-policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_age": "2m"
          }
        }
      }
    }
  }
}

PUT _cluster/settings
{
  "persistent": {
    "indices.lifecycle.poll_interval": "5s"
  }
}

PUT _index_template/template_with_ilm
{
  "index_patterns": ["filebeat-*"],
  "template": {
     "settings": {
      "index.lifecycle.name": "my-ilm-policy",
      "index.lifecycle.rollover_alias": "{index}-000001"
    },
    "aliases": {
      "{index}-000001": {
        "is_write_index": true
      }
    }
  },
  "priority": 500
}

POST filebeat-8.11.1-2024.01.04/_doc
{
  "message": "hello!"
}

# call after ilm poll_interval has elapsed
GET filebeat-8.11.1-2024.01.04/_ilm/explain

This returns the same exception which you described.

 "type" : "illegal_argument_exception",
 "reason" : "index.lifecycle.rollover_alias [{index}-000001] does not point to index [filebeat-8.11.1-2024.01.04]",

Additionally, with POST _index_template/_simulate_index/filebeat-8.11.1-2024.01.04, we can see that interpolation was not occurring on the index.lifecycle.rollover_alias field in version 7.11.1:

{
  "template" : {
    "settings" : {
      "index" : {
        "lifecycle" : {
          "name" : "my-ilm-policy",
          "rollover_alias" : "{index}-000001"
        }
      }
    },
    "aliases" : {
      "filebeat-8.11.1-2024.01.04-000001" : {
        "is_write_index" : true
      }
    }
  },
  "overlapping" : [ ]
}