Make timestamp field mapping part of data stream definition

[martijnvg]

Currently when creating a composable index template with a data stream definition, the timestamp field mapping must be defined too:

PUT /_index_template/logs_data_stream
{
  "index_patterns": [ "logs*" ],
  "data_stream": {
    "timestamp_field": "@timestamp"
  },
  "template": {
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        }
      }
    }
  }
}

The template is much cleaner to read if the mapping itself for timestamp field mapping
is inlined with the data stream definition:

PUT /_index_template/logs_data_stream
{
  "index_patterns": [ "logs*" ],
  "data_stream": {
    "timestamp_field": "@timestamp",
    "timestamp_mapping": {
      "type": "date"
    }
  }
}

Also since most of time, just "type": "date" will be provided, it makes sense to make this the default:

PUT /_index_template/logs_data_stream
{
  "index_patterns": [ "logs*" ],
  "data_stream": {
    "timestamp_field": "@timestamp"
  }
}

In the last two example the backing indices created will have the following mapping:

{
  "_doc": {
    "properties": {
      "@timestamp": {
        "type": "date"
      }
    }
  }
}

The simulate index template api should also include the inserted / generated timestamp field mapping
in the response.

[elasticmachine]

Pinging @elastic/es-search (:Search/Mapping)

[elasticmachine]

Pinging @elastic/es-core-features (:Core/Features/Data streams)

[martijnvg]

This issue was discussed and the fact that the required field mapping has to be configured the template mapping is confusing since the timestamp field name has to be configured in the data stream definition. Especially when getting started with data streams (and Elasticsearch). Also making timestamp field mapping optional, also improves the getting started experience.

Further more it should still be allowed to define the mapping for timestamp field in a component's or composable index's template section. This makes sense if all the other mappings are also defined in the template and otherwise mappings would need to be specified in two places.

[nicholasyin]

Has timestamp_field been deprecated in 7.15 as I don't find any document mentioning it? If so, how can I specify the timestamp field name?

[danhermann]

Has timestamp_field been deprecated in 7.15 as I don't find any document mentioning it? If so, how can I specify the timestamp field name?

As of now, data streams require the timestamp field to be named @timestamp.

[nicholasyin]

Has timestamp_field been deprecated in 7.15 as I don't find any document mentioning it? If so, how can I specify the timestamp field name?

As of now, data streams require the timestamp field to be named @timestamp.

K, thanks.