Skip to content

Commit 5b157b5

Browse files
committed
ingest: document fields that support templating (elastic#34536)
This change also updates many of the examples to use ecs as the example. Some additional minor improvements are also included. Part of elastic#33188
1 parent 38c2079 commit 5b157b5

File tree

1 file changed

+32
-31
lines changed

1 file changed

+32
-31
lines changed

docs/reference/ingest/ingest-node.asciidoc

+32-31
Original file line numberDiff line numberDiff line change
@@ -758,16 +758,16 @@ Accepts a single value or an array of values.
758758
[options="header"]
759759
|======
760760
| Name | Required | Default | Description
761-
| `field` | yes | - | The field to be appended to
762-
| `value` | yes | - | The value to be appended
761+
| `field` | yes | - | The field to be appended to. Supports <<accessing-template-fields,template snippets>>.
762+
| `value` | yes | - | The value to be appended. Supports <<accessing-template-fields,template snippets>>.
763763
|======
764764

765765
[source,js]
766766
--------------------------------------------------
767767
{
768768
"append": {
769-
"field": "field1",
770-
"value": ["item2", "item3", "item4"]
769+
"field": "tags",
770+
"value": ["production", "{{app}}", "{{owner}}"]
771771
}
772772
}
773773
--------------------------------------------------
@@ -794,7 +794,7 @@ the field is not a supported format or resultant value exceeds 2^63.
794794
--------------------------------------------------
795795
{
796796
"bytes": {
797-
"field": "foo"
797+
"field": "file.size"
798798
}
799799
}
800800
--------------------------------------------------
@@ -832,7 +832,7 @@ still be updated with the unconverted field value.
832832
--------------------------------------------------
833833
{
834834
"convert": {
835-
"field" : "foo",
835+
"field" : "url.port",
836836
"type": "integer"
837837
}
838838
}
@@ -856,8 +856,8 @@ in the same order they were defined as part of the processor definition.
856856
| `field` | yes | - | The field to get the date from.
857857
| `target_field` | no | @timestamp | The field that will hold the parsed date.
858858
| `formats` | yes | - | An array of the expected date formats. Can be a Joda pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
859-
| `timezone` | no | UTC | The timezone to use when parsing the date.
860-
| `locale` | no | ENGLISH | The locale to use when parsing the date, relevant when parsing month names or week days.
859+
| `timezone` | no | UTC | The timezone to use when parsing the date. Supports <<accessing-template-fields,template snippets>>.
860+
| `locale` | no | ENGLISH | The locale to use when parsing the date, relevant when parsing month names or week days. Supports <<accessing-template-fields,template snippets>>.
861861
|======
862862

863863
Here is an example that adds the parsed date to the `timestamp` field based on the `initial_date` field:
@@ -895,8 +895,8 @@ the timezone and locale values.
895895
"field" : "initial_date",
896896
"target_field" : "timestamp",
897897
"formats" : ["ISO8601"],
898-
"timezone" : "{{ my_timezone }}",
899-
"locale" : "{{ my_locale }}"
898+
"timezone" : "{{my_timezone}}",
899+
"locale" : "{{my_locale}}"
900900
}
901901
}
902902
]
@@ -1041,12 +1041,12 @@ understands this to mean `2016-04-01` as is explained in the <<date-math-index-n
10411041
|======
10421042
| Name | Required | Default | Description
10431043
| `field` | yes | - | The field to get the date or timestamp from.
1044-
| `index_name_prefix` | no | - | A prefix of the index name to be prepended before the printed date.
1045-
| `date_rounding` | yes | - | How to round the date when formatting the date into the index name. Valid values are: `y` (year), `M` (month), `w` (week), `d` (day), `h` (hour), `m` (minute) and `s` (second).
1044+
| `index_name_prefix` | no | - | A prefix of the index name to be prepended before the printed date. Supports <<accessing-template-fields,template snippets>>.
1045+
| `date_rounding` | yes | - | How to round the date when formatting the date into the index name. Valid values are: `y` (year), `M` (month), `w` (week), `d` (day), `h` (hour), `m` (minute) and `s` (second). Supports <<accessing-template-fields,template snippets>>.
10461046
| `date_formats` | no | yyyy-MM-dd'T'HH:mm:ss.SSSZ | An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a Joda pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
10471047
| `timezone` | no | UTC | The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.
10481048
| `locale` | no | ENGLISH | The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.
1049-
| `index_name_format` | no | yyyy-MM-dd | The format to be used when printing the parsed date into the index name. An valid Joda pattern is expected here.
1049+
| `index_name_format` | no | yyyy-MM-dd | The format to be used when printing the parsed date into the index name. An valid Joda pattern is expected here. Supports <<accessing-template-fields,template snippets>>.
10501050
|======
10511051

10521052
[[dissect-processor]]
@@ -1372,14 +1372,15 @@ to the requester.
13721372
[options="header"]
13731373
|======
13741374
| Name | Required | Default | Description
1375-
| `message` | yes | - | The error message of the `FailException` thrown by the processor
1375+
| `message` | yes | - | The error message thrown by the processor. Supports <<accessing-template-fields,template snippets>>.
13761376
|======
13771377

13781378
[source,js]
13791379
--------------------------------------------------
13801380
{
13811381
"fail": {
1382-
"message": "an error message"
1382+
"if" : "ctx.tags.contains('production') != true",
1383+
"message": "The production tag is not present, found tags: {{tags}}"
13831384
}
13841385
}
13851386
--------------------------------------------------
@@ -2084,7 +2085,7 @@ Removes existing fields. If one field doesn't exist, an exception will be thrown
20842085
[options="header"]
20852086
|======
20862087
| Name | Required | Default | Description
2087-
| `field` | yes | - | Fields to be removed
2088+
| `field` | yes | - | Fields to be removed. Supports <<accessing-template-fields,template snippets>>.
20882089
| `ignore_missing` | no | `false` | If `true` and `field` does not exist or is `null`, the processor quietly exits without modifying the document
20892090
|======
20902091

@@ -2094,7 +2095,7 @@ Here is an example to remove a single field:
20942095
--------------------------------------------------
20952096
{
20962097
"remove": {
2097-
"field": "foo"
2098+
"field": "user_agent"
20982099
}
20992100
}
21002101
--------------------------------------------------
@@ -2106,7 +2107,7 @@ To remove multiple fields, you can use the following query:
21062107
--------------------------------------------------
21072108
{
21082109
"remove": {
2109-
"field": ["foo", "bar"]
2110+
"field": ["user_agent", "url"]
21102111
}
21112112
}
21122113
--------------------------------------------------
@@ -2120,18 +2121,18 @@ Renames an existing field. If the field doesn't exist or the new name is already
21202121
.Rename Options
21212122
[options="header"]
21222123
|======
2123-
| Name | Required | Default | Description
2124-
| `field` | yes | - | The field to be renamed
2125-
| `target_field` | yes | - | The new name of the field
2126-
| `ignore_missing` | no | `false` | If `true` and `field` does not exist, the processor quietly exits without modifying the document
2124+
| Name | Required | Default | Description
2125+
| `field` | yes | - | The field to be renamed. Supports <<accessing-template-fields,template snippets>>.
2126+
| `target_field` | yes | - | The new name of the field. Supports <<accessing-template-fields,template snippets>>.
2127+
| `ignore_missing` | no | `false` | If `true` and `field` does not exist, the processor quietly exits without modifying the document
21272128
|======
21282129

21292130
[source,js]
21302131
--------------------------------------------------
21312132
{
21322133
"rename": {
2133-
"field": "foo",
2134-
"target_field": "foobar"
2134+
"field": "provider",
2135+
"target_field": "cloud.provider"
21352136
}
21362137
}
21372138
--------------------------------------------------
@@ -2249,18 +2250,18 @@ its value will be replaced with the provided one.
22492250
.Set Options
22502251
[options="header"]
22512252
|======
2252-
| Name | Required | Default | Description
2253-
| `field` | yes | - | The field to insert, upsert, or update
2254-
| `value` | yes | - | The value to be set for the field
2255-
| `override`| no | true | If processor will update fields with pre-existing non-null-valued field. When set to `false`, such fields will not be touched.
2253+
| Name | Required | Default | Description
2254+
| `field` | yes | - | The field to insert, upsert, or update. Supports <<accessing-template-fields,template snippets>>.
2255+
| `value` | yes | - | The value to be set for the field. Supports <<accessing-template-fields,template snippets>>.
2256+
| `override` | no | true | If processor will update fields with pre-existing non-null-valued field. When set to `false`, such fields will not be touched.
22562257
|======
22572258

22582259
[source,js]
22592260
--------------------------------------------------
22602261
{
22612262
"set": {
2262-
"field": "field1",
2263-
"value": 582.1
2263+
"field": "host.os.name",
2264+
"value": "{{os}}"
22642265
}
22652266
}
22662267
--------------------------------------------------
@@ -2313,7 +2314,7 @@ Throws an error when the field is not an array.
23132314
--------------------------------------------------
23142315
{
23152316
"sort": {
2316-
"field": "field_to_sort",
2317+
"field": "array_field_to_sort",
23172318
"order": "desc"
23182319
}
23192320
}

0 commit comments

Comments
 (0)