Skip to content

Commit fe54c2f

Browse files
author
Adam Locke
authored
[DOCS] Add dynamic runtime fields to docs (elastic#66194)
* [DOCS] Add dynamic runtime fields to docs. * Clarifying edits and example changes. * Creating better table and incorporating review comments. * Change numeral to superscript.
1 parent 9bb6a3a commit fe54c2f

File tree

3 files changed

+85
-69
lines changed

3 files changed

+85
-69
lines changed

docs/reference/mapping/dynamic/field-mapping.asciidoc

Lines changed: 31 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -1,35 +1,37 @@
11
[[dynamic-field-mapping]]
22
=== Dynamic field mapping
33

4-
By default, when a previously unseen field is found in a document,
5-
Elasticsearch will add the new field to the type mapping. This behaviour can
6-
be disabled, both at the document and at the <<object,`object`>> level, by
7-
setting the <<dynamic,`dynamic`>> parameter to `false` (to ignore new fields) or to `strict` (to throw
8-
an exception if an unknown field is encountered).
9-
10-
Assuming `dynamic` field mapping is enabled, some simple rules are used to
11-
determine which data type the field should have:
12-
13-
[horizontal]
14-
*JSON data type*:: *Elasticsearch data type*
15-
16-
`null`:: No field is added.
17-
`true` or `false`:: <<boolean,`boolean`>> field
18-
floating{nbsp}point{nbsp}number:: <<number,`float`>> field
19-
integer:: <<number,`long`>> field
20-
object:: <<object,`object`>> field
21-
array:: Depends on the first non-`null` value in the array.
22-
string:: Either a <<date,`date`>> field
23-
(if the value passes <<date-detection,date detection>>),
24-
a <<number,`double`>> or <<number,`long`>> field
25-
(if the value passes <<numeric-detection,numeric detection>>)
26-
or a <<text,`text`>> field, with a <<keyword,`keyword`>> sub-field.
27-
28-
These are the only <<mapping-types,field data types>> that are dynamically
29-
detected. All other data types must be mapped explicitly.
30-
31-
Besides the options listed below, dynamic field mapping rules can be further
32-
customised with <<dynamic-templates,`dynamic_templates`>>.
4+
When {es} detects a new field in a document, it _dynamically_ adds the field to
5+
the type mapping. The <<dynamic,`dynamic`>> parameter controls this behavior.
6+
7+
You can disable dynamic mapping, both at the document and at the
8+
<<object,`object`>> level. Setting the `dynamic` parameter to
9+
`false` ignores new fields, and `strict` rejects the document if {es}
10+
encounters an unknown field.
11+
12+
When dynamic field mapping is enabled, {es} uses the following rules to
13+
determine how to map data types for each field.
14+
15+
NOTE: These are the only <<mapping-types,field data types>> that {es} detects
16+
dynamically. All other data types must be mapped explicitly.
17+
18+
[cols="3"]
19+
|===
20+
h| JSON data type h| `dynamic:true` h| `dynamic:runtime`
21+
|`null` 2*| No field added
22+
|`true` or `false` 2*| `boolean`
23+
|floating point number | `float` | `double`
24+
|`integer` 2*| `long`
25+
|`object`<<dynamic-object-footnote,^1^>> 2*| `object`
26+
|`array` 2*| Depends on the first non-`null` value in the array
27+
|`string` that passes <<date-detection,date detection>> 2*| `date`
28+
|`string` that passes <<numeric-detection,numeric detection>> | `double` or `long` | `double`
29+
|`string` that doesn't pass `date` detection or `numeric` detection | `text` with a `.keyword` sub-field | `keyword`
30+
3+| [[dynamic-object-footnote]] ^1^Objects are always mapped as part of the `properties` section, even when the `dynamic` parameter is set to `runtime`. | |
31+
|===
32+
33+
Besides the following options, you can customize dynamic field mapping rules
34+
with <<dynamic-templates,`dynamic_templates`>>.
3335

3436
[[date-detection]]
3537
==== Date detection
@@ -129,4 +131,3 @@ PUT my-index-000001/_doc/1
129131

130132
<1> The `my_float` field is added as a <<number,`float`>> field.
131133
<2> The `my_integer` field is added as a <<number,`long`>> field.
132-
Lines changed: 46 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -1,24 +1,31 @@
11
[[dynamic]]
22
=== `dynamic`
33

4-
By default, fields can be added _dynamically_ to a document, or to
5-
<<object,inner objects>> within a document, just by indexing a document
6-
containing the new field. For instance:
4+
When you index a document containing a new field, {es} <<dynamic-mapping,adds the field dynamically>> to a document or to inner objects within a document. The
5+
following document adds the string field `username`, the object field
6+
`name`, and two string fields under the `name` object:
77

88
[source,console]
9-
--------------------------------------------------
10-
PUT my-index-000001/_doc/1 <1>
9+
----
10+
PUT my-index-000001/_doc/1
1111
{
1212
"username": "johnsmith",
13-
"name": {
13+
"name": { <1>
1414
"first": "John",
1515
"last": "Smith"
1616
}
1717
}
1818
1919
GET my-index-000001/_mapping <2>
20+
----
21+
<1> Refer to fields under the `name` object as `name.first` and `name.last`.
22+
<2> Check the mapping to view changes.
23+
24+
The following document adds two string fields: `email` and `name.middle`:
2025

21-
PUT my-index-000001/_doc/2 <3>
26+
[source,console]
27+
----
28+
PUT my-index-000001/_doc/2
2229
{
2330
"username": "marywhite",
2431
"email": "[email protected]",
@@ -29,35 +36,24 @@ PUT my-index-000001/_doc/2 <3>
2936
}
3037
}
3138
32-
GET my-index-000001/_mapping <4>
33-
--------------------------------------------------
34-
35-
<1> This document introduces the string field `username`, the object field
36-
`name`, and two string fields under the `name` object which can be
37-
referred to as `name.first` and `name.last`.
38-
<2> Check the mapping to verify the above.
39-
<3> This document adds two string fields: `email` and `name.middle`.
40-
<4> Check the mapping to verify the changes.
39+
GET my-index-000001/_mapping
40+
----
4141

42-
The details of how new fields are detected and added to the mapping is explained in <<dynamic-mapping>>.
42+
TIP: Use the <<indices-put-mapping,PUT mapping API>> to update the `dynamic`
43+
setting on existing fields.
4344

44-
The `dynamic` setting controls whether new fields can be added dynamically or
45-
not. It accepts three settings:
45+
[[dynamic-inner-objects]]
46+
==== Setting `dynamic` on inner objects
47+
<<object,Inner objects>> inherit the `dynamic` setting from their parent
48+
object or from the mapping type. In the following example, dynamic mapping is
49+
disabled at the type level, so no new top-level fields will be added
50+
dynamically.
4651

47-
[horizontal]
48-
`true`:: Newly detected fields are added to the mapping. (default)
49-
`false`:: Newly detected fields are ignored. These fields will not be indexed so will not be searchable
50-
but will still appear in the `_source` field of returned hits. These fields will not be added
51-
to the mapping, new fields must be added explicitly.
52-
`strict`:: If new fields are detected, an exception is thrown and the document is rejected. New fields
53-
must be explicitly added to the mapping.
54-
55-
The `dynamic` setting may be set at the mapping type level, and on each
56-
<<object,inner object>>. Inner objects inherit the setting from their parent
57-
object or from the mapping type. For instance:
52+
However, the `user.social_networks` object enables dynamic mapping, so you can
53+
add fields to this inner object.
5854

5955
[source,console]
60-
--------------------------------------------------
56+
----
6157
PUT my-index-000001
6258
{
6359
"mappings": {
@@ -68,20 +64,32 @@ PUT my-index-000001
6864
"name": {
6965
"type": "text"
7066
},
71-
"social_networks": { <3>
72-
"dynamic": true,
67+
"social_networks": {
68+
"dynamic": true, <3>
7369
"properties": {}
7470
}
7571
}
7672
}
7773
}
7874
}
7975
}
80-
--------------------------------------------------
76+
----
8177

82-
<1> Dynamic mapping is disabled at the type level, so no new top-level fields will be added dynamically.
78+
<1> Disables dynamic mapping at the type level.
8379
<2> The `user` object inherits the type-level setting.
84-
<3> The `user.social_networks` object enables dynamic mapping, so new fields may be added to this inner object.
80+
<3> Enables dynamic mapping for this inner object.
8581

86-
TIP: The `dynamic` setting can be updated on existing fields
87-
using the <<indices-put-mapping,PUT mapping API>>.
82+
[[dynamic-parameters]]
83+
==== Parameters for `dynamic`
84+
The `dynamic` parameter controls whether new fields are added dynamically, and
85+
accepts the following parameters:
86+
87+
[horizontal]
88+
`true`:: New fields are added to the mapping (default).
89+
`runtime`:: New fields are added to the mapping as <<runtime,runtime fields>>.
90+
These fields are not indexed, and are loaded from `_source` at query time.
91+
`false`:: New fields are ignored. These fields will not be indexed
92+
or searchable, but will still appear in the `_source` field of returned hits. These fields will not be added
93+
to the mapping, and new fields must be added explicitly.
94+
`strict`:: If new fields are detected, an exception is thrown and the document
95+
is rejected. New fields must be explicitly added to the mapping.

docs/reference/mapping/runtime.asciidoc

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,10 @@ add fields to existing documents. With _runtime fields_, you can add
66
fields to documents already indexed to {es} without reindexing your data.
77

88
You access runtime fields from the search API like any other field, and {es}
9-
sees runtime fields no differently.
9+
sees runtime fields no differently. You can define runtime fields in the
10+
<<runtime-mapping-fields,index mapping>> or in the
11+
<<runtime-search-request,search request>>. Your choice, which is part of the
12+
inherent flexibility of runtime fields.
1013

1114
[discrete]
1215
[[runtime-benefits]]
@@ -83,6 +86,10 @@ script. If you define a runtime field without a script, {es} evaluates the
8386
field at search time, looks at each document containing that field, retrieves
8487
the `_source`, and returns a value if one exists.
8588

89+
If <<dynamic-field-mapping,dynamic field mapping>> is enabled where the
90+
`dynamic` parameter is set to `runtime`, new fields are automatically added to
91+
the index mapping as runtime fields.
92+
8693
Runtime fields are similar to the <<script-fields,`script_fields`>> parameter
8794
of the `_search` request, but also make the script results available anywhere
8895
in a search request.

0 commit comments

Comments
 (0)