You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
* Changes for #52239.
* Incorporating review feedback from Julie T. Also single-sourcing nexted options in the Mapping page and referencing them in the Nested page.
* Moving tip after the introduction and clarifying limits.
* Update docs/reference/mapping.asciidoc
Co-authored-by: James Rodewig <[email protected]>
* Update docs/reference/mapping/types/nested.asciidoc
Co-authored-by: James Rodewig <[email protected]>
Co-authored-by: James Rodewig <[email protected]>
TIP: Using the KV Processor can result in field names that you cannot control. Consider using the <<flattened>> datatype instead, which maps an entire object as a single field and allows for simple searches over its contents.
Copy file name to clipboardExpand all lines: docs/reference/mapping.asciidoc
+27-17Lines changed: 27 additions & 17 deletions
Original file line number
Diff line number
Diff line change
@@ -17,7 +17,7 @@ A mapping definition has:
17
17
18
18
<<mapping-fields,Meta-fields>>::
19
19
20
-
Meta-fields are used to customize how a document's metadata associated is
20
+
Meta-fields are used to customize how a document's associated metadata is
21
21
treated. Examples of meta-fields include the document's
22
22
<<mapping-index-field,`_index`>>, <<mapping-id-field,`_id`>>, and
23
23
<<mapping-source-field,`_source`>> fields.
@@ -58,17 +58,16 @@ via the <<multi-fields>> parameter.
58
58
[float]
59
59
=== Settings to prevent mappings explosion
60
60
61
-
Defining too many fields in an index is a condition that can lead to a
61
+
Defining too many fields in an index can lead to a
62
62
mapping explosion, which can cause out of memory errors and difficult
63
-
situations to recover from. This problem may be more common than expected.
64
-
As an example, consider a situation in which every new document inserted
65
-
introduces new fields. This is quite common with dynamic mappings.
66
-
Every time a document contains new fields, those will end up in the index's
67
-
mappings. This isn't worrying for a small amount of data, but it can become a
63
+
situations to recover from.
64
+
65
+
Consider a situation where every new document inserted
66
+
introduces new fields, such as with <<dynamic-mapping,dynamic mapping>>.
67
+
Each new field is added to the index mapping, which can become a
68
68
problem as the mapping grows.
69
-
The following settings allow you to limit the number of field mappings that
70
-
can be created manually or dynamically, in order to prevent bad documents from
71
-
causing a mapping explosion:
69
+
70
+
Use the following settings to limit the number of field mappings (created manually or dynamically) and prevent documents from causing a mapping explosion:
72
71
73
72
`index.mapping.total_fields.limit`::
74
73
The maximum number of fields in an index. Field and object mappings, as well as
@@ -84,26 +83,37 @@ If you increase this setting, we recommend you also increase the
84
83
<<search-settings,`indices.query.bool.max_clause_count`>> setting, which
85
84
limits the maximum number of <<query-dsl-bool-query,boolean clauses>> in a query.
86
85
====
86
+
+
87
+
[TIP]
88
+
====
89
+
If your field mappings contain a large, arbitrary set of keys, consider using the <<flattened,flattened>> datatype.
90
+
====
87
91
88
92
`index.mapping.depth.limit`::
89
93
The maximum depth for a field, which is measured as the number of inner
90
94
objects. For instance, if all fields are defined at the root object level,
91
95
then the depth is `1`. If there is one object mapping, then the depth is
92
-
`2`, etc. The default is `20`.
96
+
`2`, etc. Default is `20`.
93
97
98
+
// tag::nested-fields-limit[]
94
99
`index.mapping.nested_fields.limit`::
95
-
The maximum number of distinct `nested` mappings in an index, defaults to `50`.
100
+
The maximum number of distinct `nested` mappings in an index. The `nested` type should only be used in special cases, when arrays of objects need to be queried independently of each other. To safeguard against poorly designed mappings, this setting
101
+
limits the number of unique `nested` types per index. Default is `50`.
102
+
// end::nested-fields-limit[]
96
103
104
+
// tag::nested-objects-limit[]
97
105
`index.mapping.nested_objects.limit`::
98
-
The maximum number of `nested` JSON objects within a single document across
99
-
all nested types, defaults to 10000.
106
+
The maximum number of nested JSON objects that a single document can contain across all
107
+
`nested` types. This limit helps to prevent out of memory errors when a document contains too many nested
108
+
objects. Default is `10000`.
109
+
// end::nested-objects-limit[]
100
110
101
111
`index.mapping.field_name_length.limit`::
102
-
Setting for the maximum length of a field name. The default value is
103
-
Long.MAX_VALUE (no limit). This setting isn't really something that addresses
112
+
Setting for the maximum length of a field name. This setting isn't really something that addresses
104
113
mappings explosion but might still be useful if you want to limit the field length.
105
114
It usually shouldn't be necessary to set this setting. The default is okay
106
-
unless a user starts to add a huge number of fields with really long names.
115
+
unless a user starts to add a huge number of fields with really long names. Default is
Copy file name to clipboardExpand all lines: docs/reference/mapping/types/nested.asciidoc
+25-27Lines changed: 25 additions & 27 deletions
Original file line number
Diff line number
Diff line change
@@ -5,14 +5,17 @@
5
5
++++
6
6
7
7
The `nested` type is a specialised version of the <<object,`object`>> datatype
8
-
that allows arrays of objects to be indexed in a way that they can be queried
8
+
that allows arrays of objects to be indexed in a way that they can be queried
9
9
independently of each other.
10
10
11
+
TIP: When ingesting key-value pairs with a large, arbitrary set of keys, you might consider modeling each key-value pair as its own nested document with `key` and `value` fields. Instead, consider using the <<flattened,flattened>> datatype, which maps an entire object as a single field and allows for simple searches over its contents.
12
+
Nested documents and queries are typically expensive, so using the `flattened` datatype for this use case is a better option.
13
+
14
+
[[nested-arrays-flattening-objects]]
11
15
==== How arrays of objects are flattened
12
16
13
-
Arrays of inner <<object,`object` fields>> do not work the way you may expect.
14
-
Lucene has no concept of inner objects, so Elasticsearch flattens object
15
-
hierarchies into a simple list of field names and values. For instance, the
17
+
Elasticsearch has no concept of inner objects. Therefore, it flattens object
18
+
hierarchies into a simple list of field names and values. For instance, consider the
16
19
following document:
17
20
18
21
[source,console]
@@ -35,7 +38,7 @@ PUT my_index/_doc/1
35
38
36
39
<1> The `user` field is dynamically added as a field of type `object`.
37
40
38
-
would be transformed internally into a document that looks more like this:
41
+
The previous document would be transformed internally into a document that looks more like this:
0 commit comments