Skip to content

Commit 9317815

Browse files
jtibshiranijtibshirani
committed
Add docs for the fields retrieval API.
1 parent e94971e commit 9317815

File tree

1 file changed

+184
-77
lines changed

1 file changed

+184
-77
lines changed

‎docs/reference/search/search-fields.asciidoc

+184-77
Original file line numberDiff line numberDiff line change
@@ -4,33 +4,203 @@
44

55
By default, each hit in the search response includes the document
66
<<mapping-source-field,`_source`>>, which is the entire JSON object that was
7-
provided when indexing the document. If you only need certain source fields in
8-
the search response, you can use the <<source-filtering,source filtering>> to
9-
restrict what parts of the source are returned.
7+
provided when indexing the document. To retrieve specific fields in the search
8+
response, you can use the `fields` parameter:
109

11-
Returning fields using only the document source has some limitations:
10+
[source,console]
11+
----
12+
GET /_search
13+
{
14+
"query": {
15+
"term": {
16+
"user.id": "8a4f500d"
17+
}
18+
},
19+
"fields": ["user.name", "timestamp"],
20+
"_source": false
21+
}
22+
----
1223

13-
* The `_source` field does not include <<multi-fields, multi-fields>> or
14-
<<alias, field aliases>>. Likewise, a field in the source does not contain
15-
values copied using the <<copy-to,`copy_to`>> mapping parameter.
16-
* Since the `_source` is stored as a single field in Lucene, the whole source
17-
object must be loaded and parsed, even if only a small number of fields are
18-
needed.
24+
The `fields` parameter consults both a document's `_source` and the index
25+
mappings to load and return values. Because it makes use of the mappings,
26+
`fields` has some advantages over referencing the `_source` directly: it
27+
accepts <<multi-fields, multi-fields>> and <<alias, field aliases>>, and
28+
also formats field values like dates in a consistent way.
1929

20-
To avoid these limitations, you can:
30+
A document's `_source` is stored as a single field in Lucene. So the whole
31+
`_source` object must be loaded and parsed even if only a small number of
32+
fields are requested. To avoid this limitation, you can try another option for
33+
loading fields:
2134

2235
* Use the <<docvalue-fields, `docvalue_fields`>>
2336
parameter to get values for selected fields. This can be a good
2437
choice when returning a fairly small number of fields that support doc values,
2538
such as keywords and dates.
26-
* Use the <<request-body-search-stored-fields, `stored_fields`>> parameter to get the values for specific stored fields. (Fields that use the <<mapping-store,`store`>> mapping option.)
39+
* Use the <<request-body-search-stored-fields, `stored_fields`>> parameter to
40+
get the values for specific stored fields (fields that use the
41+
<<mapping-store,`store`>> mapping option).
2742

28-
You can find more detailed information on each of these methods in the
43+
You can find more detailed information on each of these methods in the
2944
following sections:
3045

31-
* <<source-filtering>>
46+
* <<search-fields-param>>
3247
* <<docvalue-fields>>
3348
* <<stored-fields>>
49+
* <<source-filtering>>
50+
51+
[discrete]
52+
[[search-fields-param]]
53+
=== Fields
54+
55+
The `fields` parameter allows for retrieving a list of document fields in
56+
the search response. It consults both the document `_source` and the index
57+
mappings to return each value in a standardized way that matches its mapping
58+
type. By default, date fields are formatted according to the
59+
<<mapping-date-format,date format>> parameter in their mappings.
60+
61+
.*Example*
62+
[%collapsible]
63+
====
64+
The following search request uses the `fields` parameter to retrieve values
65+
for the `clientip` field, all fields starting with `location.`, and the
66+
`timestamp` field:
67+
68+
[source,console]
69+
----
70+
POST logs-*/_search
71+
{
72+
"query": {
73+
"match_all": {}
74+
},
75+
"fields": [
76+
"clientip",
77+
"location.*", <1>
78+
{
79+
"field": "timestamp",
80+
"format": "epoch_millis" <2>
81+
}
82+
],
83+
"_source": false
84+
}
85+
----
86+
87+
<1> Both full field names and wildcard patterns are accepted.
88+
<2> Using object notation, you can pass a `format` parameter to apply a custom
89+
format for the field's values. This is currently supported for
90+
<<date,`date` fields>> and <<date_nanos, `date_nanos` fields>>, which
91+
accept a <<mapping-date-format,date format>>.
92+
93+
The values are returned as a flat list in the `fields` section in each hit:
94+
95+
[source,console-result]
96+
----
97+
{
98+
"took" : 2,
99+
"timed_out" : false,
100+
"_shards" : {
101+
"total" : 1,
102+
"successful" : 1,
103+
"skipped" : 0,
104+
"failed" : 0
105+
},
106+
"hits" : {
107+
"total" : {
108+
"value" : 1,
109+
"relation" : "eq"
110+
},
111+
"max_score" : 1.0,
112+
"hits" : [
113+
{
114+
"_index" : "http-logs",
115+
"_id" : "1",
116+
"_score" : 1.0,
117+
"fields" : {
118+
"clientip" : [
119+
"192.0.2.0"
120+
],
121+
"location.city" : [
122+
"Toronto"
123+
],
124+
"location.country" : [
125+
"Canada"
126+
],
127+
"timestamp" : [
128+
"1593274413000"
129+
]
130+
}
131+
}
132+
]
133+
}
134+
}
135+
----
136+
// TESTRESPONSE[skip:no test set-up]
137+
138+
Only leaf fields are returned -- `fields` does not allow for fetching entire
139+
objects.
140+
141+
====
142+
143+
The `fields` parameter handles field types like <<alias, field aliases>> and
144+
<<constant-keyword, `constant_keyword`>> whose values aren't always present in
145+
the `_source`. Other mapping options are also respected, including
146+
<<ignore-above, `ignore_above`>>, <<ignore-malformed, `ignore_malformed`>> and
147+
<<null-value, `null_value`>>.
148+
149+
[discrete]
150+
[[docvalue-fields]]
151+
=== Doc value fields
152+
153+
You can use the <<docvalue-fields,`docvalue_fields`>> parameter to return
154+
<<doc-values,doc values>> for one or more fields in the search response.
155+
156+
Doc values store the same values as the `_source` but in an on-disk,
157+
column-based structure that's optimized for sorting and aggregations. Since each
158+
field is stored separately, {es} only reads the field values that were requested
159+
and can avoid loading the whole document `_source`.
160+
161+
Doc values are stored for supported fields by default. However, doc values are
162+
not supported for <<text,`text`>> or
163+
{plugins}/mapper-annotated-text-usage.html[`text_annotated`] fields.
164+
165+
.*Example*
166+
[%collapsible]
167+
====
168+
The following search request uses the `docvalue_fields` parameter to retrieve
169+
doc values for the `clientip` field, all fields starting with `location.`, and the
170+
`timestamp` field:
171+
172+
173+
[source,console]
174+
----
175+
GET /_search
176+
{
177+
"query": {
178+
"match_all": {}
179+
},
180+
"docvalue_fields": [
181+
"clientip", <1>
182+
"location.*",
183+
{
184+
"field": "timestamp",
185+
"format": "epoch_millis" <2>
186+
}
187+
]
188+
}
189+
----
190+
191+
<1> Both full field names and wildcard patterns are accepted.
192+
<2> Using object notation, you can pass a `format` parameter to apply a custom
193+
format for the field's doc values. <<date,Date fields>> support a
194+
<<mapping-date-format,date `format`>>. <<number,Numeric fields>> support a
195+
https://docs.oracle.com/javase/8/docs/api/java/text/DecimalFormat.html[DecimalFormat
196+
pattern]. Other field datatypes do not support the `format` parameter.
197+
====
198+
199+
TIP: You cannot use the `docvalue_fields` parameter to retrieve doc values for
200+
nested objects. If you specify a nested object, the search returns an empty
201+
array (`[ ]`) for the field. To access nested fields, use the
202+
<<request-body-search-inner-hits, `inner_hits`>> parameter's `docvalue_fields`
203+
property.
34204

35205
[discrete]
36206
[[source-filtering]]
@@ -122,69 +292,6 @@ GET /_search
122292
----
123293
====
124294

125-
126-
[discrete]
127-
[[docvalue-fields]]
128-
=== Doc value fields
129-
130-
You can use the <<docvalue-fields,`docvalue_fields`>> parameter to return
131-
<<doc-values,doc values>> for one or more fields in the search response.
132-
133-
Doc values store the same values as the `_source` but in an on-disk,
134-
column-based structure that's optimized for sorting and aggregations. Since each
135-
field is stored separately, {es} only reads the field values that were requested
136-
and can avoid loading the whole document `_source`.
137-
138-
Doc values are stored for supported fields by default. However, doc values are
139-
not supported for <<text,`text`>> or
140-
{plugins}/mapper-annotated-text-usage.html[`text_annotated`] fields.
141-
142-
.*Example*
143-
[%collapsible]
144-
====
145-
The following search request uses the `docvalue_fields` parameter to
146-
retrieve doc values for the following fields:
147-
148-
* Fields with names starting with `my_ip`
149-
* `my_keyword_field`
150-
* Fields with names ending with `_date_field`
151-
152-
[source,console]
153-
----
154-
GET /_search
155-
{
156-
"query": {
157-
"match_all": {}
158-
},
159-
"docvalue_fields": [
160-
"my_ip*", <1>
161-
{
162-
"field": "my_keyword_field" <2>
163-
},
164-
{
165-
"field": "*_date_field",
166-
"format": "epoch_millis" <3>
167-
}
168-
]
169-
}
170-
----
171-
172-
<1> Wildcard patten used to match field names, specified as a string.
173-
<2> Wildcard patten used to match field names, specified as an object.
174-
<3> With the object notation, you can use the `format` parameter to specify a
175-
format for the field's returned doc values. <<date,Date fields>> support a
176-
<<mapping-date-format,date `format`>>. <<number,Numeric fields>> support a
177-
https://docs.oracle.com/javase/8/docs/api/java/text/DecimalFormat.html[DecimalFormat
178-
pattern]. Other field datatypes do not support the `format` parameter.
179-
====
180-
181-
TIP: You cannot use the `docvalue_fields` parameter to retrieve doc values for
182-
nested objects. If you specify a nested object, the search returns an empty
183-
array (`[ ]`) for the field. To access nested fields, use the
184-
<<request-body-search-inner-hits, `inner_hits`>> parameter's `docvalue_fields`
185-
property.
186-
187-
188295
[discrete]
189296
[[stored-fields]]
190297
=== Stored fields

0 commit comments

Comments
 (0)