Return _parent value by default for child documents

[honzakral]

When you search or GET to retrieve documents you don't get back the value for _parent field (or _routing for that matter) which is part of the essential metadata. Meaning the document cannot be retrieved or updated without the value. This is breaking some functionality, example use cases:

• reindex (results of scan being fed into bulk) cannot work
• persistence layers on top of elasticsearch cannot update a document they retrieved from ES without additional info/queries
• you cannot GET a document you searched for (for example to retrieve fields you excluded)

I believe it would be worth the cost to retrieve the value of _parent field when retrieving a child document.

[clintongormley]

This should probably be expanded to cover other metadata fields:

• _parent
• _routing
• _timestamp (only retrievable if stored)
• _ttl (?)

[tcucchietti]

👍

I've stumbled across the following case for one of our projects : we needed to retrieve the generated timestamp for indexed document. However, it currently forces us to make another request to the cluster only to retrieve this _timestamp value.

I'm not sure if return it by default is worthy of the extra cost. Maybe using a flag (as for version parameter) to tell ES to include it in the response could do the trick.

However, it would be nice to have it included by default (if stored) when indexing a document for the first time.

[clintongormley]

@tcucchietti you don't need a separate request for this value. Just add ?fields=_timestamp,_source to your search request, and it'll return it to you

[tcucchietti]

@clintongormley Sorry, my comment was maybe misleading.

I was talking about adding it in the indexing request's answer.

For example, this request :
POST index/type { "name":"test" }

currently returns something like :

{
   "_index": "index",
   "_type": "type",
   "_id": "eo3sI8NdS32NE2JUR3HrYA",
   "_version": 1,
   "created": true
}

It would be nice to have this :

{ 
   "_index": "index",
   "_type": "type",
   "_id": "eo3sI8NdS32NE2JUR3HrYA",
   "_timestamp": 1416857203356,
   "_version": 1,
   "created": true
}

[clintongormley]

@tcucchietti Ah ok. Why do you need that? If you are wanting to set the timestamp to now() and you need it on the indexing side, why don't you just pass it as part of the indexing request?

[tcucchietti]

@clintongormley In our case, this is what was finally done and it works perfectly.

As a user, it feeled a bit awkward to have to do this on the indexing side. It's just a personal feeling, that's why I proposed some flag or configuration to achieve this.

Besides this, there could be cases where you need the exact timestamp of the document indexing in the cluster. For example, if you prepare a big bulk indexing request, setting the timestamp value while preparing it could lead to some (minor) difference between this preset value and the exact value of the indexing time.

[colings86]

+1 for returning any stored fields that are not derived from the _source field

[clintongormley]

Fixed by #11816