Skip to content

[Aggregations] Added _meta construct to inject metadata #6465

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

[Aggregations] Added _meta construct to inject metadata #6465

wants to merge 3 commits into from

Conversation

Mpdreamz
Copy link
Member

This PR adds the ability to associate a bit of state with each individual aggregation.

Use cases:

  • Typed clients can inject some state so that the deserialization
    process has enough information to deserialize the aggregations into
    the correct type. Facets return _type to signal what kind of facet the
    response corresponds with.
  • UI state, since aggregations can be nested at arbitrary depths
    injecting pieces of UI state can greatly simplify programming with
    aggregations since there are no two tree data-structures to walk anymore only the aggregation data.

thank you @martijnvg for the help reading the byte[] representation of _meta properly

  • this PR also fixes the response format of the new geo_bounds aggregation to wrap the result in a "aggregationname" : {}

Here's a gist of a sense session you can use to review this branch

Example request:

{
"aggs": {
    "name": {
      "terms": {
        "field": "title"
      },
      "meta": 2,
      "aggs": {
        "viewport" : {
          "meta" : {
               "complex" : "object"
          },
          "geo_bounds" : {
            "field" : "location"
          }
        },
        "empty_agg" : {
          "meta" : "should still be returned",
          "sum": {
            "field": "i_do_not_exist"
          }
        },

Example response:

{
   "aggregations": {
      "name": {
         "meta": 2,
         "buckets": [
            {
               "key": "banner",
               "doc_count": 1,
               "viewport": {
                  "meta": {
                    "complex": "object"
                  },
                  "bounds": {
                     "top_left": {
                        "lat": 28,
                        "lon": 21
                     },
                     "bottom_right": {
                        "lat": 28,
                        "lon": 21
                     }
                  }
               },
               "nested_terms": {
                  "meta": {
                     "complex": "object",
                     "nuber": 1
                  },
                  "buckets": [
                     {
                        "key": "a",
                        "doc_count": 1,
                        "counter": {
                           "meta": 4.1,
                           "value": 40
                        }
                     },

This PR adds the abillity to associate a bit of state with each
individual aggregation.

Usecases:
- Typed clients can inject some state so that the deserialization
  process has enough information to deserialize the aggregations into
the correct type. Facets return `_type` to signal what kind of facet the
response corresponds with.
- UI state, since aggregations can be nested at arbitraty depths
  injecting pieces of UI state can greatly simplify programmming with
aggregations since there are no two tree datastructures to walk anymore
only the aggregation data.
Mpdreamz added a commit to elastic/elasticsearch-net that referenced this pull request Jun 11, 2014
…earch#6465 is good enough to be merged starting from Elasticsearch 1.3 we can use _meta to greatly simplify parsing
"terms": {
"field": "title"
},
"_meta": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's get rid of the underscore... we moved away from that (no real need for it as all the keys under the agg are controlled by us)

@Mpdreamz
Copy link
Member Author

Thanks for review @uboness ! I wrapped the create() because I wasn't sure if updating all the Aggregation implementation constructors would break elasticsearch's public API.

I opted for _meta because adding it in the response might break existing code that uses meta as aggregation name.

Will update both property names to meta and pass metaData in the constructors of Aggregation implementations.

@uboness
Copy link
Contributor

uboness commented Jun 12, 2014

why would it break? it's under the agg name, isn't it? in any case... still... not good enough reason to introduce inconsistency in the api. That is, if we feel that something will break apis, better break them and target for a later release (one that enables breaking changes) rather than introduce inconsistency in the api (something we'll need to cary later for a long time)

…structor of Aggregations no more buildInternals*
@Mpdreamz
Copy link
Member Author

@uboness I've updated the PR with all your feedback I only have one question regarding the AggregationBuilder.java changes.

builder.field(CommonFields.META);
builder.copyCurrentStructure(parser);
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the way that we usually do this is by making toXContent final, write the name and metadata, and delegate to a doXContentBody method. This ensures that no aggregation can by-pass the serialization of the metadata.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@clintongormley
Copy link
Contributor

@Mpdreamz where are you on this one?

@Mpdreamz
Copy link
Member Author

closing in favour of #8279

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants