Skip to content

Allow fields calculated by scripting at index time #68984

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
9 tasks done
romseygeek opened this issue Feb 15, 2021 · 1 comment
Closed
9 tasks done

Allow fields calculated by scripting at index time #68984

romseygeek opened this issue Feb 15, 2021 · 1 comment
Labels
>enhancement Meta release highlight :Search Foundations/Mapping Index mappings, including merging and defining field types Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch v7.13.0

Comments

@romseygeek
Copy link
Contributor

romseygeek commented Feb 15, 2021

Runtime fields allow you to create a synthetic field derived from the values of other document properties at query time, which prioritises flexibility (you can change these fields at any time) over speed. This issue will address generating these synthetic fields at index time, trading off some of that flexibility for better query performance.

There should be as little impedance mismatch as possible between the scripts used for runtime fields and those used for index time calculations. Ideally, making a runtime field indexed should be as simple as moving the field definition from the runtime section of mappings into the properties section, eg:

"mappings" : {
  "runtime" : {
    "my_field" : {
      "type" : "long",
      "script" : " ... some script ... "
    }
  }
}

becomes:

"mappings" : {
  "properties" : {
    "my_field" : {
      "type" : "long",
      "script" : " ... some script ... "
    }
  }
}

In particular, we will need to reproduce the typed emit functions currently used by runtime scripts, and make _doc, _source and _fields available in the same way (although _fields is probably not a priority here as it essentially duplicates _source at index time).

Now that runtime fields have been moved to core, we can re-use their script factory implementations by mocking a lucene LeafReader over the contents of the parsed lucene Document that is to be indexed.

The steps to take are anticipated to be:

@romseygeek romseygeek added >enhancement :Search Foundations/Mapping Index mappings, including merging and defining field types Meta labels Feb 15, 2021
@elasticmachine elasticmachine added the Team:Search Meta label for search team label Feb 15, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

javanna added a commit to javanna/elasticsearch that referenced this issue Apr 7, 2021
javanna added a commit to javanna/elasticsearch that referenced this issue Apr 12, 2021
@javanna javanna added Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch and removed Team:Search Meta label for search team labels Jul 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement Meta release highlight :Search Foundations/Mapping Index mappings, including merging and defining field types Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch v7.13.0
Projects
None yet
Development

No branches or pull requests

3 participants