Skip to content

Move runtime fields to server #69223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 19 commits into from
Feb 22, 2021

Conversation

javanna
Copy link
Member

@javanna javanna commented Feb 18, 2021

This PR makes a start on moving runtime fields to server.

The runtime field types are now built-in. The dynamic fields builder extension (needed to make dynamic:runtime) work is removed: further simplifications are needed but are left for a follow-up. It is still possible to plug in custom runtime field types through the same extension point that the xpack plugin used to use.

The runtime fields script contexts are now also moved to server, and the corresponding whitelists are part of painless-lang with the exception of grok and dissect which will require additional work.

The runtime fields xpack plugin still exists to hold the grok and dissect extensions needed, which depend on grok and dissect lib and we will address as a follow-up. Also, the qa tests and yaml tests are yet to be moved.

Despite the need for additional work, runtime fields are fully functional and users should not be impacted by this change.

@javanna javanna added :Search/Search Search-related issues that do not fall into other categories >refactoring v8.0.0 v7.13.0 labels Feb 18, 2021
@elasticmachine elasticmachine added the Team:Search Meta label for search team label Feb 18, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

Copy link
Member

@nik9000 nik9000 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I scanned it and it looks like a good step! I'm not 10% sure what we want the grok and dissect extension to be in the end, but this does seem useful. And it is nice to zap the weird extension to the dynamic: runtime.

@@ -259,7 +259,7 @@ public String getType() {

@Override
public Set<ScriptContext<?>> getSupportedContexts() {
return Set.of(DoubleFieldScript.CONTEXT);
return Set.of();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm kind of surprised this passes without this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I poked a bit with this and I think this is because all the runtime fields contexts are now built-in and applied by ScriptModule by default.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird! I thought the scripting infra wouldn't send a Script to an engine that didn't support the context. But it looks like that isn't how it works. It more like script engines can make new contexts with this thing.

Copy link
Contributor

@romseygeek romseygeek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! I can already see some common interfaces to extract for calculated fields here as well, looking good.

@@ -219,10 +219,6 @@ protected static boolean parseObjectOrDocumentTypeProperties(String fieldName, O
if (value.equalsIgnoreCase("strict")) {
builder.dynamic(Dynamic.STRICT);
} else if (value.equalsIgnoreCase("runtime")) {
if (parserContext.supportsDynamicRuntimeMappings() == false) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So good to see this gone!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes that is what I thought too. There is more to do to simplify the abstractions here, but removing the extension point and some of the shenanigans around it feels good.

Copy link
Contributor

@jdconrad jdconrad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Painless side LGTM! Thanks for changing this.

@javanna javanna merged commit 1e61059 into elastic:master Feb 22, 2021
javanna added a commit to javanna/elasticsearch that referenced this pull request Feb 22, 2021
This commit makes a start on moving runtime fields to server.

The runtime field types are now built-in. The dynamic fields builder extension (needed to make dynamic:runtime work) is removed: further simplifications are needed but are left for a follow-up. It is still possible to plug in custom runtime field types through the same extension point that the xpack plugin used to use.

The runtime fields script contexts are now also moved to server, and the corresponding whitelists are part of painless-lang with the exception of grok and dissect which will require additional work.

The runtime fields xpack plugin still exists to hold the grok and dissect extensions needed, which depend on grok and dissect lib and we will address as a follow-up. Also, the qa tests and yaml tests are yet to be moved.

Despite the need for additional work, runtime fields are fully functional and users should not be impacted by this change.
javanna added a commit that referenced this pull request Feb 22, 2021
This commit makes a start on moving runtime fields to server.

The runtime field types are now built-in. The dynamic fields builder extension (needed to make dynamic:runtime work) is removed: further simplifications are needed but are left for a follow-up. It is still possible to plug in custom runtime field types through the same extension point that the xpack plugin used to use.

The runtime fields script contexts are now also moved to server, and the corresponding whitelists are part of painless-lang with the exception of grok and dissect which will require additional work.

The runtime fields xpack plugin still exists to hold the grok and dissect extensions needed, which depend on grok and dissect lib and we will address as a follow-up. Also, the qa tests and yaml tests are yet to be moved.

Despite the need for additional work, runtime fields are fully functional and users should not be impacted by this change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>refactoring :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team v7.13.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants