Skip to content

EQL: Optimization for string with single wildcard #62585

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
rw-access opened this issue Sep 17, 2020 · 2 comments · Fixed by #62682
Closed

EQL: Optimization for string with single wildcard #62585

rw-access opened this issue Sep 17, 2020 · 2 comments · Fixed by #62682
Labels
:Analytics/EQL EQL querying >enhancement Team:QL (Deprecated) Meta label for query languages team

Comments

@rw-access
Copy link
Contributor

rw-access commented Sep 17, 2020

I saw a few queries flying around with field == "*" syntax. In general, I think we should direct users to direct == null or != null checking, but sometimes we see usage patterns from KQL seep in.

Under the hood field == "*" gets converted to wildcard(field, "*"), which eventually turns into an AST for "like": Like(Source(), Field("field"), "%").

I think somewhere in the optimizer to we should convert this to IsNotNull which should be more performant.

@rw-access rw-access added >enhancement :Analytics/EQL EQL querying Team:QL (Deprecated) Meta label for query languages team labels Sep 17, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-ql (:Query Languages/EQL)

@jimczi
Copy link
Contributor

jimczi commented Sep 18, 2020

We have some optimizations internally when parsing a search request that rewrites a query like field:* into an exists query for the field. However we don't apply this optimization to wildcard and prefix queries, only in query_string.
So I don't know what is the step to move from wildcard(field, "*") to Like(Source(), Field("field"), "%") but for the search layer, we should be able to optimize the first case, wildcard(field, "*").

costin added a commit to costin/elasticsearch that referenced this issue Sep 21, 2020
Replace common Like and RLike queries that match all characters with
IsNotNull (exists) queries

Fix elastic#62585
costin added a commit that referenced this issue Sep 24, 2020
Replace common Like and RLike queries that match all characters with
IsNotNull (exists) queries

Fix #62585
costin added a commit that referenced this issue Sep 24, 2020
Replace common Like and RLike queries that match all characters with
IsNotNull (exists) queries

Fix #62585

(cherry picked from commit 4c23fad)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/EQL EQL querying >enhancement Team:QL (Deprecated) Meta label for query languages team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants