Skip to content

Investigate improvements to all pattern matching #36062

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jaymode opened this issue Nov 29, 2018 · 2 comments · Fixed by #66724
Closed

Investigate improvements to all pattern matching #36062

jaymode opened this issue Nov 29, 2018 · 2 comments · Fixed by #66724
Assignees
Labels
>enhancement :Security/Authorization Roles, Privileges, DLS/FLS, RBAC/ABAC Team:Security Meta label for security team

Comments

@jaymode
Copy link
Member

jaymode commented Nov 29, 2018

In #36017, a targeted improvement is made to the pattern matching for indices permissions to improve the exact match performance and building of permissions. @tvernum suggested that we can extract this logic into a Patterns class and update other places where pattern matching is done. Also, we can investigate the performance of other methods such as String#startsWith and String#endsWith in certain cases in place of a Automaton.

@jaymode jaymode added >enhancement :Security/Authorization Roles, Privileges, DLS/FLS, RBAC/ABAC labels Nov 29, 2018
@jaymode jaymode self-assigned this Nov 29, 2018
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-security

@jaymode
Copy link
Member Author

jaymode commented Mar 8, 2019

This investigation should also consider an observation from @tvernum

The way we compile automatons has certain quirks that mean, for a given pattern, it is more efficient to express it as a regular expression rather than a simple wildcard.

This means that something like logstash-*-bar-* might be better represented as /logstash-@-bar-@/ in terms of the number of states in the Automaton, which has an effect on the determinization and minimization operations.

Additionally, one item to consider is the fact that certain patterns may overlap; logstash-* encompasses logstash-bar and we can see if detecting this can also improve performance during construction of automata.

@jaymode jaymode removed their assignment Apr 19, 2019
@tvernum tvernum self-assigned this Jan 8, 2020
@rjernst rjernst added the Team:Security Meta label for security team label May 4, 2020
tvernum added a commit to tvernum/elasticsearch that referenced this issue Nov 24, 2020
This commit moves the implementation of wildcard pattern matching into
a standalone utility class ("StringMatcher").

In general, we rely on lucene Automaton objects to implement pattern
matching (wildcards and regexp) within Elasticsearch security - for
example in Index name patterns within a role.

The IndicesPermission class also has a special optimisation for exact
string matches (that is raw index names that contain no wildcards) as
using String.equals / Set.contains is more efficient for this common
case.

All of the above functionality has now been extracted into the
StringMatcher class, and it is now used in several places where it may
be more efficient that the previous use of raw Automaton objects.

A future change will expand this StringMatcher class with additional
optimisations for common use cases that are poorly handled within our
existing automaton compilation process.

Relates: elastic#36062
tvernum added a commit that referenced this issue Dec 21, 2020
This commit moves the implementation of wildcard pattern matching into
a standalone utility class ("StringMatcher").

In general, we rely on lucene Automaton objects to implement pattern
matching (wildcards and regexp) within Elasticsearch security - for
example in Index name patterns within a role.

The IndicesPermission class also has a special optimisation for exact
string matches (that is raw index names that contain no wildcards) as
using String.equals / Set.contains is more efficient for this common
case.

All of the above functionality has now been extracted into the
StringMatcher class, and it is now used in several places where it may
be more efficient that the previous use of raw Automaton objects.

A future change will expand this StringMatcher class with additional
optimisations for common use cases that are poorly handled within our
existing automaton compilation process.

Relates: #36062
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>enhancement :Security/Authorization Roles, Privileges, DLS/FLS, RBAC/ABAC Team:Security Meta label for security team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants