StringMatcherTests.testUnicodeWildcard test failure

[talevy]

There is a reproducible test failure for the testUnicodeWildcard test in security.

Fails on: master/v8.0

reproduce with:

./gradlew ':x-pack:plugin:core:test' --tests "org.elasticsearch.xpack.core.security.support.StringMatcherTests.testUnicodeWildcard" -Dtests.seed=CA428535A844FAB5 -Dtests.security.manager=true -Dtests.locale=de-CH -Dtests.timezone=Pacific/Chuuk -Druntime.java=11

Exception:

org.elasticsearch.xpack.core.security.support.StringMatcherTests > testUnicodeWildcard FAILED
    java.lang.AssertionError: Matcher [o\a*] failed to match [o\a෣එඒ඿෍ඪa] but should
        at __randomizedtesting.SeedInfo.seed([CA428535A844FAB5:4987B18C23E0F50F]:0)
        at org.junit.Assert.fail(Assert.java:88)
        at org.elasticsearch.xpack.core.security.support.StringMatcherTests.assertMatch(StringMatcherTests.java:141)
        at org.elasticsearch.xpack.core.security.support.StringMatcherTests.testUnicodeWildcard(StringMatcherTests.java:52)

[elasticmachine]

Pinging @elastic/es-security (Team:Security)

[ywangd]

TIL: The automaton building logic treats \ as the escaping char and add the following character if there is any. This means the string "o\a*" is really "oa*" for the automaton.

elasticsearch/x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/support/Automatons.java

Lines 235 to 242 in 5659b4c

	case WILDCARD_ESCAPE:
	// add the next codepoint instead, if it exists
	if (i + length < text.length()) {
	final char nextChar = text.charAt(i + length);
	length += 1;
	automata.add(Automata.makeChar(nextChar));
	break;
	} // else fallthru, lenient parsing with a trailing \

[hendrikmuhs]

I have another one and as this issue is just 3 days old I allow myself to be lazy and re-open it:

./gradlew ':x-pack:plugin:core:test' --tests "org.elasticsearch.xpack.core.security.support.StringMatcherTests.testUnicodeWildcard" -Dtests.seed=F4A452F494D1795C -Dtests.security.manager=true -Dtests.locale=sr-Latn-BA -Dtests.timezone=America/New_York -Druntime.java=11

fails reproducibly with:

rk -Druntime.java=11
  2> java.lang.IllegalArgumentException: invalid pattern [/:a*]. patterns starting with '/' indicate regular expression pattern and therefore must also end with '/'. other patterns (those that do not start with '/') will be treated as simple wildcard patterns
        at __randomizedtesting.SeedInfo.seed([F4A452F494D1795C:7761664D1F7576E6]:0)
        at org.elasticsearch.xpack.core.security.support.Automatons.buildAutomaton(Automatons.java:197)
        at org.elasticsearch.xpack.core.security.support.Automatons.lambda$pattern$2(Automatons.java:180)
        at org.elasticsearch.common.cache.Cache.computeIfAbsent(Cache.java:424)
        at org.elasticsearch.xpack.core.security.support.Automatons.pattern(Automatons.java:180)
        at org.elasticsearch.xpack.core.security.support.Automatons.buildAutomaton(Automatons.java:93)
        at org.elasticsearch.xpack.core.security.support.Automatons.lambda$patterns$0(Automatons.java:84)
        at org.elasticsearch.common.cache.Cache.computeIfAbsent(Cache.java:424)
        at org.elasticsearch.xpack.core.security.support.Automatons.patterns(Automatons.java:84)
        at org.elasticsearch.xpack.core.security.support.Automatons.predicate(Automatons.java:275)
        at org.elasticsearch.xpack.core.security.support.StringMatcher$Builder.buildAutomataPredicate(StringMatcher.java:159)
        at org.elasticsearch.xpack.core.security.support.StringMatcher$Builder.build(StringMatcher.java:122)
        at org.elasticsearch.xpack.core.security.support.StringMatcher.of(StringMatcher.java:53)
        at org.elasticsearch.xpack.core.security.support.StringMatcherTests.testUnicodeWildcard(StringMatcherTests.java:50)

[ywangd]

Thanks @hendrikmuhs It is a legit failure. I raised #70010 Thanks for bringing this to our attention.