JDK16 Triage push Coordination

[adamfarley]

Failing Test Buckets & Triager:

• OpenJDK - Adam Farley
• Functional - Andrew Leonard
• Performance - Simon Rushton
• System - Simon Rushton

Each triager to create a comment identifying failures they're working on, with test links, issue links, platforms, etc.

[lumpfish]

Issues affecting Performance (sanity all pass - these affect extended (i.e. weekly / release)):
#2253
#2254
#2255

[lumpfish]

These are machine specific issues which cause system test failures if a particular test lands on a particular machine:
adoptium/infrastructure#1907
adoptium/infrastructure#1851
adoptium/infrastructure#1777
adoptium/infrastructure#1673

This is an unresolved intermittent issue which occurs rarely:
adoptium/aqa-systemtest#274

[andrew-m-leonard]

extended.functional AIX "Dump" failures:
org.openj9.test.attachAPI.TestJcmd.testDumps
org.openj9.test.attachAPI.TestJcmd.testDumpsDefaultSettings
org.openj9.test.attachAPI.TestJcmd.testDumpsWithOptions
org.openj9.test.java.lang.management.TestOpenJ9DiagnosticsMXBean.testLocal_triggerDump
org.openj9.test.java.lang.management.TestOpenJ9DiagnosticsMXBean.testRemote_triggerDumpToFile
org.openj9.test.java.lang.management.TestOpenJ9DiagnosticsMXBean.testRemote_triggerDump
org.openj9.test.java.lang.management.TestOpenJ9DiagnosticsMXBean.testLocal_triggerDumpToFile

Grinders of TestJcmd_0 on the 4 AIX boxes:
https://ci.adoptopenjdk.net/job/Grinder/6696/
https://ci.adoptopenjdk.net/job/Grinder/6697/
https://ci.adoptopenjdk.net/job/Grinder/6698/
https://ci.adoptopenjdk.net/job/Grinder/6699/
Bad:
build-osuosl-aix71-ppc64-2
Good:
test-ibm-aix71-ppc64-2

[andrew-m-leonard]

extended.functional xLinux testDDRExt_Class_0:

Grinder across 6 nodes:
https://ci.adoptopenjdk.net/job/Grinder/6700/ test-docker-fedora33-x64-3 Failed
https://ci.adoptopenjdk.net/job/Grinder/6701/ test-ibmcloud-ubuntu1604-x64-1 Pass
https://ci.adoptopenjdk.net/job/Grinder/6702/ test-docker-ubuntu1804-x64-1 Failed
https://ci.adoptopenjdk.net/job/Grinder/6703/ test-packet-ubuntu1604-x64-3 Pass
https://ci.adoptopenjdk.net/job/Grinder/6704/ test-docker-ubuntu2010-x64-2 Failed
https://ci.adoptopenjdk.net/job/Grinder/6705/ test-ibmcloud-rhel7-x64-1 Pass

The problem seems related to the "test-docker-" nodes.
Existing issue: eclipse-openj9/openj9#11733

[lumpfish]

Other machine specific test failures already raised for openjdk / functional tests:
adoptium/infrastructure#1282
adoptium/infrastructure#1819
adoptium/infrastructure#1856
adoptium/infrastructure#1859
adoptium/infrastructure#1916
adoptium/infrastructure#1918

[adamfarley]

OpenJDK regression Platform-specific issues:

• aarch64
  • jdk_lang_0 - TestLambdaFileEncodingSerialization failure (Example) - AArch64 openjdk LambdaFileEncodingSerialization assertion failure eclipse-openj9/openj9#11915
  • jdk_util_0 - CountLargeTest.java failure (Example) - Excluded as per OpenJ9 AArch64: excluding stream/CountLargeTest #2243
• arm_linux
  • jdk_lang_2 - LambdaFileEncodingSerialization fails with the same issue mentioned above. (Example)
  • TestJlmRemote* - These all appear to fail due to build-scaleway-ubuntu1604-armv7-1 / 2 : jlm tests fail with Connection refused infrastructure#1907
• x86-32_windows
  • jdk_lang_native_win_2 - CheckHandles.java fails with an UnsatisfiedLinkError (Example) - Looks a lot like jdk_lang_native_win_0 FAILED in Test_openjdk11_hs_sanity.openjdk_x86-32_windows #2029, except where that fails trying to find CheckHandles.getProcessHandleCount(), this one fails to find FileUtils.getWinProcessHandleCount() during an execution of FileUtils.getProcessHandleCount(). Will add a mention in the issue, and add it to my "to-do" list.
• x86-64_linux
  • jdk_lang_0 - VarHandleTestMethodHandleAccessBoolean.java fails with "exit code: 137". No other information. (Example) - Issue raised. - Issue closed, all 100 re-runs passed. Will reopen if seen again.
• x86-64_windows
  • jdk_foreign_0 - TestHandshake.java fails when run with "-XX:-TieredCompilation". (Example) - Grinder running to reproduce.
    Note: That grinder found a different issue (details below), so here's a 100x grinder to identify frequency rates for each.
    Update: Grinder failed 24 times with the below issue, before running out of space to store test output. Since the issue is blocked, and the issue in question is spawned out of a bunch of "in progress" work by openj9, we won't make progress on this until that is resolved, assuming they're not both caused by the same thing. In any case, the issue references excluding the test for now, so the issue is rendered moot.
  • jdk_foreign_0 - TestHandshake.java assertion fails in the j9jcl, referencing jdk_internal_misc_ScopedMemoryAccess.cpp:40. - Looks like this issue.

OpenJDK regression Cross-platform issues

• jdk_foreign_native_0 - Failed to find tests to run (Examples exist on arm_linux, ppc64_aix, ppc64le_linux) - Excluded on J9 due to another issue, but not HS. Issue raised. - JDK16: jdk_foreign_native_# reports no tests selected (Hotspot only) #2265
• jdk11_tier1_cipher_# - Four of these fail with "UnsupportedOperationException: ByteBuffer not supported". (Examples exist on ppc64_aix, ppc64le_linux_xl, x86-32_windows, x86-64_linux, x86-64_mac, x86-64_mac_xl, x86-64_windows) - Excluded three days ago.
• jdk11_tier1_cipher_0 - GCMBufferTest.java failure ((Examples exist on x86-64_linux_xl, aarch64, x86-64_mac, x86-64_mac_xl, x86-64_windows) - Fixed by this PR once it's finished and subsequently backported. Peter Shipton is aware of the need to backport, as per this issue.

[lumpfish]

jdk-16+36_openj9-0.25.0-m2 triage

Failing jobs and their issues:

TRSS ppc64_aix: https://trss.adoptopenjdk.net/resultSummary?parentId=603cc5345730424dbc98db6c

ppc64_aix

Failing job:
https://ci.adoptopenjdk.net/job/Test_openjdk16_j9_sanity.openjdk_ppc64_aix/31/consoleFull
Issues:
eclipse-openj9/openj9#11738
eclipse-openj9/openj9#10990

Failing job:
https://ci.adoptopenjdk.net/job/Test_openjdk16_j9_extended.perf_ppc64_aix/7/console
Issues:
#2253

TRSS Other platforms: https://trss.adoptopenjdk.net/resultSummary?parentId=603cc5345730424dbc98db6e

#2253 affects extended.perf jobs on all platforms

aarch64_linux

Respin required due to missing PR: eclipse-openj9/openj9#11987

ppc64le_linux

Failing job:
https://ci.adoptopenjdk.net/job/Test_openjdk16_j9_sanity.external_ppc64le_linux/4/
Issues:
#2317

s390x_linux

Failing job:
https://ci.adoptopenjdk.net/job/Test_openjdk16_j9_sanity.external_s390x_linux/4/
Issues:
#2317

Failing job:
https://ci.adoptopenjdk.net/job/Test_openjdk16_j9_extended.openjdk_s390x_linux/9/consoleFull
Issues:
adoptium/infrastructure#1992

x86-64_linux

Failing job:
https://ci.adoptopenjdk.net/job/Test_openjdk16_j9_sanity.external_x86-64_linux/4/

Issues:
eclipse-openj9/openj9#12038
eclipse-openj9/openj9#12099

x86-64_mac

Failing job:
https://ci.adoptopenjdk.net/job/Test_openjdk16_j9_extended.openjdk_x86-64_mac/9/consoleFull
Issues:
#2327

[lumpfish]

jdk-16+36_openj9-0.25.0-m3 triage (aarch64_linux)

TRSS: https://trss.adoptopenjdk.net/resultSummary?parentId=603f73945730424dbc9ad608

Failing job:
https://ci.adoptopenjdk.net/job/Test_openjdk16_j9_sanity.openjdk_aarch64_linux/41/consoleIssues:
Issues:
eclipse-openj9/openj9#11915

Failing job:
https://ci.adoptopenjdk.net/job/Test_openjdk16_j9_extended.openjdk_aarch64_linux/10/consoleFull
Issues:
adoptium/infrastructure#2002

Failing job:
https://ci.adoptopenjdk.net/job/Test_openjdk16_j9_sanity.system_aarch64_linux/42/consoleFull
Issues:
eclipse-openj9/openj9#12124