Update build to use gradle wrapper 6.8 #65596

breskeby · 2020-11-30T11:58:10Z

Updates our build to use the latest Gradle 6.8 release which is the last release before the major 7.0 release
Resolve available gradle versions using built-in toolchain support
Fixes deprecated usage of JavaInstallationRegistry
We replace jdk handling in our build to rely on jvm detection provided by the gradle build tool itself. As we rely on environment variables pointing to jdks we wire this into the gradle jdk detection mechanism

breskeby · 2020-12-01T16:39:10Z

Raised gradle/gradle#15392 at gradle to handle the windows issues we see

breskeby · 2020-12-02T09:23:13Z

breskeby · 2020-12-21T09:08:50Z

buildSrc/src/test/java/org/elasticsearch/gradle/plugin/PluginBuildPluginTests.java

-            project.getTasks().findByName("generateNotice")
-        );
-    }
-}


This test class is not working in a unit test context anymore. The only non ignored test checked for the existence of the registered extension. Thats not enough for me to have this ported to an integ tests at the moment so I removed the unit test.

elasticmachine · 2021-01-11T08:10:27Z

Pinging @elastic/es-delivery (Team:Delivery)

breskeby · 2021-01-11T08:10:59Z

Raised gradle/gradle#15392 at gradle to handle the windows issues we see

That has been fixed in Gradle 6.8-rc-3

breskeby · 2021-01-11T14:19:27Z

@elasticmachine test this please

mark-vieira

Few comments, otherwise LGTM.

mark-vieira · 2021-01-11T21:32:31Z

buildSrc/src/main/java/org/elasticsearch/gradle/info/GlobalBuildInfoPlugin.java

+     * We resolve all available java versions using auto detected by gradles tool chain
+     * To make transition more reliable we only take env var provided installations into account for now
+     */
+    private List<JavaHome> getAvailableJavaVersions() {


The bit we lose here is that we no longer enorce that the environment variable name matches the JDK version it points at. I guess this would later fail w/ us not being able to resovle a compatible JDK.

good point. is that something you think is worth to invest in? I guess those issues are rare, but when they occur they suck to be debugged I can imagine. Maybe we want to look into this safety net in a follow up?

I think realistically what we want to do is remove the reliance on these environment variables and use Gradle's toolchain capabilities to just download a compatible JDK when necessary. These variables are only used for build time purposes, where we intend to run tests with a particular JDK implementation, that's what RUNTIME_JAVA_HOME is for.

So I think we can punt here and instead focus on ditching all usages of these things vs resolving JDKs dynamically.

I will look into this in a separate PR. I didn't want to make too many changes in this PR. So RUNTIME_JAVA_HOME is basically used to configure the jdk on CI to run all tests against?

The problem I see at the moment is that the compatibility choices (which vendor is available etc.) is limited in gradle at the moment and might not be enough. If you say that's less of an issue for build time then the toolchain capabilities in gradle might be enough at the moment to pick a proper one

mark-vieira · 2021-01-11T21:37:24Z

buildSrc/src/integTest/groovy/org/elasticsearch/gradle/fixtures/AbstractGradleFuncTest.groovy

@@ -60,6 +63,10 @@ abstract class AbstractGradleFuncTest extends Specification {
        true
    }

+    def assertOutputContainsNot(String givenOutput, String expected) {


assertOutputDoesNotContain or assertOutputMissing?

thanks. assertOutputMissing it is

mark-vieira · 2021-01-11T21:47:37Z

buildSrc/src/integTest/java/org/elasticsearch/gradle/BuildPluginIT.java

@@ -105,7 +105,9 @@ private void runInsecureArtifactRepositoryTest(final String name, final String u
            .withProjectDir(tmpDir.getRoot())
            .withArguments("clean", "hello", "-s", "-i", "--warning-mode=all", "--scan")
            .withPluginClasspath()
+            .forwardOutput()


Do we want to keep this here or was this only for testing? I'm ok with keeping this around across the board BTW.

I think we should keep it.

mark-vieira · 2021-01-11T21:49:20Z

buildSrc/src/main/java/org/elasticsearch/gradle/precommit/JarHellPrecommitPlugin.java

@@ -32,6 +32,9 @@
    public TaskProvider<? extends Task> createTask(Project project) {
        Configuration jarHellConfig = project.getConfigurations().create("jarHell");
        if (BuildParams.isInternal() && project.getPath().equals(":libs:elasticsearch-core") == false) {
+            // ideally we would configure this as a default dependency. But Default dependencies do not work correctly


I think that's fine anyway as we don't expect this to be replaced.

Other question though. How does this work for external builds? Where does the jarhell implementation come from?

You would need to add a dependency like `jarHell 'org.elasticsearch:elasticsearch-core:7.10' at the moment. In general I think we need to put some more dedicated effort into making these external available plugins work nicely for contributors. By that I mean having proper test coverage of third party plugin author use cases etc. I remember we talked about that very early on when having our first talks about the elastic build already.
We're moving in that direction with having internal plugins nowadays but there's more work required here

- resolve available gradle versions using built-in toolchain support - fixes deprecated usage of JavaInstallationRegistry

- with gradle 6.8 artifact transforms output is kept in build cache - therefore there is no skipping info seen in our tests

- Updates our build to use the latest Gradle 6.8 release which is the last release before the major 7.0 release. - Resolve available gradle versions using built-in toolchain support - Fixes deprecated usage of JavaInstallationRegistry - We replace jdk handling in our build to rely on jvm detection provided by the gradle build tool itself. As we rely on environment variables pointing to jdks we wire this into the gradle jdk detection mechanism

breskeby self-assigned this Nov 30, 2020

breskeby added :Delivery/Build Build or test infrastructure Team:Delivery Meta label for Delivery team v7.11.0 v8.0.0 labels Nov 30, 2020

breskeby force-pushed the update-gradle-to-6-8 branch from 8a7687d to c26b3cb Compare December 1, 2020 11:36

breskeby force-pushed the update-gradle-to-6-8 branch from b4a4e3b to 52a70d8 Compare December 4, 2020 12:06

breskeby force-pushed the update-gradle-to-6-8 branch from c0dcdc3 to c654b22 Compare December 21, 2020 08:55

breskeby commented Dec 21, 2020

View reviewed changes

breskeby force-pushed the update-gradle-to-6-8 branch from 456edfd to 295be18 Compare January 6, 2021 08:06

breskeby added v7.12.0 and removed v7.11.0 labels Jan 11, 2021

breskeby marked this pull request as ready for review January 11, 2021 08:10

breskeby requested review from mark-vieira and pugnascotia January 11, 2021 15:36

pugnascotia approved these changes Jan 11, 2021

View reviewed changes

mark-vieira approved these changes Jan 11, 2021

View reviewed changes

breskeby added 8 commits January 12, 2021 10:41

Update gradle to 6.8-rc-1

bc7440a

- resolve available gradle versions using built-in toolchain support - fixes deprecated usage of JavaInstallationRegistry

Add debug output for resolving jdks

9a60a22

Add RUNTIME_JAVA_HOME as resolvable tool chain

31fdbaa

Fix assertions for up-to-date Transforms

39cf274

- with gradle 6.8 artifact transforms output is kept in build cache - therefore there is no skipping info seen in our tests

Remove debug output

757d507

Add debug output for resolved jdk toolchains

e99d42b

Configure defaults for toolchain resolution in gradle func tests

e77944f

Fix plugin func test setup

0b61e3e

breskeby added 10 commits January 12, 2021 10:41

Rework toolchain resolution filter

0292688

Disable toolchain autodetection for now

a38d5b9

Update gradle to latest nightly

e8a9afa

Fix Global Info log java info string

3dd4234

Update to 6.8-rc-3 and some cleanup

bbc37af

Add missing files

eae583b

Fix artifact transform test

d3611f3

Update wrapper to 6.8-rc-5

f2dd4c3

Update to Gradle 6.8 GA

1e50f5d

Polishing func test assertion methods

77dbe62

breskeby force-pushed the update-gradle-to-6-8 branch from f717ed4 to 77dbe62 Compare January 12, 2021 09:42

breskeby merged commit a8757f9 into elastic:master Jan 12, 2021

breskeby added the backport pending label Jan 12, 2021

breskeby mentioned this pull request Jan 12, 2021

Update build to use gradle wrapper 6.8 (7.x backport) #67327

Merged

jkakavas mentioned this pull request Jan 13, 2021

1.2.840.113549.1.1.4 Signature not available for BCJSSE in fips mode #66821

Closed

breskeby removed the backport pending label Jan 18, 2021

breskeby deleted the update-gradle-to-6-8 branch January 18, 2021 09:47

jakelandis added v8.0.0-alpha1 and removed v8.0.0 labels Jul 26, 2021

Update build to use gradle wrapper 6.8 #65596

Update build to use gradle wrapper 6.8 #65596

Uh oh!

Conversation

breskeby commented Nov 30, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

breskeby commented Dec 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

breskeby commented Dec 2, 2020

Uh oh!

breskeby Dec 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Jan 11, 2021

Uh oh!

breskeby commented Jan 11, 2021

Uh oh!

breskeby commented Jan 11, 2021

Uh oh!

mark-vieira left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

breskeby commented Nov 30, 2020 •

edited

Loading

breskeby commented Dec 1, 2020 •

edited

Loading

breskeby Dec 21, 2020 •

edited

Loading