Skip to content

Change finalization of elasticsearch distribution #64403

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Nov 11, 2020

Conversation

breskeby
Copy link
Contributor

@breskeby breskeby commented Oct 30, 2020

  • Avoid to rely on afterEvaluate lifecycle hook in DistributionDownloadPlugin
  • Allows us to define test cluster only in the task creation which can be later than afterevaluate which allows us to move tasks that rely on test clusters using task avoidance api.
  • This should result in test clusters only being defined when actually used in the build
  • This unblocks further progress on Gradle task avoidance api should be used consistently in the build #56610

@breskeby breskeby self-assigned this Oct 30, 2020
@breskeby breskeby added :Delivery/Build Build or test infrastructure Team:Core/Infra Meta label for core/infra team v7.11.0 v8.0.0 labels Oct 30, 2020
@breskeby breskeby force-pushed the lazy-distribution-setup branch 2 times, most recently from d8f5f87 to 08cb078 Compare November 2, 2020 09:37
@breskeby
Copy link
Contributor Author

breskeby commented Nov 2, 2020

@elasticmachine test this please

@breskeby breskeby force-pushed the lazy-distribution-setup branch 2 times, most recently from b7675dc to 1f1b370 Compare November 4, 2020 13:17
@@ -75,6 +75,31 @@ class TestClustersPluginFuncTest extends AbstractGradleFuncTest {
assertNoCustomDistro('myCluster')
}

def "can declare test cluster in lazy evaluated task configuration block"() {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One scenario we wanna support with this PR

@breskeby breskeby force-pushed the lazy-distribution-setup branch from 587c796 to 999d39c Compare November 5, 2020 16:07
@breskeby
Copy link
Contributor Author

breskeby commented Nov 6, 2020

@elasticsearchmachine test this please

@breskeby breskeby marked this pull request as ready for review November 6, 2020 14:18
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (:Core/Infra/Build)

@breskeby breskeby force-pushed the lazy-distribution-setup branch from c2a6997 to d821451 Compare November 9, 2020 10:45

private Optional<TaskDependency> dockerBuildDependencies() {
// For non-required Docker distributions, skip building the distribution is Docker is unavailable
return (isDocker() && getFailIfUnavailable() == false && dockerSupport.get().getDockerAvailability().isAvailable == false)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't seem right. In both scenarios we return what is effectively an empty set of dependencies. If docker is available, or we don't consider this to be optional, we should return the configuration's build dependencies.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reworked this

@@ -103,13 +105,16 @@ public String toString() {
private final Property<Boolean> bundledJdk;
private final Property<Boolean> failIfUnavailable;
private final Configuration extracted;
private Function<ElasticsearchDistribution, ElasticsearchDistribution> distributionFinalizer;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't this just be Action<Distribution>? I don't see where we ever actually use the returned result of this function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

@breskeby breskeby requested a review from mark-vieira November 10, 2020 11:35
@mark-vieira
Copy link
Contributor

@breskeby not sure why this PR is showing as having a bunch of unrelated changes. Did a merge go wrong?

@breskeby
Copy link
Contributor Author

Probably a rebase. Let me check

- avoid to rely on afterEvaluate lifecycle hook
- allows us to define test cluster only in the task creation which can be later than afterevaluate
@breskeby breskeby force-pushed the lazy-distribution-setup branch from 7a8f58e to bb199cf Compare November 10, 2020 18:24
@breskeby
Copy link
Contributor Author

@mark-vieira fixed

Copy link
Contributor

@mark-vieira mark-vieira left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes LGTM

@breskeby breskeby merged commit 65e760f into elastic:master Nov 11, 2020
breskeby added a commit to breskeby/elasticsearch that referenced this pull request Nov 11, 2020
- Avoid to rely on afterEvaluate lifecycle hook in DistributionDownloadPlugin
- Allows us to define test cluster only in the task creation which can be later than afterevaluate
which allows us to move tasks that rely on test clusters using task avoidance api.
- This should result in test clusters only being defined when actually used in the build
- This unblocks further progress on elastic#56610
@mark-vieira mark-vieira added Team:Delivery Meta label for Delivery team and removed Team:Core/Infra Meta label for core/infra team labels Nov 11, 2020
breskeby added a commit that referenced this pull request Nov 12, 2020
- Avoid to rely on afterEvaluate lifecycle hook in DistributionDownloadPlugin
- Allows us to define test cluster only in the task creation which can be later than afterevaluate
which allows us to move tasks that rely on test clusters using task avoidance api.
- This should result in test clusters only being defined when actually used in the build
- This unblocks further progress on #56610
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Delivery/Build Build or test infrastructure Team:Delivery Meta label for Delivery team v7.11.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants