Skip to content

Improve build scan metadata #44247

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jul 16, 2019
Merged

Conversation

mark-vieira
Copy link
Contributor

This PR does a couple of things:

First, we move the logic for some of our custom build scan into the build itself vs having this live in our Jenkins configuration. Having the conditional tagging logic in our Jenkins config is awkward for a few reasons.

  1. Any actual conditional logic has to be done in bash, rather than in Groovy.
  2. There's a good deal of duplication as this gets copied to all our builder macros.
  3. For any complex custom values we have to workaround Jenkins odd handling of string escaping which doesn't often align with normal shell escaping rules.
  4. We can't easily add custom data computed from actual build configuration.

Second, we've added some addition bits of information, mainly to fill in context gaps with the goal of making it less necessary to have to go back to the original Jenkins job page to get that information.

  1. The actual git commit, as well as link to the commit info on GitHub.
  2. The git branch.
  3. For CI builds, include the list of changes included in the job run.

@mark-vieira mark-vieira added the :Delivery/Build Build or test infrastructure label Jul 11, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra

Copy link
Contributor

@alpar-t alpar-t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but left some suggestions.

While at it it would be great to have information about the CI worker this is what I most often have to go back to jenkins for.
Jenkins sets NODE_LABELS we could prefix them and add them as tags here.

Instead of an included file, I'm wondering if it would be better to start a base plugin that would apply configuration to the root plugin and have this live in buildSrc along with everything else. We're starting to have more and more configuration that applies to the root project only.

}

// Jenkins-specific build scan metadata
if (System.getenv('JENKINS_URL')) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could check if this contains elastic.co to make sure it's our CI and accept the terms of service if so so no additional property needs to be passed in Jenkins.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Eventually we'll be pushing scans to Gradle Enterprise which doesn't require accepting the ToS so I think that might be unnecessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've gone ahead and implemented this. I agree, it'd be nice to ditch the need to pass the ToS param in Jenkins for the time being.

@mark-vieira
Copy link
Contributor Author

While at it it would be great to have information about the CI worker this is what I most often have to go back to jenkins for.

Indeed. Is it the labels specifically that you find yourself looking for? Is the information in the infrastructure section of the build scan insufficient?

@mark-vieira
Copy link
Contributor Author

Instead of an included file, I'm wondering if it would be better to start a base plugin that would apply configuration to the root plugin and have this live in buildSrc along with everything else. We're starting to have more and more configuration that applies to the root project only.

Yeah, I definitely didn't want to expand the root build.gradle any further. We should certainly work towards trimming that down and moving that logic into plugins of some sort, whether script plugins or binary plugins in buildSrc is almost a matter of preference.

@mark-vieira
Copy link
Contributor Author

mark-vieira commented Jul 12, 2019

Jenkins sets NODE_LABELS we could prefix them and add them as tags here.

I've added these as custom values rather than tags to try and minimize a bit of noise as some of these are long (like the worker name) and there can be lots of them. We can still search and filter builds in Gradle Enterprise with this method though. So we could say "show me all build that ran on a worker with the metal label".

@mark-vieira
Copy link
Contributor Author

@elasticmachine run elasticsearch-ci/packaging-sample

@mark-vieira
Copy link
Contributor Author

@atorok here's an example of the node labels: https://scans.gradle.com/s/hnaify7kg33vg/custom-values#L2-L5

def jenkinsUrl = System.getenv('JENKINS_URL') ? new URL(System.getenv('JENKINS_URL')) : null

// Accept Gradle ToS when project property org.elasticsearch.acceptScanTOS=true or this is an Elastic CI build
if (jenkinsUrl?.host?.endsWith('elastic.co') || Boolean.valueOf(project.findProperty('org.elasticsearch.acceptScanTOS') ?: "false")) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could just ditch the property, I don't see a reason to opt out of CI

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use this property to opt-in locally.

@mark-vieira mark-vieira merged commit 2797b24 into elastic:master Jul 16, 2019
@mark-vieira mark-vieira deleted the build-scan-metadata branch July 16, 2019 16:34
mark-vieira added a commit that referenced this pull request Jul 16, 2019
(cherry picked from commit 2797b24)
mark-vieira added a commit that referenced this pull request Jul 22, 2019
(cherry picked from commit 2797b24)
mark-vieira added a commit that referenced this pull request Jul 22, 2019
(cherry picked from commit 2797b24)
mark-vieira added a commit that referenced this pull request Jul 22, 2019
(cherry picked from commit 2797b24)
@mark-vieira mark-vieira added the Team:Delivery Meta label for Delivery team label Nov 11, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Delivery/Build Build or test infrastructure Team:Delivery Meta label for Delivery team v6.8.3 v7.3.1 v7.4.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants