(PA-1941) Add OpenSSL 1.1.0 #82

caseywilliams · 2018-07-26T23:51:22Z

~~This PR is a WIP; Do not merge~~ This is ready to merge!

mcdonaldseanp · 2018-07-30T16:29:11Z

jenkins please test with puppet-agent#master on aix61-POWERfa,debian9-64a,debian9-32a,fedora26-64a,fedora27-64a,osx1012-64a,osx1013-64a,redhat5-64a,redhat6-64a,redhat6-32a,redhat7-64a,solaris11-64a,windows10ent-32a

caseywilliams · 2018-08-02T22:06:34Z

jenkins please test this with puppet-agent#master on aix61-POWERfa,arista4-32a,cumulus25-64a,debian8-64a,debian8-32a,debian9-64a,debian9-32a,fedora26-64a,fedora27-64a,fedora28-64a,osx1012-64a,osx1013-64a,redhat5-64a,redhat6-64a,redhat6-32a,redhat7-64a,redhat7-POWERa,redhat7-AARCH64a,redhatfips7-64a,sles11-64a,sles11-32a,sles12-64a,sles12-POWERa,solaris10-64a,solaris10-SPARCa,solaris11-64a,solaris11-SPARCa,ubuntu1404-64a,ubuntu1404-32a,ubuntu1604-64a,ubuntu1604-32a,ubuntu1604-POWERa,ubuntu1804-64a,windows10ent-32a,windows2012r2-64a

This commit migrates build requirements from pkg.build_requires statements in component files to platform files. As part of this, statements have been added pruning out EL builds from using pkg.build_requires at all.

This commit changes the location of build requirements from component configuration to platform configuration.

This commit moves the build requirements from component files to the platform files.

This commit updates the netdev and fedora 25 platforms to use platform files to specify build requirements.

This commit removes all pkg.build_requires statements from components that reference external packages. We have migrated all external package deps to platform files, where we can have more control over them

There was a typo in a regex check.

OpenSSL is failing to operate on aarch64 linux when built in a standard format. Per suggestion of OpenSSL themselves https://github.com/openssl/openssl/blob/OpenSSL_1_1_0h/INSTALL#L678L680 this commit patches the ssl configuration to use -O2 instead of -O3 when compiling the tool. This seems to have fixed the issue

caseywilliams · 2018-08-03T23:47:50Z

puppetlabs/puppet-agent#1503 is required for puppet-agent#master for when this is merged and promoted

caseywilliams · 2018-08-03T23:52:26Z

Here are some green agent acceptance tests of puppet-agent#master using openssl 1.1.0 (with AIX 7.1 and 7.2 failures due to my own configuration mistakes - those have passed separately before, though, and I'll retest them tonight). I also confirmed that the agent 5.5.x, PDK, and bolt runtimes still build for all of their supported platforms with the build_requires migrations that have been made here.

ekinanp · 2018-08-04T00:20:01Z

configs/platforms/cumulus-2.2-amd64.rb

@@ -18,9 +28,8 @@
 echo 'deb http://osmirror.delivery.puppetlabs.net/debian/ wheezy main
 deb http://osmirror.delivery.puppetlabs.net/debian/ wheezy-updates main' >> /etc/apt/sources.list
 apt-get update -qq
-apt-get install -qy --no-install-recommends build-essential make quilt pkg-config debhelper devscripts rsync
+apt-get install -qy --no-install-recommends build-essential make quilt pkg-config debhelper devscripts rsync #{packages.join(' ')}


Nitpick, but would be nice to include build-essential, make, etc. in the packages array too.

ekinanp · 2018-08-04T00:25:32Z

configs/components/openssl-1.1.0.rb

@@ -0,0 +1,145 @@
+component 'openssl' do |pkg, settings, platform|


There's some common stuff here w/ the other openssl.rb file. Least the ENVIRONMENT, FLAGS, TARGETS + INSTALL STEPS. Could there be a _base_openssl.rb file?

geoffnichols · 2018-08-04T03:02:22Z

So great to see build_requires consolidated into the platform configs here!!

💯

mcdonaldseanp · 2018-08-06T16:22:40Z

The few style changes requested are probably valid, but we need to move forward with ruby so IMO we should merge this

branan · 2018-08-06T16:35:48Z

Given we're going to have OpenSSL 1.0 and 1.1 side-by-side for quite a while (the lifetime of 5.5.x), should we factor out a base openssl class? Did the configuration end up different enough between the two that it's not worth it?

branan · 2018-08-06T16:37:38Z

configs/components/_base-ruby.rb

@@ -140,11 +116,27 @@
  #                   -Sean P. McDonald 07/01/2016
  lib_type = platform.architecture == "x64" ? "seh" : "sjlj"


Looks like you replaced usage of lib_type with gcc_postfix - should this line be removed?

branan · 2018-08-06T16:38:22Z

configs/components/_base-ruby.rb

@@ -140,11 +116,27 @@
  #                   -Sean P. McDonald 07/01/2016
  lib_type = platform.architecture == "x64" ? "seh" : "sjlj"

+  if platform.architecture == "x64"
+    gcc_postfix = 'seh'


Super bikesheddy nitpick: would suffix be better?

branan · 2018-08-06T16:45:39Z

configs/components/augeas.rb

    if platform.architecture =~ /aarch64|ppc64le|s390x/
      pkg.build_requires "runtime-#{settings[:runtime_project]}"
      pkg.environment "PATH", "/opt/pl-build-tools/bin:$(PATH):#{settings[:bindir]}"
      pkg.environment "CFLAGS", settings[:cflags]
      pkg.environment "LDFLAGS", settings[:ldflags]
    end
  elsif platform.is_deb?
-    pkg.build_requires 'libreadline-dev'
    if platform.name =~ /debian-9/
      pkg.requires 'libreadline7'


Should these just get bounced up to the platform level as well?

I believe pkg.requires if for runtime requirements to be added to the package, so I think these need to stay.

Now that I think about it, this... actually probably does nothing, since we build the runtime as a tarball and not as a full package?

branan · 2018-08-06T16:46:08Z

configs/components/augeas.rb

@@ -90,7 +79,6 @@
      pkg.environment "PKG_CONFIG_PATH", "/opt/csw/lib/pkgconfig"


I think ideally we'd want to move all these environment variables up to platform files too, but that might be too much for this PR :)

There's also no way to force environment from platform files, we would need to add that to vanagon

Yeah, we need to enable that - a ton of the duplication across components is environment variables

branan

The only one of my comments I really want to see addressed is the dead code lib_type. The others are more "open questions and improvements for the future"

Security updates

caseywilliams added the do not merge label Jul 26, 2018

mcdonaldseanp mentioned this pull request Jul 30, 2018

(PA-1942) Update OpenSSL to 1.1.0 in puppet-agent#master #73

Closed

caseywilliams force-pushed the PA-1941/openssl-1.1.0 branch 2 times, most recently from a1ff875 to dbb4b43 Compare August 2, 2018 17:59

caseywilliams force-pushed the PA-1941/openssl-1.1.0 branch 3 times, most recently from 0ebc02b to 5f4b6af Compare August 2, 2018 23:58

Sean P. McDonald and others added 13 commits August 2, 2018 16:59

(PA-1941) Update OpenSSL to 1.1.0 in puppet-agent#master

e65a242

(PA-1941) Use a newer perl to configure openssl 1.1.0 on EL 5

1251d42

(PA-2024) Update EL to get build requirements from platform file

3929222

This commit migrates build requirements from pkg.build_requires statements in component files to platform files. As part of this, statements have been added pruning out EL builds from using pkg.build_requires at all.

(PA-1941) Set OpenSSL version explicitly in PDK and Bolt

e3ff415

(PA-2024) Move debian-based build requirements to platform files

d46ffb5

(PA-2024) Update fedora/SLES build requires

674382c

This commit changes the location of build requirements from component configuration to platform configuration.

(PA-2024) Move solaris build requirements to platform files

d295436

(PA-2024) Move macOS build requirements to platform files

1d9a6bb

(PA-2024) Move Windows build requires to platform files

c76df8c

This commit moves the build requirements from component files to the platform files.

(PA-2024) Move AIX build requirements to platform files

80e0abe

(PA-2024) Update netdev/f25 to get build_requires from platform files

94a538d

This commit updates the netdev and fedora 25 platforms to use platform files to specify build requirements.

(PA-2024) Remove all build_requires from components

b50f7ec

This commit removes all pkg.build_requires statements from components that reference external packages. We have migrated all external package deps to platform files, where we can have more control over them

(maint) Fix typo

eb16100

There was a typo in a regex check.

caseywilliams force-pushed the PA-1941/openssl-1.1.0 branch from 5f4b6af to eb16100 Compare August 2, 2018 23:59

caseywilliams force-pushed the PA-1941/openssl-1.1.0 branch from cedc607 to 9a9814b Compare August 3, 2018 21:02

caseywilliams mentioned this pull request Aug 3, 2018

(PA-1941) Fix libnames for new OpenSSL dlls puppetlabs/puppet-agent#1503

Merged

caseywilliams removed the do not merge label Aug 3, 2018

caseywilliams changed the title ~~(PA 1941) Add OpenSSL 1.1.0~~ (PA-1941) Add OpenSSL 1.1.0 Aug 4, 2018

ekinanp reviewed Aug 4, 2018

View reviewed changes

geoffnichols approved these changes Aug 4, 2018

View reviewed changes

mcdonaldseanp force-pushed the PA-1941/openssl-1.1.0 branch from 9a9814b to 3e4231b Compare August 6, 2018 16:11

mcdonaldseanp approved these changes Aug 6, 2018

View reviewed changes

branan reviewed Aug 6, 2018

View reviewed changes

branan suggested changes Aug 6, 2018

View reviewed changes

mcdonaldseanp force-pushed the PA-1941/openssl-1.1.0 branch from 894afe5 to 5ce6df8 Compare August 6, 2018 17:39

(maint) remove lib_type dead code

5ce6df8

branan merged commit 1a62356 into puppetlabs:master Aug 6, 2018

witjoh pushed a commit to witjoh/puppet-runtime that referenced this pull request Feb 24, 2021

Merge pull request puppetlabs#82 from DavidS/security-updates

7ea865b

Security updates

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

(PA-1941) Add OpenSSL 1.1.0 #82

(PA-1941) Add OpenSSL 1.1.0 #82

caseywilliams commented Jul 26, 2018 •

edited

Loading

mcdonaldseanp commented Jul 30, 2018

caseywilliams commented Aug 2, 2018

caseywilliams commented Aug 3, 2018

caseywilliams commented Aug 3, 2018 •

edited

Loading

ekinanp Aug 4, 2018

ekinanp Aug 4, 2018 •

edited

Loading

geoffnichols commented Aug 4, 2018

mcdonaldseanp commented Aug 6, 2018

branan commented Aug 6, 2018

branan Aug 6, 2018

branan Aug 6, 2018

mcdonaldseanp Aug 6, 2018

branan Aug 6, 2018

mcdonaldseanp Aug 6, 2018

branan Aug 6, 2018

branan Aug 6, 2018

mcdonaldseanp Aug 6, 2018

branan Aug 6, 2018

branan left a comment

		@@ -0,0 +1,145 @@
		component 'openssl' do \|pkg, settings, platform\|

		@@ -140,11 +116,27 @@
		# -Sean P. McDonald 07/01/2016
		lib_type = platform.architecture == "x64" ? "seh" : "sjlj"

		@@ -90,7 +79,6 @@
		pkg.environment "PKG_CONFIG_PATH", "/opt/csw/lib/pkgconfig"

(PA-1941) Add OpenSSL 1.1.0 #82

(PA-1941) Add OpenSSL 1.1.0 #82

Conversation

caseywilliams commented Jul 26, 2018 • edited Loading

mcdonaldseanp commented Jul 30, 2018

caseywilliams commented Aug 2, 2018

caseywilliams commented Aug 3, 2018

caseywilliams commented Aug 3, 2018 • edited Loading

Choose a reason for hiding this comment

ekinanp Aug 4, 2018 • edited Loading

Choose a reason for hiding this comment

geoffnichols commented Aug 4, 2018

mcdonaldseanp commented Aug 6, 2018

branan commented Aug 6, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

branan left a comment

Choose a reason for hiding this comment

caseywilliams commented Jul 26, 2018 •

edited

Loading

caseywilliams commented Aug 3, 2018 •

edited

Loading

ekinanp Aug 4, 2018 •

edited

Loading