Skip to content

Commit 5b754f3

Browse files
bastelfreakbastelfreak
committed
Make $pe_installer_source more flexible
This makes the peadm::install plan more flexible. Previously the variable $pe_installer_source could point to an absolute URL directly to the desired .tar.gz. Now it can also point to a web directory. Then peadm will calculate the .tar.gz name and fetch it from the web directory. This applies for the peadm::upgrade and peadm::install plan.
1 parent be0fb06 commit 5b754f3

File tree

7 files changed

+168
-41
lines changed

7 files changed

+168
-41
lines changed

‎REFERENCE.md

+49-11
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@
3636
* [`peadm::node_manager_yaml_location`](#peadm--node_manager_yaml_location)
3737
* [`peadm::oid`](#peadm--oid)
3838
* [`peadm::pe_db_names`](#peadm--pe_db_names)
39+
* [`peadm::pe_installer_source`](#peadm--pe_installer_source): calculates the PE installer URL and archive name
3940
* [`peadm::plan_step`](#peadm--plan_step)
4041
* [`peadm::recovery_opts_all`](#peadm--recovery_opts_all)
4142
* [`peadm::recovery_opts_default`](#peadm--recovery_opts_default)
@@ -844,6 +845,40 @@ Data type: `String`
844845

845846

846847

848+
### <a name="peadm--pe_installer_source"></a>`peadm::pe_installer_source`
849+
850+
Type: Puppet Language
851+
852+
calculates the PE installer URL and archive name
853+
854+
#### `peadm::pe_installer_source(Optional[Stdlib::HTTPSUrl] $pe_installer_source = undef, Optional[Peadm::Pe_version] $version = undef, Optional[String[1]] $platform = undef)`
855+
856+
The peadm::pe_installer_source function.
857+
858+
Returns: `Hash[String[1],String[1]]`
859+
860+
##### `pe_installer_source`
861+
862+
Data type: `Optional[Stdlib::HTTPSUrl]`
863+
864+
The URL to download the Puppet Enterprise installer media from. If not
865+
specified, PEAdm will attempt to download PE installation media from its
866+
standard public source. When specified, PEAdm will download directly from the
867+
URL given. Can be an URL, that ends with a /, to a web directory that
868+
contains the original archives or an absolute URL to the .tar.gz archive.
869+
870+
##### `version`
871+
872+
Data type: `Optional[Peadm::Pe_version]`
873+
874+
The desired version for PE. This is optional for custom provided absolute URLs.
875+
876+
##### `platform`
877+
878+
Data type: `Optional[String[1]]`
879+
880+
The platform we're on, for example el-9-x86_64 (osfamily short name - version - arch)
881+
847882
### <a name="peadm--plan_step"></a>`peadm::plan_step`
848883

849884
Type: Ruby 4.x API
@@ -2106,7 +2141,8 @@ Data type: `Optional[Stdlib::HTTPSUrl]`
21062141
The URL to download the Puppet Enterprise installer media from. If not
21072142
specified, PEAdm will attempt to download PE installation media from its
21082143
standard public source. When specified, PEAdm will download directly from the
2109-
URL given.
2144+
URL given. Can be an URL, that ends with a /, to a web directory that
2145+
contains the original archives or an absolute URL to the .tar.gz archive.
21102146

21112147
Default value: `undef`
21122148

@@ -2597,12 +2633,12 @@ The following parameters are available in the `peadm::upgrade` plan:
25972633
* [`stagingdir`](#-peadm--upgrade--stagingdir)
25982634
* [`uploaddir`](#-peadm--upgrade--uploaddir)
25992635
* [`begin_at_step`](#-peadm--upgrade--begin_at_step)
2636+
* [`version`](#-peadm--upgrade--version)
26002637
* [`primary_host`](#-peadm--upgrade--primary_host)
26012638
* [`replica_host`](#-peadm--upgrade--replica_host)
26022639
* [`compiler_hosts`](#-peadm--upgrade--compiler_hosts)
26032640
* [`primary_postgresql_host`](#-peadm--upgrade--primary_postgresql_host)
26042641
* [`replica_postgresql_host`](#-peadm--upgrade--replica_postgresql_host)
2605-
* [`version`](#-peadm--upgrade--version)
26062642
* [`token_file`](#-peadm--upgrade--token_file)
26072643
* [`download_mode`](#-peadm--upgrade--download_mode)
26082644
* [`permit_unsafe_versions`](#-peadm--upgrade--permit_unsafe_versions)
@@ -2643,7 +2679,9 @@ Data type: `Optional[Stdlib::HTTPSUrl]`
26432679
The URL to download the Puppet Enterprise installer media from. If not
26442680
specified, PEAdm will attempt to download PE installation media from its
26452681
standard public source. When specified, PEAdm will download directly from the
2646-
URL given.
2682+
URL given. Can be an URL, that ends with a /, to a web directory that
2683+
contains the original archives or an absolute URL to the .tar.gz archive.
2684+
If it's an URL ending with the archive name, you don't need to set $version.
26472685

26482686
Default value: `undef`
26492687

@@ -2693,6 +2731,14 @@ The step where the plan should start. If not set, it will start at the beginning
26932731

26942732
Default value: `undef`
26952733

2734+
##### <a name="-peadm--upgrade--version"></a>`version`
2735+
2736+
Data type: `Optional[Peadm::Pe_version]`
2737+
2738+
The desired version for PE. This is optional for custom provided absolute URLs.
2739+
2740+
Default value: `undef`
2741+
26962742
##### <a name="-peadm--upgrade--primary_host"></a>`primary_host`
26972743

26982744
Data type: `Peadm::SingleTargetSpec`
@@ -2729,14 +2775,6 @@ Data type: `Optional[Peadm::SingleTargetSpec]`
27292775

27302776

27312777

2732-
Default value: `undef`
2733-
2734-
##### <a name="-peadm--upgrade--version"></a>`version`
2735-
2736-
Data type: `Optional[Peadm::Pe_version]`
2737-
2738-
2739-
27402778
Default value: `undef`
27412779

27422780
##### <a name="-peadm--upgrade--token_file"></a>`token_file`

‎functions/pe_installer_source.pp

+46
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
#
2+
# @summary calculates the PE installer URL and archive name
3+
#
4+
# @param pe_installer_source
5+
# The URL to download the Puppet Enterprise installer media from. If not
6+
# specified, PEAdm will attempt to download PE installation media from its
7+
# standard public source. When specified, PEAdm will download directly from the
8+
# URL given. Can be an URL, that ends with a /, to a web directory that
9+
# contains the original archives or an absolute URL to the .tar.gz archive.
10+
#
11+
# @param version
12+
# The desired version for PE. This is optional for custom provided absolute URLs.
13+
#
14+
# @param platform
15+
# The platform we're on, for example el-9-x86_64 (osfamily short name - version - arch)
16+
#
17+
# @author Tim Meusel <[email protected]>
18+
#
19+
function peadm::pe_installer_source (
20+
Optional[Stdlib::HTTPSUrl] $pe_installer_source = undef,
21+
Optional[Peadm::Pe_version] $version = undef,
22+
Optional[String[1]] $platform = undef,
23+
) >> Hash[String[1],String[1]] {
24+
if $pe_installer_source {
25+
# custom URL ends with /, so we assume it's a webdir with the original installer
26+
if $pe_installer_source[-1] == '/' {
27+
assert_type(Peadm::Pe_version, $version)
28+
assert_type(String[1], $platform)
29+
$_version = $version
30+
$pe_tarball_name = "puppet-enterprise-${version}-${platform}.tar.gz"
31+
$pe_tarball_source = "${pe_installer_source}${pe_tarball_name}"
32+
} else {
33+
$pe_tarball_name = $pe_installer_source.split('/')[-1]
34+
$pe_tarball_source = $pe_installer_source
35+
$_version = $pe_tarball_name.split('-')[2]
36+
}
37+
$data = { 'url' => $pe_tarball_source, 'filename' => $pe_tarball_name, 'version' => pick($_version,$version), }
38+
} else {
39+
assert_type(Peadm::Pe_version, $version)
40+
assert_type(String[1], $platform)
41+
$pe_tarball_name = "puppet-enterprise-${version}-${platform}.tar.gz"
42+
$pe_tarball_source = "https://s3.amazonaws.com/pe-builds/released/${version}/${pe_tarball_name}"
43+
$data = { 'url' => $pe_tarball_source, 'filename' => $pe_tarball_name, 'version' => $version }
44+
}
45+
$data
46+
}

‎plans/install.pp

+2-1
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,8 @@
1515
# The URL to download the Puppet Enterprise installer media from. If not
1616
# specified, PEAdm will attempt to download PE installation media from its
1717
# standard public source. When specified, PEAdm will download directly from the
18-
# URL given.
18+
# URL given. Can be an URL, that ends with a /, to a web directory that
19+
# contains the original archives or an absolute URL to the .tar.gz archive.
1920
# @param ldap_config
2021
# If specified, configures PE RBAC DS with the supplied configuration hash.
2122
# The parameter should be set to a valid set of connection settings as

‎plans/subplans/install.pp

+7-12
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,8 @@
3232
# The URL to download the Puppet Enterprise installer media from. If not
3333
# specified, PEAdm will attempt to download PE installation media from its
3434
# standard public source. When specified, PEAdm will download directly from the
35-
# URL given.
35+
# URL given. Can be an URL, that ends with a /, to a web directory that
36+
# contains the original archives or an absolute URL to the .tar.gz archive.
3637
#
3738
plan peadm::subplans::install (
3839
# Standard
@@ -237,27 +238,21 @@
237238
)
238239
}
239240

240-
if $pe_installer_source {
241-
$pe_tarball_name = $pe_installer_source.split('/')[-1]
242-
$pe_tarball_source = $pe_installer_source
243-
} else {
244-
$pe_tarball_name = "puppet-enterprise-${version}-${platform}.tar.gz"
245-
$pe_tarball_source = "https://s3.amazonaws.com/pe-builds/released/${version}/${pe_tarball_name}"
246-
}
241+
$pe_installer = peadm::pe_installer_source($pe_installer_source, $version, $platform)
247242

248-
$upload_tarball_path = "${uploaddir}/${pe_tarball_name}"
243+
$upload_tarball_path = "${uploaddir}/${pe_installer['filename']}"
249244

250245
if $download_mode == 'bolthost' {
251246
# Download the PE tarball and send it to the nodes that need it
252247
run_plan('peadm::util::retrieve_and_upload', $pe_installer_targets,
253-
source => $pe_tarball_source,
254-
local_path => "${stagingdir}/${pe_tarball_name}",
248+
source => $pe_installer['url'],
249+
local_path => "${stagingdir}/${pe_installer['filename']}",
255250
upload_path => $upload_tarball_path,
256251
)
257252
} else {
258253
# Download PE tarballs directly to nodes that need it
259254
run_task('peadm::download', $pe_installer_targets,
260-
source => $pe_tarball_source,
255+
source => $pe_installer['url'],
261256
path => $upload_tarball_path,
262257
)
263258
}

‎plans/upgrade.pp

+13-17
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,9 @@
1515
# The URL to download the Puppet Enterprise installer media from. If not
1616
# specified, PEAdm will attempt to download PE installation media from its
1717
# standard public source. When specified, PEAdm will download directly from the
18-
# URL given.
18+
# URL given. Can be an URL, that ends with a /, to a web directory that
19+
# contains the original archives or an absolute URL to the .tar.gz archive.
20+
# If it's an URL ending with the archive name, you don't need to set $version.
1921
# @param final_agent_state
2022
# Configures the state the puppet agent should be in on infrastructure nodes
2123
# after PE is upgraded successfully.
@@ -31,6 +33,8 @@
3133
# Directory the installer tarball will be uploaded to or expected to be in
3234
# for offline usage.
3335
# @param begin_at_step The step where the plan should start. If not set, it will start at the beginning
36+
# @param version
37+
# The desired version for PE. This is optional for custom provided absolute URLs.
3438
#
3539
plan peadm::upgrade (
3640
# Standard
@@ -119,21 +123,13 @@
119123

120124
$platform = run_task('peadm::precheck', $primary_target).first['platform']
121125

122-
if $pe_installer_source {
123-
$pe_tarball_name = $pe_installer_source.split('/')[-1]
124-
$pe_tarball_source = $pe_installer_source
125-
$_version = $pe_tarball_name.split('-')[2]
126-
} else {
127-
$_version = $version
128-
$pe_tarball_name = "puppet-enterprise-${_version}-${platform}.tar.gz"
129-
$pe_tarball_source = "https://s3.amazonaws.com/pe-builds/released/${_version}/${pe_tarball_name}"
130-
}
126+
$pe_installer = peadm::pe_installer_source($pe_installer_source, $version, $platform)
131127

132-
$upload_tarball_path = "${uploaddir}/${pe_tarball_name}"
128+
$upload_tarball_path = "${uploaddir}/${pe_installer['filename']}"
133129

134130
peadm::assert_supported_bolt_version()
135131

136-
peadm::assert_supported_pe_version($_version, $permit_unsafe_versions)
132+
peadm::assert_supported_pe_version($pe_installer['version'], $permit_unsafe_versions)
137133

138134
# Gather certificate extension information from all systems
139135
$cert_extensions_temp = run_task('peadm::cert_data', $all_targets).reduce({}) |$memo,$result| {
@@ -208,14 +204,14 @@
208204
if $download_mode == 'bolthost' {
209205
# Download the PE tarball on the nodes that need it
210206
run_plan('peadm::util::retrieve_and_upload', $pe_installer_targets,
211-
source => $pe_tarball_source,
212-
local_path => "${stagingdir}/${pe_tarball_name}",
207+
source => $pe_installer['url'],
208+
local_path => "${stagingdir}/${pe_installer['filename']}",
213209
upload_path => $upload_tarball_path,
214210
)
215211
} else {
216212
# Download PE tarballs directly to nodes that need it
217213
run_task('peadm::download', $pe_installer_targets,
218-
source => $pe_tarball_source,
214+
source => $pe_installer['url'],
219215
path => $upload_tarball_path,
220216
)
221217
}
@@ -402,7 +398,7 @@
402398
# doesn't deal well with the PuppetDB database being on a separate node.
403399
# So, move it aside before running the upgrade.
404400
$pdbapps = '/opt/puppetlabs/server/apps/puppetdb/cli/apps'
405-
$workaround_delete_reports = $arch['disaster-recovery'] and $_version =~ SemVerRange('>= 2019.8')
401+
$workaround_delete_reports = $arch['disaster-recovery'] and $pe_installer['version'] =~ SemVerRange('>= 2019.8')
406402
if $workaround_delete_reports {
407403
# lint:ignore:strict_indent
408404
run_command(@("COMMAND"/$), $replica_target)
@@ -454,7 +450,7 @@
454450
)
455451
}
456452
457-
peadm::check_version_and_known_hosts($current_pe_version, $_version, $r10k_known_hosts)
453+
peadm::check_version_and_known_hosts($current_pe_version, $pe_installer['version'], $r10k_known_hosts)
458454
459455
return("Upgrade of Puppet Enterprise ${arch['architecture']} completed.")
460456
}

‎spec/functions/pe_installer_source_spec.rb

+47
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# frozen_string_literal: true
2+
3+
require 'spec_helper'
4+
5+
describe 'peadm::pe_installer_source' do
6+
it 'exists' do
7+
is_expected.not_to be_nil
8+
end
9+
10+
context 'when called with no parameters' do
11+
it { is_expected.to run.with_params.and_raise_error(Puppet::PreformattedError) }
12+
end
13+
context 'when called with absolute url and version' do
14+
result = {
15+
'url' => 'https://url/puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
16+
'filename' => 'puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
17+
'version' => '2019.8.12'
18+
}
19+
it { is_expected.to run.with_params('https://url/puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz', '2019.8.12').and_return(result) }
20+
end
21+
context 'when called with absolute url' do
22+
result = {
23+
'url' => 'https://url/puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
24+
'filename' => 'puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
25+
'version' => '2019.8.12'
26+
}
27+
it { is_expected.to run.with_params('https://url/puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz').and_return(result) }
28+
end
29+
context 'when called with url and version and platform' do
30+
result = {
31+
'url' => 'https://url/puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
32+
'filename' => 'puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
33+
'version' => '2019.8.12'
34+
}
35+
it { is_expected.to run.with_params('https://url/', '2019.8.12', 'el-8-x86_64').and_return(result) }
36+
end
37+
context 'when called without url and with version and platform' do
38+
result = {
39+
'url' => 'https://s3.amazonaws.com/pe-builds/released/2019.8.12/puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
40+
'filename' => 'puppet-enterprise-2019.8.12-el-8-x86_64.tar.gz',
41+
'version' => '2019.8.12'
42+
}
43+
it do
44+
is_expected.to run.with_params(nil, '2019.8.12', 'el-8-x86_64').and_return(result)
45+
end
46+
end
47+
end

‎spec/plans/upgrade_spec.rb

+4
Original file line numberDiff line numberDiff line change
@@ -30,6 +30,7 @@ def allow_standard_non_returning_calls
3030

3131
expect_task('peadm::cert_data').return_for_targets('primary' => trusted_primary).be_called_times(2)
3232

33+
allow_task('peadm::precheck').return_for_targets('primary' => { 'hostname' => 'primary', 'platform' => 'el-7.11-x86_64' })
3334
expect(run_plan('peadm::upgrade',
3435
'primary_host' => 'primary',
3536
'version' => '2021.7.9')).to be_ok
@@ -45,6 +46,7 @@ def allow_standard_non_returning_calls
4546

4647
expect_task('peadm::cert_data').return_for_targets('primary' => trusted_primary,
4748
'compiler' => trusted_compiler).be_called_times(2)
49+
allow_task('peadm::precheck').return_for_targets('primary' => { 'hostname' => 'primary', 'platform' => 'el-7.11-x86_64' })
4850

4951
expect(run_plan('peadm::upgrade',
5052
'primary_host' => 'primary',
@@ -95,6 +97,8 @@ def allow_standard_non_returning_calls
9597

9698
expect_task('peadm::cert_data').return_for_targets('primary' => trusted_primary).be_called_times(2)
9799
expect_task('peadm::get_group_rules').return_for_targets('primary' => { '_output' => '{"rules": []}' })
100+
101+
allow_task('peadm::precheck').return_for_targets('primary' => { 'hostname' => 'primary', 'platform' => 'el-7.11-x86_64' })
98102
end
99103

100104
it 'updates pe.conf if r10k_known_hosts is set' do

0 commit comments

Comments
 (0)