Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support configurable installer target upload path #376

Merged
merged 1 commit into from
Sep 19, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 40 additions & 18 deletions REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -1648,6 +1648,8 @@ The following parameters are available in the `peadm::install` plan:
* [`pe_installer_source`](#-peadm--install--pe_installer_source)
* [`ldap_config`](#-peadm--install--ldap_config)
* [`final_agent_state`](#-peadm--install--final_agent_state)
* [`stagingdir`](#-peadm--install--stagingdir)
* [`uploaddir`](#-peadm--install--uploaddir)
* [`primary_host`](#-peadm--install--primary_host)
* [`replica_host`](#-peadm--install--replica_host)
* [`compiler_hosts`](#-peadm--install--compiler_hosts)
Expand All @@ -1665,7 +1667,6 @@ The following parameters are available in the `peadm::install` plan:
* [`deploy_environment`](#-peadm--install--deploy_environment)
* [`license_key_file`](#-peadm--install--license_key_file)
* [`license_key_content`](#-peadm--install--license_key_content)
* [`stagingdir`](#-peadm--install--stagingdir)
* [`download_mode`](#-peadm--install--download_mode)
* [`permit_unsafe_versions`](#-peadm--install--permit_unsafe_versions)
* [`token_lifetime`](#-peadm--install--token_lifetime)
Expand Down Expand Up @@ -1730,6 +1731,24 @@ after PE is configured successfully.

Default value: `'running'`

##### <a name="-peadm--install--stagingdir"></a>`stagingdir`

Data type: `Optional[String]`

Directory on the Bolt host where the installer tarball will be cached if
download_mode is 'bolthost' (default)

Default value: `undef`

##### <a name="-peadm--install--uploaddir"></a>`uploaddir`

Data type: `Optional[String]`

Directory the installer tarball will be uploaded to or expected to be in
for offline usage.

Default value: `undef`

##### <a name="-peadm--install--primary_host"></a>`primary_host`

Data type: `Peadm::SingleTargetSpec`
Expand Down Expand Up @@ -1860,14 +1879,6 @@ Data type: `Optional[String]`



Default value: `undef`

##### <a name="-peadm--install--stagingdir"></a>`stagingdir`

Data type: `Optional[String]`



Default value: `undef`

##### <a name="-peadm--install--download_mode"></a>`download_mode`
Expand Down Expand Up @@ -2028,14 +2039,15 @@ The following parameters are available in the `peadm::upgrade` plan:
* [`pe_installer_source`](#-peadm--upgrade--pe_installer_source)
* [`final_agent_state`](#-peadm--upgrade--final_agent_state)
* [`r10k_known_hosts`](#-peadm--upgrade--r10k_known_hosts)
* [`stagingdir`](#-peadm--upgrade--stagingdir)
* [`uploaddir`](#-peadm--upgrade--uploaddir)
* [`primary_host`](#-peadm--upgrade--primary_host)
* [`replica_host`](#-peadm--upgrade--replica_host)
* [`compiler_hosts`](#-peadm--upgrade--compiler_hosts)
* [`primary_postgresql_host`](#-peadm--upgrade--primary_postgresql_host)
* [`replica_postgresql_host`](#-peadm--upgrade--replica_postgresql_host)
* [`version`](#-peadm--upgrade--version)
* [`token_file`](#-peadm--upgrade--token_file)
* [`stagingdir`](#-peadm--upgrade--stagingdir)
* [`download_mode`](#-peadm--upgrade--download_mode)
* [`permit_unsafe_versions`](#-peadm--upgrade--permit_unsafe_versions)
* [`begin_at_step`](#-peadm--upgrade--begin_at_step)
Expand Down Expand Up @@ -2100,6 +2112,24 @@ information for hostname, key-type and public key.

Default value: `undef`

##### <a name="-peadm--upgrade--stagingdir"></a>`stagingdir`

Data type: `String`

Directory on the Bolt host where the installer tarball will be cached if
download_mode is 'bolthost' (default)

Default value: `'/tmp'`

##### <a name="-peadm--upgrade--uploaddir"></a>`uploaddir`

Data type: `String`

Directory the installer tarball will be uploaded to or expected to be in
for offline usage.

Default value: `'/tmp'`

##### <a name="-peadm--upgrade--primary_host"></a>`primary_host`

Data type: `Peadm::SingleTargetSpec`
Expand Down Expand Up @@ -2154,14 +2184,6 @@ Data type: `Optional[String]`

Default value: `undef`

##### <a name="-peadm--upgrade--stagingdir"></a>`stagingdir`

Data type: `String`



Default value: `'/tmp'`

##### <a name="-peadm--upgrade--download_mode"></a>`download_mode`

Data type: `Enum[direct,bolthost]`
Expand Down
7 changes: 4 additions & 3 deletions documentation/install.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,24 +119,25 @@ Example params.json Bolt parameters file (shown: Standard):
}
```

Review the [peadm::install plan](../plans/install.pp) to learn about more advanced installation options. For example, it is possible to: supply an ssh private key and git clone URL for a control-repo as part of installation; supply the LDAP configuration data for PE; and similar complete automation tie-ins.
Review the [peadm::install plan](../plans/install.pp) to learn about more advanced installation options. For example, it is possible to: supply an ssh private key and git clone URL for a control-repo as part of installation; supply the LDAP configuration data for PE; specify where the installer tarball is uploaded; and similar complete automation tie-ins.

## Offline usage

The peadm::install plan downloads installation content from an online repository by default. To perform an offline installation, you can prefetch the needed content and place it in the staging directory. If content is available in the staging directory, peadm::install will not try to download it.
The peadm::install plan downloads installation content from an online repository by default. To perform an offline installation, you can prefetch the needed content and place it in the staging directory on the Bolt host. If content is available in the staging directory, peadm::install will not try to download it.

The default staging directory is `/tmp`. If a different staging dir is being used, it can be specified using the `stagingdir` parameter to the peadm::install plan.

The content needed is the PE installation tarball for the target version. The installation content should be in the staging dir, and should have its original name. E.g. `/tmp/puppet-enterprise-2021.4.0-el-7-x86_64.tar.gz`.

Installation content can be downloaded from [https://puppet.com/try-puppet/puppet-enterprise/download/](https://puppet.com/try-puppet/puppet-enterprise/download/).

If you wish to prevent the bolt host from transferring the installer tarball to the targets you can place the installer tar files on the target hosts in the staging directory. This can save time over slow networks, but is best to just perform the bolt task on the target node (puppetserver). The tar file will need to exist in the staging directory for the following configuration:
If you wish to prevent the bolt host from transferring the installer tarball to the targets you can place the installer tar files on the target hosts in the _upload directory_. This can save time over slow networks, but is best to just perform the bolt task on the target node (puppetserver). The default _upload directory_ is `/tmp`. If a different upload dir is being used, it can be specified using the `uploaddir` parameter to the peadm::install plan. With default parameters the tar file will need to exist in the directories for offline configuration:

* /tmp on the Bolt host
* /tmp on the primary
* /tmp on the primary PuppetDB PostgreSQL (if used)
* /tmp on the replica PuppetDB PostgreSQL (if used)

## Online usage

The peadm::install plan can be configured to download installation content directly to hosts. To configure online installation, set the `download_mode` parameter of the `peadm::install` plan to `direct`. The direct mode is often more efficient when PE hosts have a route to the internet.
Expand Down
4 changes: 3 additions & 1 deletion documentation/upgrade.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,14 +38,16 @@ bolt plan run peadm::upgrade --params @params.json

## Offline Usage

The `peadm::upgrade` plan downloads installation content from an online repository by default. To perform an offline installation, you can prefetch the needed content and place it in the staging directory. If content is available in the staging directory, `peadm::upgrade` will not try to download it.
The `peadm::upgrade` plan downloads installation content from an online repository by default. To perform an offline installation, you can prefetch the needed content and place it in the staging directory on the Bolt host. If content is available in the staging directory, `peadm::upgrade` will not try to download it.

The default staging directory is `/tmp`. If a different staging dir is being used, it can be specified using the `stagingdir` parameter to the peadm::upgrade plan.

The content needed is the PE installation tarball for the target version. The installation content should be in the staging dir, and should have its original name. E.g. `/tmp/puppet-enterprise-2019.2.2-el-7-x86_64.tar.gz`.

Installation content can be downloaded from [https://puppet.com/try-puppet/puppet-enterprise/download/](https://puppet.com/try-puppet/puppet-enterprise/download/).

If you wish to prevent the bolt host from transferring the installer tarball to the targets you can place the installer tar files on the target hosts in the _upload directory_. This can save time over slow networks, but is best to just perform the bolt task on the target node (puppetserver). The default _upload directory_ is `/tmp`. If a different upload dir is being used, it can be specified using the `uploaddir` parameter to the peadm::upgrade plan.

Note if `pe_repo` classes have been declared (for agents with different OS/platform than your primary server - see [Install \*nix agents with PE package management](https://www.puppet.com/docs/pe/2021.7/installing_nix_agents.html#install_nix_agents_pe_package_management)), you must also stage the agent repo tarballs for the new PE version before running the `peadm::upgrade` plan, otherwise the plan will fail when it runs puppet to complete the upgrade, as it will not be able to download the agent repos from the internet.

See [Upgrade agents without internet access](https://www.puppet.com/docs/pe/2021.7/upgrading_agents.html#upgrade_agents_without_internet_access) for the location of the agent staging directory on the primary server. From the **Puppet Release Information & Downloads** page linked in the agent upgrade documentation, select the required PE version and navigate to the **Agent Repos** section to download the corresponding agent repo tarball. Ensure the files are prefetched/downloaded from the correct section - the agent repo staging directory expects the agent ___repo___ tarballs (`.tar.gz`), NOT the agent package downloads available from the same page.
Expand Down
8 changes: 8 additions & 0 deletions plans/install.pp
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@
# @param final_agent_state
# Configures the state the puppet agent should be in on infrastructure nodes
# after PE is configured successfully.
# @param stagingdir
# Directory on the Bolt host where the installer tarball will be cached if
# download_mode is 'bolthost' (default)
# @param uploaddir
# Directory the installer tarball will be uploaded to or expected to be in
# for offline usage.
#
plan peadm::install (
# Standard
Expand Down Expand Up @@ -62,6 +68,7 @@

# Other
Optional[String] $stagingdir = undef,
Optional[String] $uploaddir = undef,
Enum['running', 'stopped'] $final_agent_state = 'running',
Enum['direct', 'bolthost'] $download_mode = 'bolthost',
Boolean $permit_unsafe_versions = false,
Expand Down Expand Up @@ -103,6 +110,7 @@

# Other
stagingdir => $stagingdir,
uploaddir => $uploaddir,
download_mode => $download_mode,
permit_unsafe_versions => $permit_unsafe_versions,
token_lifetime => $token_lifetime,
Expand Down
3 changes: 2 additions & 1 deletion plans/subplans/install.pp
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@

# Other
String $stagingdir = '/tmp',
String $uploaddir = '/tmp',
Enum[direct,bolthost] $download_mode = 'bolthost',
Boolean $permit_unsafe_versions = false,
String $token_lifetime = '1y',
Expand Down Expand Up @@ -221,7 +222,7 @@
$pe_tarball_source = "https://s3.amazonaws.com/pe-builds/released/${version}/${pe_tarball_name}"
}

$upload_tarball_path = "/tmp/${pe_tarball_name}"
$upload_tarball_path = "${uploaddir}/${pe_tarball_name}"

if $download_mode == 'bolthost' {
# Download the PE tarball and send it to the nodes that need it
Expand Down
11 changes: 9 additions & 2 deletions plans/upgrade.pp
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,13 @@
# r10k_remote host when using ssh. you must provide \$r10k_known_hosts
# information in the form of an array of hashes with 'name', 'type' and 'key'
# information for hostname, key-type and public key.
#
# @param stagingdir
# Directory on the Bolt host where the installer tarball will be cached if
# download_mode is 'bolthost' (default)
# @param uploaddir
# Directory the installer tarball will be uploaded to or expected to be in
# for offline usage.
#
plan peadm::upgrade (
# Standard
Peadm::SingleTargetSpec $primary_host,
Expand All @@ -48,6 +54,7 @@
# Other
Optional[String] $token_file = undef,
String $stagingdir = '/tmp',
String $uploaddir = '/tmp',
Enum['running', 'stopped'] $final_agent_state = 'running',
Enum[direct,bolthost] $download_mode = 'bolthost',
Boolean $permit_unsafe_versions = false,
Expand Down Expand Up @@ -120,7 +127,7 @@
$pe_tarball_source = "https://s3.amazonaws.com/pe-builds/released/${_version}/${pe_tarball_name}"
}

$upload_tarball_path = "/tmp/${pe_tarball_name}"
$upload_tarball_path = "${uploaddir}/${pe_tarball_name}"

peadm::assert_supported_bolt_version()

Expand Down