support configurable installer target upload path

Upload path was hard-coded to /tmp. Very large files can be problematic
as it is memory backed and typically not limited by default. With ~1.4G
used by the PE installer, unexpected bolt failures on systems with low
to no swap occur (high performance or containers).

Author: h0tw1r3

REFERENCE.md

@@ -1648,6 +1648,8 @@ The following parameters are available in the `peadm::install` plan:
 * [`pe_installer_source`](#-peadm--install--pe_installer_source)
 * [`ldap_config`](#-peadm--install--ldap_config)
 * [`final_agent_state`](#-peadm--install--final_agent_state)
+* [`stagingdir`](#-peadm--install--stagingdir)
+* [`uploaddir`](#-peadm--install--uploaddir)
 * [`primary_host`](#-peadm--install--primary_host)
 * [`replica_host`](#-peadm--install--replica_host)
 * [`compiler_hosts`](#-peadm--install--compiler_hosts)
@@ -1665,7 +1667,6 @@ The following parameters are available in the `peadm::install` plan:
 * [`deploy_environment`](#-peadm--install--deploy_environment)
 * [`license_key_file`](#-peadm--install--license_key_file)
 * [`license_key_content`](#-peadm--install--license_key_content)
-* [`stagingdir`](#-peadm--install--stagingdir)
 * [`download_mode`](#-peadm--install--download_mode)
 * [`permit_unsafe_versions`](#-peadm--install--permit_unsafe_versions)
 * [`token_lifetime`](#-peadm--install--token_lifetime)
@@ -1730,6 +1731,24 @@ after PE is configured successfully.
 
 Default value: `'running'`
 
+##### <a name="-peadm--install--stagingdir"></a>`stagingdir`
+
+Data type: `Optional[String]`
+
+Directory on the Bolt host where the installer tarball will be cached if
+download_mode is 'bolthost' (default)
+
+Default value: `undef`
+
+##### <a name="-peadm--install--uploaddir"></a>`uploaddir`
+
+Data type: `Optional[String]`
+
+Directory the installer tarball will be uploaded to or expected to be in
+for offline usage.
+
+Default value: `undef`
+
 ##### <a name="-peadm--install--primary_host"></a>`primary_host`
 
 Data type: `Peadm::SingleTargetSpec`
@@ -1860,14 +1879,6 @@ Data type: `Optional[String]`
 
 
 
-Default value: `undef`
-
-##### <a name="-peadm--install--stagingdir"></a>`stagingdir`
-
-Data type: `Optional[String]`
-
-
-
 Default value: `undef`
 
 ##### <a name="-peadm--install--download_mode"></a>`download_mode`
@@ -2028,14 +2039,15 @@ The following parameters are available in the `peadm::upgrade` plan:
 * [`pe_installer_source`](#-peadm--upgrade--pe_installer_source)
 * [`final_agent_state`](#-peadm--upgrade--final_agent_state)
 * [`r10k_known_hosts`](#-peadm--upgrade--r10k_known_hosts)
+* [`stagingdir`](#-peadm--upgrade--stagingdir)
+* [`uploaddir`](#-peadm--upgrade--uploaddir)
 * [`primary_host`](#-peadm--upgrade--primary_host)
 * [`replica_host`](#-peadm--upgrade--replica_host)
 * [`compiler_hosts`](#-peadm--upgrade--compiler_hosts)
 * [`primary_postgresql_host`](#-peadm--upgrade--primary_postgresql_host)
 * [`replica_postgresql_host`](#-peadm--upgrade--replica_postgresql_host)
 * [`version`](#-peadm--upgrade--version)
 * [`token_file`](#-peadm--upgrade--token_file)
-* [`stagingdir`](#-peadm--upgrade--stagingdir)
 * [`download_mode`](#-peadm--upgrade--download_mode)
 * [`permit_unsafe_versions`](#-peadm--upgrade--permit_unsafe_versions)
 * [`begin_at_step`](#-peadm--upgrade--begin_at_step)
@@ -2100,6 +2112,24 @@ information for hostname, key-type and public key.
 
 Default value: `undef`
 
+##### <a name="-peadm--upgrade--stagingdir"></a>`stagingdir`
+
+Data type: `String`
+
+Directory on the Bolt host where the installer tarball will be cached if
+download_mode is 'bolthost' (default)
+
+Default value: `'/tmp'`
+
+##### <a name="-peadm--upgrade--uploaddir"></a>`uploaddir`
+
+Data type: `String`
+
+Directory the installer tarball will be uploaded to or expected to be in
+for offline usage.
+
+Default value: `'/tmp'`
+
 ##### <a name="-peadm--upgrade--primary_host"></a>`primary_host`
 
 Data type: `Peadm::SingleTargetSpec`
@@ -2154,14 +2184,6 @@ Data type: `Optional[String]`
 
 Default value: `undef`
 
-##### <a name="-peadm--upgrade--stagingdir"></a>`stagingdir`
-
-Data type: `String`
-
-
-
-Default value: `'/tmp'`
-
 ##### <a name="-peadm--upgrade--download_mode"></a>`download_mode`
 
 Data type: `Enum[direct,bolthost]`

documentation/install.md

@@ -119,24 +119,25 @@ Example params.json Bolt parameters file (shown: Standard):
 }
 ```
 
-Review the [peadm::install plan](../plans/install.pp) to learn about more advanced installation options. For example, it is possible to: supply an ssh private key and git clone URL for a control-repo as part of installation; supply the LDAP configuration data for PE; and similar complete automation tie-ins.
+Review the [peadm::install plan](../plans/install.pp) to learn about more advanced installation options. For example, it is possible to: supply an ssh private key and git clone URL for a control-repo as part of installation; supply the LDAP configuration data for PE; specify where the installer tarball is uploaded; and similar complete automation tie-ins.
 
 ## Offline usage
 
-The peadm::install plan downloads installation content from an online repository by default. To perform an offline installation, you can prefetch the needed content and place it in the staging directory. If content is available in the staging directory, peadm::install will not try to download it.
+The peadm::install plan downloads installation content from an online repository by default. To perform an offline installation, you can prefetch the needed content and place it in the staging directory on the Bolt host. If content is available in the staging directory, peadm::install will not try to download it.
 
 The default staging directory is `/tmp`. If a different staging dir is being used, it can be specified using the `stagingdir` parameter to the peadm::install plan.
 
 The content needed is the PE installation tarball for the target version. The installation content should be in the staging dir, and should have its original name. E.g. `/tmp/puppet-enterprise-2021.4.0-el-7-x86_64.tar.gz`.
 
 Installation content can be downloaded from [https://puppet.com/try-puppet/puppet-enterprise/download/](https://puppet.com/try-puppet/puppet-enterprise/download/).
 
-If you wish to prevent the bolt host from transferring the installer tarball to the targets you can place the installer tar files on the target hosts in the staging directory.   This can save time over slow networks, but is best to just perform the bolt task on the target node (puppetserver).  The tar file will need to exist in the staging directory for the following configuration:
+If you wish to prevent the bolt host from transferring the installer tarball to the targets you can place the installer tar files on the target hosts in the _upload directory_. This can save time over slow networks, but is best to just perform the bolt task on the target node (puppetserver). The default _upload directory_ is `/tmp`. If a different upload dir is being used, it can be specified using the `uploaddir` parameter to the peadm::install plan. With default parameters the tar file will need to exist in the directories for offline configuration:
 
 * /tmp on the Bolt host
 * /tmp on the primary
 * /tmp on the primary PuppetDB PostgreSQL (if used)
 * /tmp on the replica PuppetDB PostgreSQL (if used)
+
 ## Online usage
 
 The peadm::install plan can be configured to download installation content directly to hosts. To configure online installation, set the `download_mode` parameter of the `peadm::install` plan to `direct`. The direct mode is often more efficient when PE hosts have a route to the internet.

documentation/upgrade.md

@@ -38,14 +38,16 @@ bolt plan run peadm::upgrade --params @params.json
 
 ## Offline Usage
 
-The `peadm::upgrade` plan downloads installation content from an online repository by default. To perform an offline installation, you can prefetch the needed content and place it in the staging directory. If content is available in the staging directory, `peadm::upgrade` will not try to download it.
+The `peadm::upgrade` plan downloads installation content from an online repository by default. To perform an offline installation, you can prefetch the needed content and place it in the staging directory on the Bolt host. If content is available in the staging directory, `peadm::upgrade` will not try to download it.
 
 The default staging directory is `/tmp`. If a different staging dir is being used, it can be specified using the `stagingdir` parameter to the peadm::upgrade plan.
 
 The content needed is the PE installation tarball for the target version. The installation content should be in the staging dir, and should have its original name. E.g. `/tmp/puppet-enterprise-2019.2.2-el-7-x86_64.tar.gz`.
 
 Installation content can be downloaded from [https://puppet.com/try-puppet/puppet-enterprise/download/](https://puppet.com/try-puppet/puppet-enterprise/download/).
 
+If you wish to prevent the bolt host from transferring the installer tarball to the targets you can place the installer tar files on the target hosts in the _upload directory_. This can save time over slow networks, but is best to just perform the bolt task on the target node (puppetserver). The default _upload directory_ is `/tmp`. If a different upload dir is being used, it can be specified using the `uploaddir` parameter to the peadm::upgrade plan.
+
 Note if `pe_repo` classes have been declared (for agents with different OS/platform than your primary server - see [Install \*nix agents with PE package management](https://www.puppet.com/docs/pe/2021.7/installing_nix_agents.html#install_nix_agents_pe_package_management)), you must also stage the agent repo tarballs for the new PE version before running the `peadm::upgrade` plan, otherwise the plan will fail when it runs puppet to complete the upgrade, as it will not be able to download the agent repos from the internet.
 
 See [Upgrade agents without internet access](https://www.puppet.com/docs/pe/2021.7/upgrading_agents.html#upgrade_agents_without_internet_access) for the location of the agent staging directory on the primary server. From the **Puppet Release Information & Downloads** page linked in the agent upgrade documentation, select the required PE version and navigate to the **Agent Repos** section to download the corresponding agent repo tarball. Ensure the files are prefetched/downloaded from the correct section - the agent repo staging directory expects the agent ___repo___ tarballs (`.tar.gz`), NOT the agent package downloads available from the same page.

plans/install.pp

@@ -24,6 +24,12 @@
 # @param final_agent_state
 #   Configures the state the puppet agent should be in on infrastructure nodes
 #   after PE is configured successfully.
+# @param stagingdir
+#   Directory on the Bolt host where the installer tarball will be cached if
+#   download_mode is 'bolthost' (default)
+# @param uploaddir
+#   Directory the installer tarball will be uploaded to or expected to be in
+#   for offline usage.
 #
 plan peadm::install (
   # Standard
@@ -62,6 +68,7 @@
 
   # Other
   Optional[String]           $stagingdir             = undef,
+  Optional[String]           $uploaddir              = undef,
   Enum['running', 'stopped'] $final_agent_state      = 'running',
   Enum['direct', 'bolthost'] $download_mode          = 'bolthost',
   Boolean                    $permit_unsafe_versions = false,
@@ -103,6 +110,7 @@
 
     # Other
     stagingdir                     => $stagingdir,
+    uploaddir                      => $uploaddir,
     download_mode                  => $download_mode,
     permit_unsafe_versions         => $permit_unsafe_versions,
     token_lifetime                 => $token_lifetime,

plans/subplans/install.pp

@@ -66,6 +66,7 @@
 
   # Other
   String                $stagingdir             = '/tmp',
+  String                $uploaddir              = '/tmp',
   Enum[direct,bolthost] $download_mode          = 'bolthost',
   Boolean               $permit_unsafe_versions = false,
   String                $token_lifetime         = '1y',
@@ -221,7 +222,7 @@
     $pe_tarball_source = "https://s3.amazonaws.com/pe-builds/released/${version}/${pe_tarball_name}"
   }
 
-  $upload_tarball_path = "/tmp/${pe_tarball_name}"
+  $upload_tarball_path = "${uploaddir}/${pe_tarball_name}"
 
   if $download_mode == 'bolthost' {
     # Download the PE tarball and send it to the nodes that need it

plans/upgrade.pp

@@ -24,7 +24,13 @@
 #   r10k_remote host when using ssh. you must provide \$r10k_known_hosts
 #   information in the form of an array of hashes with 'name', 'type' and 'key'
 #   information for hostname, key-type and public key.
-# 
+# @param stagingdir
+#   Directory on the Bolt host where the installer tarball will be cached if
+#   download_mode is 'bolthost' (default)
+# @param uploaddir
+#   Directory the installer tarball will be uploaded to or expected to be in
+#   for offline usage.
+#
 plan peadm::upgrade (
   # Standard
   Peadm::SingleTargetSpec           $primary_host,
@@ -48,6 +54,7 @@
   # Other
   Optional[String]           $token_file             = undef,
   String                     $stagingdir             = '/tmp',
+  String                     $uploaddir              = '/tmp',
   Enum['running', 'stopped'] $final_agent_state      = 'running',
   Enum[direct,bolthost]      $download_mode          = 'bolthost',
   Boolean                    $permit_unsafe_versions = false,
@@ -120,7 +127,7 @@
     $pe_tarball_source = "https://s3.amazonaws.com/pe-builds/released/${_version}/${pe_tarball_name}"
   }
 
-  $upload_tarball_path = "/tmp/${pe_tarball_name}"
+  $upload_tarball_path = "${uploaddir}/${pe_tarball_name}"
 
   peadm::assert_supported_bolt_version()