Update module to support puppetcore on Windows #766

joshcooper · 2025-03-20T00:12:01Z

This is analogous to #757 but for windows.

Updates the puppet_agent::install_task so it's possible to install msi-based packages from https://artifacts-puppetcore.puppet.com. Credentials are required when installing from puppetcore. The username defaults to forge-key and the password must be set to your forge API token.

When using WinRM, you must specify Windows credentials (using --user and --password arguments) to log into the host and puppetcore credentials to download the MSI (as username and password task parameters)

$ /opt/puppetlabs/bolt/bin/bolt task run puppet_agent::install \
  collection=puppetcore8 \
  version=8.11.0 \
  username=forge-key \
  password=... \
  --targets 'winrm://HOST' \
  --user Administrator \
  --password ...

Also updates the puppet_agent class so it's possible to manage agent versions over time:

class { 'puppet_agent':
  package_version => '8.11.0',
  collection => 'puppetcore8',
  password => Sensitive(...)
}

The puppet_agent::prepare::package class on Windows normally uses a file resource to download the MSI from the source. However, due to a bug in puppet, it is not possible to pass credentials as the userinfo component of the URL, e.g. https://forge-id:[email protected] So instead use a powershell script to download the MSI. That code is borrowed from https://github.com/klab-systems/puppet_core_agent, credit to @klab-systems

manifests/prepare/package.pp

cthorn42 · 2025-03-31T20:17:11Z

Wanted to discuss the old 'puppet8-nightly' collection. With the new layout do we still require a different collection? Or can we get by with just always using puppetcore8, and then just adding logic to determine if we should set the dev parameter to true or false depending on the package version?
I think we already have ticketed the updating of https://github.com/puppetlabs/puppetlabs-puppet_agent/blob/14b1844f60a225387157c002c01c8c9d983c6fa2/acceptance/tests/test_upgrade_puppet7_to_puppet8.rb#L28C25-L28C40, and that is the only obvious usage of that puppet8-nightly collection.

joshcooper · 2025-04-01T04:43:58Z

Or can we get by with just always using puppetcore8, and then just adding logic to determine if we should set the dev parameter to true or false depending on the package version?

The dev parameter is mainly for PE CI, so that it can test agent upgrades to a version that's newer than what it ships with. It's not a direct replacement for puppetcore8-nightlies because the last passing agent SHA isn't always promoted to PE.

Beaker tests like test_upgrade_pouppet7_to_puppet8 run in our internal jenkins, so can already install from our internal nightly repos. I think d we're already overriding the yum_source, etc to point to artifactory. I'm not sure if we need to change the collection in the test? I can't access jenkins because of the lame Jenkins plugin/SAML bug, "You are now logged out of Jenkins, however this has not logged you out of SAML."

manifests/prepare/package.pp

tasks/install_powershell.json

cthorn42 · 2025-04-03T21:07:36Z

templates/download_puppet.ps1.epp

+    "os_version" = "<%= $facts['os']['release']['major'] %>"
+    "os_arch" = "<%= $facts['os']['architecture'] %>"
+    "fips" = "<%= $facts['fips_enabled'] %>"
+}


Attempts to install a nightly dev build is failing, failed when version=8.11.0.22.gb1c13c900 is set:

Started on winrm://hot-connection.delivery.puppetlabs.net... Failed on winrm://hot-connection.delivery.puppetlabs.net: The task failed with exit code 1 Downloading the Puppet Agent installer on HOT-CONNECTION from https://artifacts-puppetcore.puppet.com/v1/download?version=8.11.0.22.gb1c13c900&os_name=windows&os_version=10&os_arch=x64&fips=false Download exception: The remote server returned an error: (404) Not Found. Retrying... [1/5] Download exception: The remote server returned an error: (404) Not Found. Retrying... [2/5] Download exception: The remote server returned an error: (404) Not Found. Retrying... [3/5] Download exception: The remote server returned an error: (404) Not Found. Retrying... [4/5] Download exception: The remote server returned an error: (404) Not Found. Retrying... [5/5] Download exception: The remote server returned an error: (404) Not Found.

Install

❯ /opt/puppetlabs/bolt/bin/bolt task run --no-ssl-verify --modulepath /home/josh/work/modules.install puppet_agent::install collection=puppetcore8 version=8.11.0.60.g0b032a12d --user Administrator --password '...' --targets winrm://richest-quality.delivery.puppetlabs.net ... Running task 'puppet_agent::install' on winrm://richest-quality.delivery.puppetlabs.net Started on winrm://richest-quality.delivery.puppetlabs.net... ... Finished on winrm://richest-quality.delivery.puppetlabs.net: Downloading the Puppet Agent installer on RICHEST-QUALITY from https://artifacts-puppetcore.puppet.com/v1/download?version=8.11.0.60.g0b032a12d&os_name=windows&os_version=10&os_arch=x64&fips=false&dev=true Installing the Puppet Agent on RICHEST-QUALITY... Deleting C:\Users\Administrator\AppData\Local\Temp\puppet-agent-x64.msi and C:\Users\Administrator\AppData\Local\Temp\2025-04-18T01-48-21-puppet-install.log Puppet Agent installed on RICHEST-QUALITY Finished: task puppet_agent::install with 0 failures in 76.29 sec Successful on 1 target: winrm://richest-quality.delivery.puppetlabs.net Ran on 1 target in 1 min, 16 sec

Upgrade

C:\> type upgrade.pp class { 'puppet_agent': package_version => "8.11.0.49.g2057f249c", service_names => [], collection => "puppetcore8", username => "forge-key", password => Sensitive("..."), } C:\> puppet apply --debug upgrade.pp ... Notice: /Stage[main]/Puppet_agent::Prepare::Package/File[C:\Users\ADMINI~1\AppData\Local\Temp\1\download_puppet.ps1]/content: changed [redacted] to [redacted] Debug: /Stage[main]/Puppet_agent::Prepare::Package/File[C:\Users\ADMINI~1\AppData\Local\Temp\1\download_puppet.ps1]: The container Class[Puppet_agent::Prepare::Package] will propagate my refresh event Debug: /Stage[main]/Puppet_agent::Prepare::Package/Exec[Download Puppet Agent]/creates: Checking that 'creates' path 'C:\ProgramData\Puppetlabs\packages\puppet-agent-8.11.0.49.g2057f249c-x64.msi' exists Debug: Exec[Download Puppet Agent](provider=windows): Executing '["C:\\Windows\\system32\\WindowsPowerShell\\v1.0\\powershell.exe", "-ExecutionPolicy", "Bypass", "-NoProfile", "-NoLogo", "-NonInteractive", "C:\\Users\\ADMINI~1\\AppData\\Local\\Temp\\1\\download_puppet.ps1"]' Debug: Executing: 'C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -NoProfile -NoLogo -NonInteractive C:\Users\ADMINI~1\AppData\Local\Temp\1\download_puppet.ps1' Notice: /Stage[main]/Puppet_agent::Prepare::Package/Exec[Download Puppet Agent]/returns: executed successfully ... Notice: /Stage[main]/Puppet_agent::Install::Windows/Exec[prerequisites_check.ps1]/returns: executed successfully ... Debug: Exec[install_puppet.ps1](provider=windows): Executing 'C:\Windows\system32\cmd.exe /S /c start /b C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Bypass -NoProfile -NoLogo -NonInteractive -Command C:\Users\ADMINI~1\AppData\Local\Temp\1\install_puppet.ps1 -PuppetPID 6480 -Source 'C:\ProgramData\Puppetlabs\packages\puppet-agent-8.11.0.49.g2057f249c-x64.msi' -Logfile 'C:\Users\ADMINI~1\AppData\Local\Temp\1\puppet-2025_04_18-02_26-installer.log' -InstallDir '' -PuppetMaster 'puppet' -PuppetStartType '' -InstallArgs 'REINSTALLMODE="""amus"""' ' ... Notice: /Stage[main]/Puppet_agent::Install::Windows/Exec[install_puppet.ps1]/returns: executed successfully ... C:\ProgramData\PuppetLabs>type "c:\Program Files\Puppet Labs\Puppet\VERSION" 8.11.0.49.g2057f249c

And the upgrade is idempotent:

C:\ProgramData\PuppetLabs>puppet apply upgrade.pp Notice: Compiled catalog for richest-quality.delivery.puppetlabs.net in environment production in 0.42 seconds Notice: Applied catalog in 0.05 seconds

Now possible to run the install task specifying puppetcore collection: ``` /opt/puppetlabs/bolt/bin/bolt task run puppet_agent::install \ collection=puppetcore8 \ version=8.11.0 \ username=forge-key \ password=${PUPPET_FORGE_TOKEN} \ --targets 'winrm://HOST' \ --user Administrator \ --password ... ``` If the `windows_source` class parameter is explicitly given, then the task will use that. Also add additional logging as to where we are downloading the MSI from and the exception message if downloading fails.

bastelfreak · 2025-04-16T22:24:38Z

manifests/prepare/package.pp

-    source  => $source,
-    require => File[$puppet_agent::params::local_packages_dir],
+  if $puppet_agent::collection and $puppet_agent::collection =~ /core/ and $facts['os']['family'] =~ /windows/ {
+    $download_username = getvar('puppet_agent::username', 'forge-key')


why do you use getvar, instead of accessing the variable directly?

I was mainly following the getvar convention used in this module.

manifests/prepare/package.pp

When using the puppetcore collection on Windows, if we detect the installed version does not match, then upgrade the MSI. Due to a puppet bug, we cannot pass credentials in the `source` parameter. And `curl.exe` is not present in our puppet-agent packages. So use powershell to download. Co-authored-by: Kevin <[email protected]>

If `$puppet_agent::collection` was undef, then the regex match would result in an error: $ bundle exec puppet apply -e '$foo = undef if $foo =~ /core/ { }' Error: Evaluation Error: Left match operand must result in a String value. Got an Undef Value.

Use '@api private' for private classes, so they are excluded from REFERENCES.md Use 'String[1]' for username parameter. If it is specified, then it should not be empty. Use 'Sensitive[String[1]]' for password parameter. Use ruby-style way of conditionally setting a variable.

Dev builds have more than 3 dotted components

joshcooper changed the title ~~Private windows~~ Update module to support puppetcore on Windows Mar 20, 2025

cthorn42 reviewed Mar 21, 2025

View reviewed changes

manifests/prepare/package.pp Show resolved Hide resolved

cthorn42 reviewed Mar 26, 2025

View reviewed changes

manifests/prepare/package.pp Outdated Show resolved Hide resolved

joshcooper force-pushed the private_windows branch 3 times, most recently from a57e338 to 11b51fb Compare March 31, 2025 18:25

joshcooper marked this pull request as ready for review March 31, 2025 19:06

joshcooper requested review from bastelfreak and a team as code owners March 31, 2025 19:06

cthorn42 mentioned this pull request Apr 1, 2025

Add puppetcore macos support #769

Open

joshcooper commented Apr 2, 2025

View reviewed changes

manifests/prepare/package.pp Outdated Show resolved Hide resolved

joshcooper commented Apr 2, 2025

View reviewed changes

manifests/prepare/package.pp Show resolved Hide resolved

joshcooper commented Apr 2, 2025

View reviewed changes

manifests/prepare/package.pp Outdated Show resolved Hide resolved

joshcooper commented Apr 2, 2025

View reviewed changes

tasks/install_powershell.json Outdated Show resolved Hide resolved

cthorn42 reviewed Apr 3, 2025

View reviewed changes

joshcooper force-pushed the private_windows branch from 852b037 to a0e3944 Compare April 16, 2025 22:20

bastelfreak reviewed Apr 16, 2025

View reviewed changes

manifests/prepare/package.pp Outdated Show resolved Hide resolved

joshcooper and others added 4 commits April 17, 2025 19:19

Add support for development MSI builds

3f27ba3

Dev builds have more than 3 dotted components

joshcooper force-pushed the private_windows branch from a0e3944 to 3f27ba3 Compare April 18, 2025 02:21

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update module to support puppetcore on Windows #766

Update module to support puppetcore on Windows #766

joshcooper commented Mar 20, 2025

cthorn42 commented Mar 31, 2025

joshcooper commented Apr 1, 2025 •

edited

Loading

cthorn42 Apr 3, 2025

joshcooper Apr 18, 2025

bastelfreak Apr 16, 2025

joshcooper Apr 18, 2025

Update module to support puppetcore on Windows #766

Are you sure you want to change the base?

Update module to support puppetcore on Windows #766

Conversation

joshcooper commented Mar 20, 2025

cthorn42 commented Mar 31, 2025

joshcooper commented Apr 1, 2025 • edited Loading

cthorn42 Apr 3, 2025

Choose a reason for hiding this comment

joshcooper Apr 18, 2025

Choose a reason for hiding this comment

Install

Upgrade

bastelfreak Apr 16, 2025

Choose a reason for hiding this comment

joshcooper Apr 18, 2025

Choose a reason for hiding this comment

joshcooper commented Apr 1, 2025 •

edited

Loading