Is Debian a second class operating system for Puppet?

[bastelfreak]

For the past years it was always a problem to get packages for Debian. Debian 12 got released in June last year. The puppet-agent package got published a few weeks ago. Bolt, PDK, PuppetDB, Puppetserver are still missing.

Why?

And I asked on slack about PuppetDB and Puppetserver and why they weren't published together with the Agent. The response was basically and all three are maintained by different teams and the last effort was only about the agent. PuppetDB and Puppetserver have to to requested separately. As a user that's really not a sufficient answer. The Debian release dates aren't a secret. Debian even communicates them in advance and provides beta/rc images so people can work early on supporting them.

• Debian 12 agent was published around 9 months after Debian 12 release
• CentOS Stream 9 support was published as puppet agent 7.14, released in January 2022, 4 month after CentOS Stream 9 release (which I think is already quite long)

Also there's a serious packaging bug for Debian packages. It is known since years but nobody cares to fix it. That just makes upgrades so so complicated. It was reported in 2019: https://puppet.atlassian.net/browse/CPR-699. And it's still a problem: https://puppet.atlassian.net/browse/CPR-861 / https://puppet.atlassian.net/browse/PA-4995.

And one last thing: Multiple times I heard from Puppet employees

Debian/Ubuntu do not have a high priority because we don't see many users

• How do you measure this?
• How do you expect to gain any Debian/Ubuntu users when they are treated like this? You won't get users when you don't offer packages.

[bastelfreak]

Another reason why it feels like Debian is a second class OS: puppet-release-bookworm.deb is missing at https://apt.puppet.com/.

[joshcooper]

There are stream specific release packages for bookworm at:

https://apt.puppet.com/puppet7-release-bookworm.deb
https://apt.puppet.com/puppet8-release-bookworm.deb

I think there's some debate as to whether having release packages (like puppet-release-bookworm.deb) that install the latest major version are a good idea. Pretty sure, that'll lead to a "bad time".

[evgeni]

Sure, but it's inconsistent when some Debian/Ubuntu versions have it, and some not.

[bastelfreak]

I think there's some debate as to whether having release packages

probably. But it was never communicated that those packages won't be provided anymore and now it's inconsistent. And beaker has a function to test against latest puppet. In that case it will use those deb/rpm packages to get the latest major version. I don't mind it if they disappear, but then it would be nice if that information is published anywhere.

[binford2k]

@joshcooper is actually working on a build system change that should make it easier for you to build packages for platforms we haven't gotten to yet (Arch, even 😉), and even to contribute those upstream. It's too early to promise anything, it might not even end up working out. But watch for improvements soon!

[Valantin]

On my organization we are migrating all to ubuntu 22.04 and debian 12 to be compliant with our IT resource usage policy but we are stuck with the puppetserver because of this latency in the release.

Can I help in some way to do a bust to release?

[tuxmea]

Any update or feedback? @davidsandilands

[ekohl]

Note that Debian 11 standard support ends in about 2 months. This is a blocker to upgrade

[davidsandilands]

@GSPatton would you have an update for PuppetDB/Puppetserver ?

[GSPatton]

Yes, we have made Debian 12 support a priority and it will be worked on in Q3. Sorry for the delay in this, we appreciate your patience as we make our way through the backlog!

[bastelfreak]

Thanks a lot for the update! Can we track that effort anywhere so we don't have to keep asking?

as we make our way through the backlog

Is that backlog public anywhere? Or accessible for partners?

[alaunay]

Can you right now add Debian 13 (trixie), due around mid-2015, to the backlog, so this nightmare scenario does not reproduce ?

[smortex]

Adding more info about consumers of Puppet / PuppetServer / PuppetDB beyond organizations that use Puppet for configuration management:

1. People who work on modules had to delay adding support for Debian 12 to their modules until the AIO package was available. With Debian providing a recent Puppet Agent package when Debian 12 was released made a lot of people requesting support for Debian 12 for modules they used, and we (speaking as @voxpupuli and @opus-codium) preferred delaying this support rather than providing a support that could possibly break when AIO packages are available. This was not great for maintainers of the modules nor for consumers of these modules. Now that Debian 12 packages are available, the situation is better, but a few modules still have not had the needed support for Debian 12 they deserve yet, so we are still "recovering";
2. People who work on modules also sometimes need a full infra for running CI, this is notably the case for @voxpupuli's https://github.com/voxpupuli/puppet-puppetboard which is still blocked today (Add Debian 12 support voxpupuli/puppet-puppetboard#396)
3. Third parties around the Puppet ecosystem also want "official" packages for Debian 12 before claiming they support Debian 12. Speaking as a member of @choria-io on this one.

So, as far as I am concerned I work or contribute to 7 different entities which are waiting for a complete Debian 12 package set. Or said differently, all people I work with in the Puppet world are waiting for Debian 12 packages for some aspect of their work.

[evgeni]

Same for @theforrman - we can't ship Foreman on Debian 12 until there are AIO packages for agent (those exist now, but came late) and server (still mia).

[tuxmea]

Some more feedback:

1. Foreman: The Foreman community is waiting for Debian 12 Puppet Server packages.
2. A customer (large German bank) migrated to Debian 12 using the Debian provided packages. Customer left a clear statement: "Why should we buy PE if Puppet is unable to provide packages for a widely used Linux distribution?"

Most customers which run Debian as their main OS are disappointed and look for other solutions.
Puppet is successfully destroying its community of users.

[swantzter]

I too manage Debian based servers with puppet, I use both puppet and Debian for the same reason: to have a stable and predictable environment. I want to be able to focus more on running business software than to fiddle around with my servers, package updates, and config changes. That's why I want a rock solid foundation.

The fact that puppetserver and puppetdb still isn't available for Debian 12 is to me a direct opposite to the reasons one would use puppet in the first place, which I feel is unfortunate.

[smoeding]

The community seems to be a good point here.

Maybe product managers at Perforce only look at the numbers for Puppet Enterprise because in the end only that creates revenue.

But without a decent support for the common open source systems like Debian the product might loose its value when the community stops providing modules and product bug fixes. In the end that will hurt PE as well since a product not delivering the business value for the client will be kicked rather quickly. Management doesn't care about the name of the tool in use.

[davidsandilands]

I want to assure you this isn't the case and particular work has gone in across the products to look at finding the right balance and priority. We obviously can't defend the length of time its taken to get to Debian 12 for server finally being prioritized in this quarter but the work Gavin and team have put in to improve the pipelines and catching up on agents first then following up on server should ensure in future we won't be so far behind and will be aware of what we need to do.

[tuxmea]

I still wonder why Debian is not a PE supported platform.
In Germany we have many customers using Debian and moving them to PE is impossible as they want to stay with Debian.
One customer recently moved from Puppet Inc to Debian packages for puppet agent and server.
But now they are disconnected from any further update from Puppet Inc and again they see no value in using PE.

Product management at Puppet is refusing to take money, which customers would pay.

[davidsandilands]

I think this has often been a dilemma that gets reviewed periodically of how many server and agent OS we support. S390x RedHat agent is another common ask but it was found not to have the demand for PE when it was made supported. With RedHat looking to essentially end RHEL clones the outlook may be very different. Do let @GSPatton know about customers who are asking or equally for any of the other tooling / products let me and @binford2k know.

[bastelfreak]

My impression is that customers first start with open source and eventually switch to PE. And we cannot go that path without reliable open source packages for Debian. And customers communicated clearly to us that they lost faith into PE and sometimes into Puppet as a whole, because of the "volatile" Debian support.

[kenyon]

My impression is that customers first start with open source and eventually switch to PE.

Can confirm, that's the path we took too.

[brechsteiner]

My company would have even bought a subscription for the community version if we had gotten the Debian packages for it. But Puppetlabs was not interested in us as a customer.

[tuxmea]

Please also start providing nightlies for Debian testing.

[davidsandilands]

@tuxmea sorry to clarify when you say "Debian testing" do you mean https://wiki.debian.org/DebianTesting / Debian 13

[smortex]

I am pretty sure @tuxmea is asking for nightly puppetserver / puppetdb packages in order to test them before they are officially out for Debian 12 (stable) 😉

Debian packages for Debian testing may be useful to some people, but Debian testing (and sid) is not intended for production and I know no-one using it for production (I only know Debian developers using sid/testing for their day-to-day work and from time to time they have to deal with some trouble).

[tuxmea]

@davidsandilands Yes. Debian Testing = the next upcoming stable release of Debian (13 at the moment, or - to be more precise: trixie). See https://www.debian.org/releases/

[tuxmea]

Offering Puppetserver/PuppetDB nightlies for trixie would allow people to test functionality prior upgrading to trixie

[davidsandilands]

@GSPatton you able to comment when we would expect ?

[brechsteiner]

First we are blocked from upgrading to Debian 12 due to the missing Puppet Agent packages for Debian, now Debian 11 is entering the Long Term Support (LTS) phase and we are blocked from upgrading our Debian 11 servers to Debian 12 due to the missing Puppet Server and PuppetDB packages.

Hey @davidsandilands will there be Puppetserver packages for Debian 12 someday?

[davidsandilands]

Hey @brechsteiner there will be Debian 12 packages but we are waiting for confirmation of a committed engineering date.

[brechsteiner]

@davidsandilands Debian 12 was released on June 10, 2023. You must have mixed it up with Debian 13?

[brechsteiner]

@davidsandilands ?

[ekohl]

What I think @davidsandilands tried to say was that it's being worked on, but they can't commit to a date when it'll be ready.

[brechsteiner]

ah okay I misunderstood that, I just wanted to send a reference to your post in the foreman forum that you would also be happy about the packages

[zipkid]

I don't know if i should just ask, plead, cry, weep, SCREAM.... !
Is there ANYTHING happening for this?

[davidsandilands]

@GSPatton any idea if there is a committed date or progress known?

[smoeding]

Snapshot builds for Puppetserver and PuppetDB have appeared in the bookworm nightly repo: http://nightlies.puppetlabs.com/apt/pool/bookworm/puppet8-nightly/p/index.html

Kudos to the engineering team! 👏

[kenyon]

Just upgraded my home puppetserver to Debian 12 and used the latest nightly packages. Working for me!

[davidsandilands]

Hey all I wanted to just confirm that as this in nightlies assuming no bugs Debian 12 Puppet Server, PuppetDB this should be released in the October release

[bastelfreak]

That's great!
Is there a chance we can get Debian 13 nightlies as well? The stable release is in roughly 6-7 months, but the testing packages are already available.

[davidsandilands]

I thought you might say that at the risk of losing any credibility leave it with me and I will get back here

[brechsteiner]

https://groups.google.com/g/puppet-announce/c/5oxWmczZDf8/m/veazKaDhAAAJ

[davidsandilands]

The nightlies have stopped with the announcement of https://www.puppet.com/blog/open-source-puppet-updates-2025 and beyond any potential major vulnerabilities the only packaging to be released by Perforce/Puppet will be in the private/EULA'd area. So I would see the new package pipeline the community with @binford2k @nmburgan and others have been working on to be the place where Debian can be better priortised

Assuming there is no disagreement here I think it would make sense to close this discussion.

[alaunay]

Where are those community packages ?

[zipkid]

https://overlookinfratech.com/downloads/

[alaunay]

Thanks !
I just hope this will be clearly documented with a visible link in the puppet docs.

[bastelfreak]

I think we had enough discussions here about this topic and can close it.