Use custom OIDs for peadm role and avil. group

[reidmv]

Previously, peadm used pp_application and pp_cluster for denoting each node's purpose. This conflicted with some existing customer's usage of those attributes. In order to avoid collision with customer OID usage, this commit modifies peadm to use custom OIDs.

For now, the raw OIDs are used instead of shortnames in order to avoid needing to configure each compiler to know the customer OID-to-shortname mappings. This results in a slight diminishment of user experience in the console, but is fully functional and requires no post-install configuration or maintenance.

This is a breaking change.

TODO: provide a plan to update and re-issue certs on existing installs to use the new OIDs. DONE

Closes #69

[vchepkov]

It seems that with this change peadm_role extension will become mandatory?
I have been using upgrade plan successfully on existing installation, which wasn't created with peadm module. Would it be possible to make this check configurable?

Thanks

[reidmv]

@vchepkov you are correct, the new OIDs will be required to use the (imminent) 1.x version of peadm to upgrade a deployment created with an older version.

Some utilities are provided to make this doable, if likely still arduous. See this doc for details.

[reidmv]

@bwilcox prepping this PR as the last before calling peadm 1.0.0. I believe this resolves the problems of

• Overwriting customer's csr_attributes.yaml file; peadm should now simply merge in the values it needs
• Collision with customer use of pp_application or pp_cluster; peadm now uses its own, custom OIDs

If you get a chance would like to hear your thoughts!

[vchepkov]

Seems unnecessary in standard two node installation, don't you think?

[vchepkov]

Also, it doesn't look like upgrade_trusted_facts plan handles case where existing nodes don't have any extensions currently, plan doesn't have an option to supply a role for the target

[vchepkov]

I tried to convert a standalone node, to see how it works.
I ended up with empty attributes:

# cat /etc/puppetlabs/puppet/csr_attributes.yaml 
---
extension_requests:
  1.3.6.1.4.1.34380.1.1.9812: 
  1.3.6.1.4.1.34380.1.1.9813: 

also:

• upgrade_trusted_facts plan needs to run agent once to fix the services
• custom_trusted_oid_mapping.yaml should be deployed on the master

[reidmv]

Seems unnecessary in standard two node installation, don't you think?

Maybe, but in this case I think we need to preserve the requirement in order to homogenize environmental expectations in anticipation of future development and use cases. After this change the module will be elevated to 1.0.0 status, subject to SemVer versioning requirements, and the leeway to make breaking changes will diminish.

The module design's errs on the side of "sameness" and simplicity of process, rather than minimalism of output. Each plan aims to run as straightforwardly as possible, without major conditional forking, regardless of which architecture is in use. The upgrade plan is the only plan currently to use some dynamism in determining system roles automatically, but early versions of the module made much heavier use of it and may again in the future. If roles cannot be determined confidently from core requirements, that will result in more complex, more difficult to develop and maintain workflow automation actions.

We need both role and group trusted facts for the Extra Large architecture, which is the primary focus of this module, and so in service to "sameness" Standard and Large deployment roles created by this module should be identified the same way.

It doesn't look like upgrade_trusted_facts plan handles case where existing nodes don't have any extensions currently, plan doesn't have an option to supply a role for the target

All infrastructure deployed by peadm must have trusted facts today. If it does not, it was not deployed by peadm, and the peadm module is not designed to manage it. That's a base requirement we can rely on, which simplifies the code written in workflow automation actions. This is an example of why I'm pushing hard to preserve sameness; to preserve the ability not to account for or accommodate edge cases or variations in workflow code.

The peadm::misc:: upgrade_trusted_facts plan exists to make a bridge onto 1.0.0, but as a one-time-per-user action I don't think it's going to get much more investment unless it's found to have a bug or doesn't function as needed. In the intended workflow, it should be as simple as:

1. Try to upgrade using 1.0.0; get error message
2. Run peadm::misc:: upgrade_trusted_facts once, against all PE infrastructure targets
3. Per the documentation, re-run the peadm::setup::node_manager class to update the rules to use peadm-specific OIDs. Technically this is optional; the old groups will still work, since all the nodes still have their old pp_application and pp_cluster extensions in addition to the new peadm specific ones
4. Finish by running the upgrade, without error

I converted a standalone node, to see how it works. It seems upgrade_trusted_facts plan needs to run agent once to fix the services, and also to deploy custom_trusted_oid_mapping.yaml on the master

This sounds interesting. I've just finished testing on an Extra Large cluster and it works there... can you clarify the result here? Did the plan run to completion on the standalone node, or did it fail at some step?

[vchepkov]

I understand your concerns. But since module doesn't support deploying PE LTS version (puppet 5), there are got to be lots of installations out there built without peadm module, so bringing them up into 'compliance' to use upgrade plan would be nice.
I can try to see what can be done to convert systems without extensions and report back, if you are interested.

[reidmv]

Ah, that's interesting. Conversion, or adoption, of deployments not originally created using peadm is not a use case I've been considering.

For this iteration scope is limited to making sure 0.5.x deployments can be converted to 1.x deployments. However, especially now that adding cert extensions has been figured out with peadm::misc::upgrade_trusted_facts, it would be pretty straightforward to write an actual peadm::convert plan.

Are you currently using the upgrade plan on systems not deployed with peadm?

[reidmv]

I'm going to explore creating a full convert plan for this supporting both upgrade from 0.5.x to 1.x, as well as adoption of infrastructure not originally deployed with peadm. Some commits incoming, may not be stable/tested for a bit.

[vchepkov]

Yes, I use upgrade plan, even on a single monolithic servers, because it's very convenient.
I will try your new plan today and report back.

I have mentioned custom_trusted_oid_mapping.yaml before, I tried it myself and it work great

# cat /etc/puppetlabs/puppet/custom_trusted_oid_mapping.yaml 
---
oid_mapping:
  1.3.6.1.4.1.34380.1.1.9812:
    shortname: peadm_role
    longname: 'PE node role'
  1.3.6.1.4.1.34380.1.1.9813:
    shortname: peadm_availability_group
    longname: 'PE HA group'

Then you can use human-frendly names in the classification:

# puppet resource pe_node_group 'PE Master A'
pe_node_group { 'PE Master A':
  ensure             => 'present',
  environment        => 'production',
  environment_trumps => false,
  id                 => '769bc7cd-0560-4832-8112-5897abd1973c',
  parent             => 'All Nodes',
  provider           => 'ruby',
  rule               => ['and',
  ['=',
    ['trusted', 'extensions', 'peadm_availability_group'],
    'A'],
  ['=',
    ['trusted', 'extensions', 'peadm_role'],
    'puppet/master']],
}

[vchepkov]

So, I was not able to use new plan, unfortunately, because it is using puppet ssl command, which is not available on PE2018.

For standard architecture, though I was able to use a PE plan, I just created

/etc/puppetlabs/puppet/csr_attributes.yaml 
---
extension_requests:
  1.3.6.1.4.1.34380.1.1.9812: puppet/master
  1.3.6.1.4.1.34380.1.1.9813: A

and run command puppet infrastructure run regenerate_master_certificate

That's it conversion is done :), since you really don't need any additional classification groups here

[reidmv]

That's great! Yeah, peadm has not supported 2018.1.x since the 0.4.x branch; there is an upgrade plan in the 0.4.x versions too though that will continue to support 2018.1. The path to the next LTS will likely be to use 0.4.x to upgrade the 2018.1 system(s) to the next LTS, then convert them, then use the 1.x and newer versions of peadm moving forward.

Thanks for all your feedback on this @vchepkov! It resulted in the peadm::convert plan which is a better user experience than the peadm::misc::upgrade_trusted_facts would have been.