(GH-1350) Configure plugins using other plugins

[nicklewis]

Previously, the plugins section of bolt.yaml was required to be
completely static. Now, plugin can use the same _plugin hashes as in
inventory.yaml to define their configuration. For example, this allows
secrets for plugins to be encrypted or read from vault.

We now instantiate (and thus validate) all plugins that are configured in
bolt.yaml at the time the plugin system is initialized.

We also now support _plugin lookups under the plugin_hooks key for
similar reasons.
Closes GH-1350

[nicklewis]

This currently depends on the resolve_references refactor from #1374. I'll rebase once that's merged.

[puppetcla]

CLA signed by all contributors.

[donoghuc]

I think we need to define how we would expect populating the entire section for plugins or plugin_hooks should behave. Currently it seems like for plugins the settings are ignored, and for plugin_hooks parameters retrieved from the plugin_hook in the yaml are attempted to be passed again to the task that implements the plugin.

plugins:
  _plugin: yaml
  filepath: /home/cas/working_dir/bolt/Boltdir/pkcs.yaml
plugin_hooks:
  _plugin: yaml
  filepath: /home/cas/working_dir/bolt/Boltdir/puppet_install.yaml

puppet_install.yaml

puppet_library:
  plugin: puppet_agent
  stop_service: false
  _run_as:
    _plugin: prompt
    message: run as user

pkcs7.yaml

pkcs7:
  private-key: ~/working_dir/dynamic-inventory-demo/keys/public.pkcs7.pem
  public-key: ~/working_dir/dynamic-inventory-demo/keys/private.pkcs7.pem
  keysize: 4096

cas@cas-ThinkPad-T460p:~/working_dir/bolt$ bolt apply -e "notice('hji')" -t damp-quartz.delivery.puppetlabs.net
Task yaml::resolve_reference:
 has no parameter named 'puppet_library'

[nicklewis]

Pushed a new commit to handle setting plugin_hooks with a top-level plugin reference. We now evaluate the plugins first and then merge them over top of the defaults rather than vice versa.

Also squashed in a change to check whether plugins is set with a top-level plugin reference and fail.

[donoghuc]

This looks good. In testing this out I found a couple places where a stacktrace would surface when assuming keys are in a incorrectly formed plugin reference. The simplest example is with a bolt.yaml simply have a key for plugins:

plugins:

But validating user input there does not seem like part of this work.

[donoghuc]

@nicklewis Can you add a CHANGELOG update for this issue?

[nicklewis]

Rebased and updated the changelog