feat: Verbose logging with flag

[jkayani]

Description

Adds a verbose output flag so that users can troubleshoot AVP usage easier. Goal is to be able to trace:

• The final configuration Viper sees
• The processing being done for each placeholder
• API calls and responses from backends for both authentication and getting secrets

Assuming this looks worthwhile, I think it'd be good for bug reports to include this info to help us debug what's happening. This PR amends the issue template to say that.

Fixes: #307

Checklist

Please make sure that your PR fulfills the following requirements:

• Reviewed the guidelines for contributing to this repository
• The commit message follows the Conventional Commits Guidelines.
• Tests for the changes have been updated
• Are you adding dependencies? If so, please run go mod tidy -compat=1.17 to ensure only the minimum is pulled in.
• Docs have been added / updated
• Optional. My organization is added to USERS.md.

Type of Change

• Bugfix
• Feature
• Code style update (formatting, local variables)
• Refactoring (no functional changes, no api changes)
• New tests
• Build/CI related changes
• Documentation content changes
• Other (please describe)

Other information

[jkayani]

Sample output from stderr after running AVP on a template pulling secrets from IBM Cloud SM:

2022/06/02 00:42:42 reading configuration from config file /Users/jkayani/Documents/ops/argo/vault/ibm-sm-creds.yaml, overriding any previous settings
2022/06/02 00:42:42 reading configuration from environment, overriding any previous settings
2022/06/02 00:42:42 AVP configured with the following settings:
2022/06/02 00:42:42 avp_type: ibmsecretsmanager

2022/06/02 00:42:42 avp_ibm_api_key: REDACTED

2022/06/02 00:42:42 avp_ibm_instance_url: REDACTED

2022/06/02 00:42:42 avp_kv_version: 2

2022/06/02 00:42:42 IBM Cloud Secrets Manager enabling 3 API call retries with 20 seconds between tries
2022/06/02 00:42:42 IBM Cloud Secrets Manager listing secrets of from group default starting at offset 0
2022/06/02 00:42:44 IBM Cloud Secrets Manager list secrets in group HTTP response: {
    REDACTED
}

2022/06/02 00:42:44 IBM Cloud Secrets Manager get secret: getting secret beep of type arbitrary from group default
2022/06/02 00:42:44 IBM Cloud Secrets Manager get unversioned secret REDACTED HTTP response: {
    REDACTED
}

2022/06/02 00:42:44 calling GetSecrets to get all secrets from backend because avp.kubernetes.io/path is set to ibmcloud/arbitrary/secrets/groups/default
2022/06/02 00:42:44 found placeholder beep with modifiers []
2022/06/02 00:42:44 found placeholder path:ibmcloud/imported_cert/secrets/groups/default#certs with modifiers [ jsonPath {.certificate}]
2022/06/02 00:42:44 calling GetIndividualSecret for secret certs from path ibmcloud/imported_cert/secrets/groups/default at version 
2022/06/02 00:42:44 IBM Cloud Secrets Manager list secrets in group: cache hit group default
2022/06/02 00:42:44 IBM Cloud Secrets Manager get secret: getting secret certs of type imported_cert from group default
2022/06/02 00:42:45 IBM Cloud Secrets Manager get unversioned secret REDACTED HTTP response: {
    REDACTED
}

2022/06/02 00:42:45 processing modifier jsonPath with args ["jsonPath" "{.certificate}"]

Some of the output is out of order, probably b/c of the goroutines being used for this particular backend. It's still mostly traceable though especially with the original template.

[jkayani]

Using the global singleton Viper here since the verbose output setting isn't specific to any backend or even any part of the code (it's used when doing auth, replacements in the template, and when retrieving from backends)

[codecov-commenter]

Codecov Report

Merging #350 (16624e9) into main (81b61bb) will increase coverage by 0.84%.
The diff coverage is 87.69%.

@@            Coverage Diff             @@
##             main     #350      +/-   ##
==========================================
+ Coverage   76.68%   77.53%   +0.84%     
==========================================
  Files          22       22              
  Lines         948     1006      +58     
==========================================
+ Hits          727      780      +53     
- Misses        137      142       +5     
  Partials       84       84              

Impacted Files	Coverage Δ
pkg/kube/client.go	0.00% <0.00%> (ø)
pkg/utils/util.go	60.78% <0.00%> (-2.49%)	⬇️
pkg/auth/vault/approle.go	68.42% <66.66%> (+3.71%)	⬆️
pkg/auth/vault/github.go	64.70% <66.66%> (+4.70%)	⬆️
pkg/auth/vault/kubernetes.go	64.70% <66.66%> (+2.20%)	⬆️
pkg/backends/ibmsecretsmanager.go	78.85% <85.71%> (+0.28%)	⬆️
pkg/config/config.go	83.78% <91.66%> (+0.21%)	⬆️
cmd/generate.go	74.19% <100.00%> (+2.26%)	⬆️
pkg/backends/awssecretsmanager.go	70.83% <100.00%> (+2.65%)	⬆️
pkg/backends/azurekeyvault.go	78.57% <100.00%> (+5.04%)	⬆️
... and 7 more

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

[werne2j]

@jkayani does it make sense to add tests for the verbose logging?

[jkayani]

@werne2j I don't know, what do you think? On one hand I don't like taking the hit on code coverage, but on the other, writing tests for this seems a bit silly, since it's really just supposed to be useful debug information.

I wonder if I can refactor so that the calls to the log function aren't wrapped in the conditional and instead the conditional is moved to the function definition - that'd probably reduce the code coverage hit (since the tests we have now would exercise those lines) and make the code a bit cleaner

[werne2j]

@jkayani I think moving the logic to the utility function makes sense.

[werne2j]

Looks good!