Do not show error messages even if neither `DD_API_KEY` nor `DD_KMS_API_KEY` is set when Lambda Extension is running #102

skatsuta · 2022-03-17T16:35:14Z

What does this PR do?

When the Datadog Lambda Extension is running, do not show error messages even if neither DD_API_KEY nor DD_KMS_API_KEY is set.

Motivation

In #101, it became clear that when the Datadog Lambda Extension is running, this library does not require either DD_API_KEY or DD_KMS_API_KEY.

Therefore, when the Extension is running, even if neither DD_API_KEY nor DD_KMS_API_KEY is set, error messages should not appear.

Testing Guidelines

Same as #101. No error messages were shown even though neither DD_API_KEY nor DD_KMS_API_KEY was set 👍

Additional Notes

Types of changes

Bug fix
New feature
Breaking change
Misc (docs, refactoring, dependency upgrade, etc.)

Checklist

This PR's description is comprehensive
This PR contains breaking changes that are documented in the description
This PR introduces new APIs or parameters that are documented and unlikely to change in the foreseeable future
This PR impacts documentation, and it has been updated (or a ticket has been logged)
This PR's changes are covered by the automated tests
This PR collects user input/sensitive content into Datadog

…KEY is set when Lambda Extension is running

codecov-commenter · 2022-03-17T17:50:29Z

Codecov Report

Merging #102 (abb5d73) into main (23c42de) will increase coverage by 0.09%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main     #102      +/-   ##
==========================================
+ Coverage   85.21%   85.31%   +0.09%     
==========================================
  Files          13       13              
  Lines         778      783       +5     
==========================================
+ Hits          663      668       +5     
  Misses         91       91              
  Partials       24       24

Impacted Files	Coverage Δ
ddlambda.go	`78.57% <100.00%> (+0.67%)`	⬆️
internal/metrics/listener.go	`84.42% <100.00%> (+0.25%)`	⬆️

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

skatsuta · 2022-03-18T13:01:21Z

@tianchu Thanks for running PR checks.
It seems integration tests have failed due to the lack of AWS creds:
https://github.com/DataDog/datadog-lambda-go/runs/5589674292?check_suite_focus=true

Run ./run_integration_tests.sh
/home/runner/work/datadog-lambda-go/datadog-lambda-go/tests/integration_tests
No AWS credentials were found in the environment.
Note that only Datadog employees can run these integration tests.

Can you run those tests? Or is this error ignorable?

tianchu · 2022-03-18T13:58:45Z

@skatsuta Thanks for your contribution! The changes looks largely fine, I'll review and test. BTW, were you able to test and confirm the changes allowed you to use the secret arn?

skatsuta · 2022-03-18T14:45:40Z

@tianchu Yes, I have confirmed the following with my Lambda:

Only DD_API_KEY_SECRET_ARN was set and neither DD_API_KEY nor DD_KMS_API_KEY was set to the Lambda's env vars
DD Extension (Serverless Agent) was able to send traces and metrics using DD_API_KEY_SECRET_ARN
No error messages were shown from this library, which is expected by this PR's changes

Part of CloudWatch Logs of the Lambda in my test:

2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Datadog extension version : 21\|Datadog environment variables: DD_API_KEY_SECRET_ARN=***\|DD_ENV=development\|DD_LOG_LEVEL=DEBUG\|DD_SERVICE=<REDACTED>\|DD_TRACE_ENABLED=true\|DD_VERSION=<REDACTED>\|
--
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Starting daemon to receive messages from runtime...
--
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Unable to restore the state from file
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Found DD_API_KEY_SECRET_ARN value, trying to use it.
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| Using API key set in Secrets Manager.
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Using a SyncForwarder with a 5s timeout
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| Retry queue storage on disk is disabled
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| 'telemetry.dogstatsd.aggregator_channel_latency_buckets' is empty, falling back to default values
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| 'telemetry.dogstatsd.listeners_latency_buckets' is empty, falling back to default values
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| 'telemetry.dogstatsd.listeners_channel_latency_buckets' is empty, falling back to default values
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Forwarder started
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| dogstatsd-udp: 127.0.0.1:8125 successfully initialized
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Enabling logs collection HTTP route
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Subscribing to Logs for types: [platform function extension]
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| dogstatsd-udp: starting to listen on 127.0.0.1:8125
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| WARN \| Error loading config: open /var/task/datadog.yaml: no such file or directory
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| Features detected from environment:
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Setting DefaultEnv to "development" (from 'env' config option)
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Could not get hostname via gRPC: grpc client disabled via cmd_port: -1. Falling back to other methods.
LOGS	Name: datadog-agent	State: Subscribed	Types: [platform,function,extension]
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| Starting a serverless logs-agent...
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| logs-agent started
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Adding AWS Logs collection source
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Acquired hostname from OS: "169.254.104.221". Core agent was unreachable at "/opt/datadog-agent/bin/agent/agent": fork/exec /opt/datadog-agent/bin/agent/agent: no such file or directory.
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Trace writer initialized (climit=200 qsize=78)
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Stats writer initialized (climit=20 qsize=83)
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| serverless agent ready in 109.998276ms
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| Listening for traces at http://localhost:8126
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Starting concentrator

...(snip)...

2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Hit on the serverless.Hello route.
--
2022/03/17 13:51:27 {     "status": "debug",     "message": "datadog: Will use the Serverless Agent" }
EXTENSION	Name: datadog-agent	State: Ready	Events: [INVOKE,SHUTDOWN]
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Received invocation event...
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| could not find a unique value for runtime
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| The flush strategy end has decided to not flush at moment: starting
2022/03/17 13:51:27 {     "status": "debug",     "message": "datadog: Merge X-Ray Traces is off, using trace context from Datadog only" }
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Skipping received empty payload
2022/03/17 13:51:27 Datadog Tracer v1.36.2 INFO: DATADOG TRACER CONFIGURATION {
  ...(snip)...
}
2022/03/17 13:51:27 {     "status": "debug",     "message": "datadog: Could not convert TraceContext to a SpanContext (most likely TraceContext was empty)" }

...(snip)...

2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Hit on the serverless.Flush route.
--
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Received a runtimeDone log message for the current invocation <REDACTED>
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| The flush strategy end has decided to flush at moment: stopping
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Beginning metrics flush at time 1647525087
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Received a Flush trigger
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Beginning logs flush at time 1647525087
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| INFO \| Triggering a flush in the logs-agent
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Flush in the logs-agent done.
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Finished logs flush that was started at time 1647525087
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Flushing 2 sketches to the forwarder
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Sending sketches payload : {
  ...(snip)...
}
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| The payload was not too big, returning the full payload
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Beginning traces flush at time 1647525087
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Serializing 1 tracer payloads.
2022-03-17 13:51:27 UTC \| DD_EXTENSION \| DEBUG \| Sending trace payload : {
  ...(snip)...
}
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| INFO \| Successfully posted payload to "https://6-0-0-app.agent.datadoghq.com/api/beta/sketches?api_key=********************************", the agent will only log transaction success every 500 transactions
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| SyncForwarder has flushed 1 transactions
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| Flushing 2 series to the forwarder
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| The payload was not too big, returning the full payload
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| Flushed traces to the API; time: 633.619078ms, bytes: 1669
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| Finished traces flush that was started at time 1647525087
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| SyncForwarder has flushed 1 transactions
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| Flushing 1 service checks to the forwarder
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| The payload was not too big, returning the full payload
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| SyncForwarder has flushed 1 transactions
2022-03-17 13:51:28 UTC \| DD_EXTENSION \| DEBUG \| Finished metrics flush that was started at time 1647525087

tianchu

Code changes look great and integration tests passed too.

skatsuta · 2022-03-19T08:05:44Z

@tianchu Thank you so much for your support, I'm looking forward to the next release!

tianchu · 2022-03-20T21:54:26Z

@skatsuta Thank you for your contribution! See https://github.com/DataDog/datadog-lambda-go/releases/tag/v1.4.0

skatsuta · 2022-03-22T15:59:25Z

@tianchu Awesome! Thanks!

…KEY is set when Lambda Extension is running (DataDog#102)

* Create codeql-analysis.yml (DataDog#100) * Create codeql-analysis.yml * Update codeql-analysis.yml * Update run_integration_tests.sh * Do not show error messages even if neither DD_API_KEY nor DD_KMS_API_KEY is set when Lambda Extension is running (DataDog#102) * Bump version to 1.4.0 * Bump go + fasthttp + lint (DataDog#104) * Consolidate serverless configurations into one place (DataDog#105) * Update README.md * Update README.md * Bump dd-trace-go to latest version to address some vulnerabilities (DataDog#109) * Bump dd-trace-go to latest version to address some vulnaribilities * update go.sum with `go mod tidy` * Bump version to 1.6.0 * bump codeql (DataDog#112) * Bump dd-trace-go to v1.41 (DataDog#115) * Bump version to 1.7.0 * [SLS-2330] Add support for universal instrumentation with the extension (DataDog#116) add option to use universal instrumentation * [EEP-444] include error in failed metric send log (DataDog#118) Co-authored-by: Corey Griffin <[email protected]> * [SLS-2492] Upgrade aws sdk v2 (DataDog#113) upgrade sdk * Bump version to 1.8.0 * Use new account in integration tests (DataDog#119) * set the architecture explicitely (DataDog#122) * mask init runtime logs (DataDog#123) * Update libs (DataDog#121) * bump go 1.18 (DataDog#125) * Retry sending trace payloads on failure. (DataDog#128) * Bump version to 1.9.0 * Update DD Trace to v1.51.0(DataDog#133) * Bump go version to 1.20 (DataDog#140) Bump go version to 1.20 * Upgrade version of dd-trace-go to v1.54.1 (DataDog#141) * Bump version to 1.10.0 * Propagate trace context from SQS events (DataDog#142) * Default parent id to be trace id if not found elsewhere. * Look for trace context in context object as well as headers. * Apply trace context before starting the function execution span. * Update signature in tests. * Add spanid of execution span to context. * Do not ignore priority "-128". * Test that default parent id set to trace id. * Test span id added to context. * Test uses trace context from context object. * Bump version to 1.11.0 * feat: automate AppSec enablement setup (e.g: `AWS_LAMBDA_RUNTIME_API`) (DataDog#143) * feat: honor AWS_LAMBDA_EXEC_WRAPPER when AWS Lambda does not In order to simplify onboarding & make it more uniform across languages, inspect the value of the `AWS_LAMBDA_EXEC_WRAPPER` environment variable and apply select environment variable changes it perofrms upon decorating a handler. This is necessary/useful because that environment variable is not honored by custom runtimes (`provided`, `provided.al2`) as well as the `go1.x` runtime (which is a glorified provided runtime). The datadog Lambda wrapper starts a proxy to inject ASM functionality directly on the Lambda runtime API instead of having to manually instrument each and every lambda handler/application, and modifies `AWS_LAMBDA_RUNTIME_API` to instruct Lambda language runtime client libraries to go through it instead of directly interacting with the Lambda control plane. APPSEC-11534 * pivot to a different, cheaper strategy * typo fix * PR feedback * minor fixups * add warning in go1.x runtime if lambda.norpc build tag was not enabled * Bump version to 1.12.0 * Re-add configs after upstream rebase * Bump packages * Remove deprecated `io/ioutil` calls --------- Co-authored-by: Tian Chu <[email protected]> Co-authored-by: Soshi Katsuta <[email protected]> Co-authored-by: Maxime David <[email protected]> Co-authored-by: kimi <[email protected]> Co-authored-by: Kimi Wu <[email protected]> Co-authored-by: Dylan Yang <[email protected]> Co-authored-by: Corey Griffin <[email protected]> Co-authored-by: Corey Griffin <[email protected]> Co-authored-by: Marcin Rabenda <[email protected]> Co-authored-by: Rey Abolofia <[email protected]> Co-authored-by: Rey Abolofia <[email protected]> Co-authored-by: Andrew Rodriguez <[email protected]> Co-authored-by: Ivan Topolcic <[email protected]> Co-authored-by: Romain Marcadier <[email protected]>

Do not show error messages even if neither DD_API_KEY nor DD_KMS_API_…

abb5d73

…KEY is set when Lambda Extension is running

skatsuta marked this pull request as ready for review March 17, 2022 16:35

skatsuta requested a review from a team as a code owner March 17, 2022 16:35

skatsuta mentioned this pull request Mar 17, 2022

Add support for DD_API_KEY_SECRET_ARN environment variable #101

Closed

10 tasks

tianchu self-assigned this Mar 17, 2022

tianchu added the bug Something isn't working label Mar 17, 2022

tianchu approved these changes Mar 18, 2022

View reviewed changes

tianchu merged commit 151ed77 into DataDog:main Mar 18, 2022

skatsuta deleted the do-not-error-if-extension-is-running branch March 19, 2022 08:05

peterdeme pushed a commit to spacelift-io/datadog-lambda-go that referenced this pull request Nov 15, 2023

Do not show error messages even if neither DD_API_KEY nor DD_KMS_API_…

c3c91a8

…KEY is set when Lambda Extension is running (DataDog#102)

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Do not show error messages even if neither `DD_API_KEY` nor `DD_KMS_API_KEY` is set when Lambda Extension is running #102

Do not show error messages even if neither `DD_API_KEY` nor `DD_KMS_API_KEY` is set when Lambda Extension is running #102

Uh oh!

skatsuta commented Mar 17, 2022

Uh oh!

codecov-commenter commented Mar 17, 2022

Uh oh!

skatsuta commented Mar 18, 2022

Uh oh!

tianchu commented Mar 18, 2022 •

edited

Loading

Uh oh!

skatsuta commented Mar 18, 2022 •

edited

Loading

Uh oh!

tianchu left a comment

Uh oh!

skatsuta commented Mar 19, 2022

Uh oh!

tianchu commented Mar 20, 2022

Uh oh!

skatsuta commented Mar 22, 2022

Uh oh!

Uh oh!

Do not show error messages even if neither DD_API_KEY nor DD_KMS_API_KEY is set when Lambda Extension is running #102

Do not show error messages even if neither DD_API_KEY nor DD_KMS_API_KEY is set when Lambda Extension is running #102

Uh oh!

Conversation

skatsuta commented Mar 17, 2022

What does this PR do?

Motivation

Testing Guidelines

Additional Notes

Types of changes

Checklist

Uh oh!

codecov-commenter commented Mar 17, 2022

Codecov Report

Uh oh!

skatsuta commented Mar 18, 2022

Uh oh!

tianchu commented Mar 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

skatsuta commented Mar 18, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tianchu left a comment

Choose a reason for hiding this comment

Uh oh!

skatsuta commented Mar 19, 2022

Uh oh!

tianchu commented Mar 20, 2022

Uh oh!

skatsuta commented Mar 22, 2022

Uh oh!

Uh oh!

Do not show error messages even if neither `DD_API_KEY` nor `DD_KMS_API_KEY` is set when Lambda Extension is running #102

Do not show error messages even if neither `DD_API_KEY` nor `DD_KMS_API_KEY` is set when Lambda Extension is running #102

tianchu commented Mar 18, 2022 •

edited

Loading

skatsuta commented Mar 18, 2022 •

edited

Loading