-
Notifications
You must be signed in to change notification settings - Fork 20
patch upstream ec2 and eks resource detectors #91
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
patch upstream ec2 and eks resource detectors #91
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Testing:
- Did we do an E2E test to ensure that we no longer hit the 1000s timeout? Sounds like it was trivially reproduced in certain contract tests, right?
- I can see why unit tests would be tricky. I think it's ok to skip for this time. Lets make sure we have a backlog item (at some priority) to get upstream to merge these changes into latest release.
aws-opentelemetry-distro/src/amazon/opentelemetry/distro/_instrumentation_patch.py
Outdated
Show resolved
Hide resolved
aws-opentelemetry-distro/src/amazon/opentelemetry/distro/_instrumentation_patch.py
Show resolved
Hide resolved
aws-opentelemetry-distro/src/amazon/opentelemetry/distro/_instrumentation_patch.py
Show resolved
Hide resolved
aws-opentelemetry-distro/src/amazon/opentelemetry/distro/_instrumentation_patch.py
Show resolved
Hide resolved
I did test with varying
Upstream is not maintaining this package and it is in ADOT team's backlog: open-telemetry/opentelemetry-python-contrib/issues/1899 |
AwsEc2ResourceDetector failed: timed out |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Functional
Issue:
The upstream resource detector package is quite outdated since the last release of v2.0.1. And there is no plan for a new release for now.
https://github.com/open-telemetry/opentelemetry-python-contrib/commits/main/sdk-extension/opentelemetry-sdk-extension-aws
In our distro we want to consume this particular commit where the timeout for EC2 and EKS resource detectors were significantly shortened to 5 seconds. The current timeouts are quite long (1000s and 2000s respectively) and may cause application fail to start in certain cases.
Description of changes:
Using the existing mechanism of temporarily patching the upstream functions, I'm adding patches for the
_aws_http_request
methods in the upstream.Testing:
I couldn't come up with a way of writing unit tests to see if the patching works since the only change in the patched method is the timeout. Mocking the
urlopen()
to return a custom response from the patch didn't work as the mock would return the same for the original method too.I did test the change with a sample app where I added a log statement "Hello from the patch_ec2_aws_http_request" to the patch to ensure it was being applied.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.