Skip to content

Commit 1847f38

Browse files
aabmassaabmass
committed
Cleanup OTLP exporter compression options, add tests
1 parent 1af9f87 commit 1847f38

File tree

7 files changed

+166
-57
lines changed

7 files changed

+166
-57
lines changed

‎CHANGELOG.md

+16-14
Original file line numberDiff line numberDiff line change
@@ -8,51 +8,53 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88

99
### Added
1010
- Add `max_attr_value_length` support to Jaeger exporter
11-
([#1633])(https://github.com/open-telemetry/opentelemetry-python/pull/1633)
11+
([#1633](https://github.com/open-telemetry/opentelemetry-python/pull/1633))
1212

1313
### Changed
1414
- Rename `IdsGenerator` to `IdGenerator`
15-
([#1651])(https://github.com/open-telemetry/opentelemetry-python/pull/1651)
15+
([#1651](https://github.com/open-telemetry/opentelemetry-python/pull/1651))
1616
- Make TracerProvider's resource attribute private
17-
([#1652])(https://github.com/open-telemetry/opentelemetry-python/pull/1652)
17+
([#1652](https://github.com/open-telemetry/opentelemetry-python/pull/1652))
1818
- Rename Resource's `create_empty` to `get_empty`
19-
([#1653])(https://github.com/open-telemetry/opentelemetry-python/pull/1653)
19+
([#1653](https://github.com/open-telemetry/opentelemetry-python/pull/1653))
2020
- Renamed `BatchExportSpanProcessor` to `BatchSpanProcessor` and `SimpleExportSpanProcessor` to
2121
`SimpleSpanProcessor`
22-
([#1656])(https://github.com/open-telemetry/opentelemetry-python/pull/1656)
22+
([#1656](https://github.com/open-telemetry/opentelemetry-python/pull/1656))
2323
- Rename `DefaultSpan` to `NonRecordingSpan`
24-
([#1661])(https://github.com/open-telemetry/opentelemetry-python/pull/1661)
24+
([#1661](https://github.com/open-telemetry/opentelemetry-python/pull/1661))
2525
- Moving `Getter`, `Setter` and `TextMapPropagator` out of `opentelemetry.trace.propagation` and
2626
into `opentelemetry.propagators`
27-
([#1662])(https://github.com/open-telemetry/opentelemetry-python/pull/1662)
27+
([#1662](https://github.com/open-telemetry/opentelemetry-python/pull/1662))
2828
- Rename `BaggagePropagator` to `W3CBaggagePropagator`
29-
([#1663])(https://github.com/open-telemetry/opentelemetry-python/pull/1663)
29+
([#1663](https://github.com/open-telemetry/opentelemetry-python/pull/1663))
3030
- Rename `JaegerSpanExporter` to `JaegerExporter` and rename `ZipkinSpanExporter` to `ZipkinExporter`
31-
([#1664])(https://github.com/open-telemetry/opentelemetry-python/pull/1664)
31+
([#1664](https://github.com/open-telemetry/opentelemetry-python/pull/1664))
32+
- Cleanup OTLP exporter compression options, add tests
33+
([#1671](https://github.com/open-telemetry/opentelemetry-python/pull/1671))
3234

3335
## [0.18b0](https://github.com/open-telemetry/opentelemetry-python/releases/tag/v0.18b0) - 2021-02-16
3436

3537
### Added
3638
- Add urllib to opentelemetry-bootstrap target list
37-
([#1584])(https://github.com/open-telemetry/opentelemetry-python/pull/1584)
39+
([#1584](https://github.com/open-telemetry/opentelemetry-python/pull/1584))
3840

3941
## [1.0.0rc1](https://github.com/open-telemetry/opentelemetry-python/releases/tag/v1.0.0rc1) - 2021-02-12
4042

4143
### Changed
4244
- Tracer provider environment variables are now consistent with the rest
43-
([#1571](https://github.com/open-telemetry/opentelemetry-python/pull/1571)])
45+
([#1571](https://github.com/open-telemetry/opentelemetry-python/pull/1571))
4446
- Rename `TRACE_` to `TRACES_` for environment variables
45-
([#1595](https://github.com/open-telemetry/opentelemetry-python/pull/1595)])
47+
([#1595](https://github.com/open-telemetry/opentelemetry-python/pull/1595))
4648
- Limits for Span attributes, events and links have been updated to 128
47-
([1597](https://github.com/open-telemetry/opentelemetry-python/pull/1597)])
49+
([1597](https://github.com/open-telemetry/opentelemetry-python/pull/1597))
4850
- Read-only Span attributes have been moved to ReadableSpan class
4951
([#1560](https://github.com/open-telemetry/opentelemetry-python/pull/1560))
5052
- `BatchExportSpanProcessor` flushes export queue when it reaches `max_export_batch_size`
5153
([#1521])(https://github.com/open-telemetry/opentelemetry-python/pull/1521)
5254

5355
### Added
5456
- Added `end_on_exit` argument to `start_as_current_span`
55-
([#1519](https://github.com/open-telemetry/opentelemetry-python/pull/1519)])
57+
([#1519](https://github.com/open-telemetry/opentelemetry-python/pull/1519))
5658
- Add `Span.set_attributes` method to set multiple values with one call
5759
([#1520](https://github.com/open-telemetry/opentelemetry-python/pull/1520))
5860
- Make sure Resources follow semantic conventions

‎exporter/opentelemetry-exporter-otlp/src/opentelemetry/exporter/otlp/__init__.py

+4-10
Original file line numberDiff line numberDiff line change
@@ -22,20 +22,14 @@
2222
The **OTLP Span Exporter** allows to export `OpenTelemetry`_ traces to the
2323
`OTLP`_ collector.
2424
25+
You can configure the exporter with the following environment variables
26+
27+
- :envvar:`OTEL_EXPORTER_OTLP_SPAN_COMPRESSION`
28+
- :envvar:`OTEL_EXPORTER_OTLP_COMPRESSION`
2529
2630
.. _OTLP: https://github.com/open-telemetry/opentelemetry-collector/
2731
.. _OpenTelemetry: https://github.com/open-telemetry/opentelemetry-python/
2832
29-
.. envvar:: OTEL_EXPORTER_OTLP_COMPRESSION
30-
31-
The :envvar:`OTEL_EXPORTER_OTLP_COMPRESSION` environment variable allows a
32-
compression algorithm to be passed to the OTLP exporter. The compression
33-
algorithms that are supported include gzip and no compression. The value should
34-
be in the format of a string "gzip" for gzip compression, and no value specified
35-
if no compression is the desired choice.
36-
Additional details are available `in the specification
37-
<https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/protocol/exporter.md#opentelemetry-protocol-exporter>`_.
38-
3933
.. code:: python
4034
4135
from opentelemetry import trace

‎exporter/opentelemetry-exporter-otlp/src/opentelemetry/exporter/otlp/exporter.py

+25-29
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,6 @@
1414

1515
"""OTLP Exporter"""
1616

17-
import enum
1817
import logging
1918
from abc import ABC, abstractmethod
2019
from collections.abc import Mapping, Sequence
@@ -40,6 +39,7 @@
4039
from opentelemetry.proto.resource.v1.resource_pb2 import Resource
4140
from opentelemetry.sdk.environment_variables import (
4241
OTEL_EXPORTER_OTLP_CERTIFICATE,
42+
OTEL_EXPORTER_OTLP_COMPRESSION,
4343
OTEL_EXPORTER_OTLP_ENDPOINT,
4444
OTEL_EXPORTER_OTLP_HEADERS,
4545
OTEL_EXPORTER_OTLP_INSECURE,
@@ -54,9 +54,22 @@
5454
ExportServiceRequestT = TypeVar("ExportServiceRequestT")
5555
ExportResultT = TypeVar("ExportResultT")
5656

57+
_ENVIRON_TO_COMPRESSION = {
58+
None: None,
59+
"gzip": Compression.Gzip,
60+
"deflate": Compression.Deflate,
61+
}
5762

58-
class OTLPCompression(enum.Enum):
59-
gzip = "gzip"
63+
64+
def environ_to_compression(environ_key: str) -> Optional[Compression]:
65+
environ_value = environ.get(environ_key)
66+
if environ_value not in _ENVIRON_TO_COMPRESSION:
67+
raise Exception(
68+
'Invalid value "{}" for compression envvar {}'.format(
69+
environ_value, environ_key
70+
)
71+
)
72+
return _ENVIRON_TO_COMPRESSION[environ_value]
6073

6174

6275
def _translate_key_values(key: Text, value: Any) -> KeyValue:
@@ -87,7 +100,7 @@ def _translate_key_values(key: Text, value: Any) -> KeyValue:
87100
return KeyValue(key=key, value=any_value)
88101

89102

90-
def _get_resource_data(
103+
def get_resource_data(
91104
sdk_resource_instrumentation_library_data: Dict[
92105
SDKResource, ResourceDataT
93106
],
@@ -160,7 +173,7 @@ def __init__(
160173
credentials: Optional[ChannelCredentials] = None,
161174
headers: Optional[Sequence] = None,
162175
timeout: Optional[int] = None,
163-
compression: str = None,
176+
compression: Optional[Compression] = None,
164177
):
165178
super().__init__()
166179

@@ -187,30 +200,15 @@ def __init__(
187200
)
188201
self._collector_span_kwargs = None
189202

190-
if compression is None:
191-
compression_algorithm = Compression.NoCompression
192-
elif (
193-
compression in OTLPCompression._value2member_map_
194-
and OTLPCompression(compression) is OTLPCompression.gzip
195-
):
196-
compression_algorithm = Compression.Gzip
197-
else:
198-
compression_str = environ.get(OTEL_EXPORTER_OTLP_INSECURE)
199-
if compression_str is None:
200-
compression_algorithm = Compression.NoCompression
201-
elif (
202-
compression_str in OTLPCompression._value2member_map_
203-
and OTLPCompression(compression_str) is OTLPCompression.gzip
204-
):
205-
compression_algorithm = Compression.Gzip
206-
else:
207-
raise ValueError(
208-
"OTEL_EXPORTER_OTLP_COMPRESSION environment variable does not match gzip."
209-
)
203+
compression = (
204+
environ_to_compression(OTEL_EXPORTER_OTLP_COMPRESSION)
205+
if compression is None
206+
else compression
207+
) or Compression.NoCompression
210208

211209
if insecure:
212210
self._client = self._stub(
213-
insecure_channel(endpoint, compression=compression_algorithm)
211+
insecure_channel(endpoint, compression=compression)
214212
)
215213
return
216214

@@ -226,9 +224,7 @@ def __init__(
226224
environ.get(OTEL_EXPORTER_OTLP_CERTIFICATE)
227225
)
228226
self._client = self._stub(
229-
secure_channel(
230-
endpoint, credentials, compression=compression_algorithm
231-
)
227+
secure_channel(endpoint, credentials, compression=compression)
232228
)
233229

234230
@abstractmethod

‎exporter/opentelemetry-exporter-otlp/src/opentelemetry/exporter/otlp/trace_exporter/__init__.py

+13-3
Original file line numberDiff line numberDiff line change
@@ -17,13 +17,14 @@
1717
from os import environ
1818
from typing import Optional, Sequence
1919

20-
from grpc import ChannelCredentials
20+
from grpc import ChannelCredentials, Compression
2121

2222
from opentelemetry.exporter.otlp.exporter import (
2323
OTLPExporterMixin,
24-
_get_resource_data,
2524
_load_credential_from_file,
2625
_translate_key_values,
26+
environ_to_compression,
27+
get_resource_data,
2728
)
2829
from opentelemetry.proto.collector.trace.v1.trace_service_pb2 import (
2930
ExportTraceServiceRequest,
@@ -40,6 +41,7 @@
4041
from opentelemetry.proto.trace.v1.trace_pb2 import Status
4142
from opentelemetry.sdk.environment_variables import (
4243
OTEL_EXPORTER_OTLP_SPAN_CERTIFICATE,
44+
OTEL_EXPORTER_OTLP_SPAN_COMPRESSION,
4345
OTEL_EXPORTER_OTLP_SPAN_ENDPOINT,
4446
OTEL_EXPORTER_OTLP_SPAN_HEADERS,
4547
OTEL_EXPORTER_OTLP_SPAN_INSECURE,
@@ -80,6 +82,7 @@ def __init__(
8082
credentials: Optional[ChannelCredentials] = None,
8183
headers: Optional[Sequence] = None,
8284
timeout: Optional[int] = None,
85+
compression: Optional[Compression] = None,
8386
):
8487
if insecure is None:
8588
insecure = environ.get(OTEL_EXPORTER_OTLP_SPAN_INSECURE)
@@ -97,6 +100,12 @@ def __init__(
97100
int(environ_timeout) if environ_timeout is not None else None
98101
)
99102

103+
compression = (
104+
environ_to_compression(OTEL_EXPORTER_OTLP_SPAN_COMPRESSION)
105+
if compression is None
106+
else compression
107+
)
108+
100109
super().__init__(
101110
**{
102111
"endpoint": endpoint
@@ -106,6 +115,7 @@ def __init__(
106115
"headers": headers
107116
or environ.get(OTEL_EXPORTER_OTLP_SPAN_HEADERS),
108117
"timeout": timeout or environ_timeout,
118+
"compression": compression,
109119
}
110120
)
111121

@@ -274,7 +284,7 @@ def _translate_data(
274284
].spans.append(CollectorSpan(**self._collector_span_kwargs))
275285

276286
return ExportTraceServiceRequest(
277-
resource_spans=_get_resource_data(
287+
resource_spans=get_resource_data(
278288
sdk_resource_instrumentation_library_spans,
279289
ResourceSpans,
280290
"spans",

‎exporter/opentelemetry-exporter-otlp/tests/test_otlp_exporter_mixin.py

+27
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
from unittest import TestCase
2+
from unittest.mock import patch
3+
4+
from grpc import Compression
5+
6+
from opentelemetry.exporter.otlp.exporter import environ_to_compression
7+
8+
9+
class TestOTLPExporterMixin(TestCase):
10+
def test_environ_to_compression(self):
11+
with patch.dict(
12+
"os.environ",
13+
{
14+
"test_gzip": "gzip",
15+
"test_deflate": "deflate",
16+
"test_invalid": "some invalid compression",
17+
},
18+
):
19+
self.assertEqual(
20+
environ_to_compression("test_gzip"), Compression.Gzip
21+
)
22+
self.assertEqual(
23+
environ_to_compression("test_deflate"), Compression.Deflate
24+
)
25+
self.assertIsNone(environ_to_compression("missing_key"),)
26+
with self.assertRaises(Exception):
27+
environ_to_compression("test_invalid")

‎exporter/opentelemetry-exporter-otlp/tests/test_otlp_trace_exporter.py

+56-1
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@
2020

2121
from google.protobuf.duration_pb2 import Duration
2222
from google.rpc.error_details_pb2 import RetryInfo
23-
from grpc import ChannelCredentials, StatusCode, server
23+
from grpc import ChannelCredentials, Compression, StatusCode, server
2424

2525
from opentelemetry.exporter.otlp.trace_exporter import OTLPSpanExporter
2626
from opentelemetry.proto.collector.trace.v1.trace_service_pb2 import (
@@ -46,7 +46,9 @@
4646
from opentelemetry.proto.trace.v1.trace_pb2 import Span as OTLPSpan
4747
from opentelemetry.proto.trace.v1.trace_pb2 import Status
4848
from opentelemetry.sdk.environment_variables import (
49+
OTEL_EXPORTER_OTLP_COMPRESSION,
4950
OTEL_EXPORTER_OTLP_SPAN_CERTIFICATE,
51+
OTEL_EXPORTER_OTLP_SPAN_COMPRESSION,
5052
OTEL_EXPORTER_OTLP_SPAN_ENDPOINT,
5153
OTEL_EXPORTER_OTLP_SPAN_HEADERS,
5254
OTEL_EXPORTER_OTLP_SPAN_TIMEOUT,
@@ -174,6 +176,7 @@ def tearDown(self):
174176
+ "/fixtures/test.cert",
175177
OTEL_EXPORTER_OTLP_SPAN_HEADERS: "key1=value1,key2=value2",
176178
OTEL_EXPORTER_OTLP_SPAN_TIMEOUT: "10",
179+
OTEL_EXPORTER_OTLP_SPAN_COMPRESSION: "gzip",
177180
},
178181
)
179182
@patch("opentelemetry.exporter.otlp.exporter.OTLPExporterMixin.__init__")
@@ -186,6 +189,7 @@ def test_env_variables(self, mock_exporter_mixin):
186189
self.assertEqual(kwargs["endpoint"], "collector:4317")
187190
self.assertEqual(kwargs["headers"], "key1=value1,key2=value2")
188191
self.assertEqual(kwargs["timeout"], 10)
192+
self.assertEqual(kwargs["compression"], Compression.Gzip)
189193
self.assertIsNotNone(kwargs["credentials"])
190194
self.assertIsInstance(kwargs["credentials"], ChannelCredentials)
191195

@@ -220,6 +224,57 @@ def test_otlp_headers_from_env(self, mock_ssl_channel, mock_secure):
220224
exporter._headers, (("key3", "value3"), ("key4", "value4"))
221225
)
222226

227+
# pylint: disable=no-self-use
228+
def test_otlp_compression_from_env(self):
229+
# Specifying kwarg should take precedence over env
230+
with patch(
231+
"opentelemetry.exporter.otlp.exporter.insecure_channel"
232+
) as mock_insecure_channel, patch.dict(
233+
"os.environ", {OTEL_EXPORTER_OTLP_COMPRESSION: "gzip"}
234+
):
235+
OTLPSpanExporter(
236+
insecure=True, compression=Compression.NoCompression
237+
)
238+
mock_insecure_channel.assert_called_once_with(
239+
"localhost:4317", compression=Compression.NoCompression
240+
)
241+
242+
# No env or kwarg should be NoCompression
243+
with patch(
244+
"opentelemetry.exporter.otlp.exporter.insecure_channel"
245+
) as mock_insecure_channel, patch.dict("os.environ", {}):
246+
OTLPSpanExporter(insecure=True)
247+
mock_insecure_channel.assert_called_once_with(
248+
"localhost:4317", compression=Compression.NoCompression
249+
)
250+
251+
# Just OTEL_EXPORTER_OTLP_COMPRESSION should work
252+
with patch(
253+
"opentelemetry.exporter.otlp.exporter.insecure_channel"
254+
) as mock_insecure_channel, patch.dict(
255+
"os.environ", {OTEL_EXPORTER_OTLP_COMPRESSION: "deflate"}
256+
):
257+
OTLPSpanExporter(insecure=True)
258+
mock_insecure_channel.assert_called_once_with(
259+
"localhost:4317", compression=Compression.Deflate
260+
)
261+
262+
# OTEL_EXPORTER_OTLP_SPAN_COMPRESSION as higher priority than
263+
# OTEL_EXPORTER_OTLP_COMPRESSION
264+
with patch(
265+
"opentelemetry.exporter.otlp.exporter.insecure_channel"
266+
) as mock_insecure_channel, patch.dict(
267+
"os.environ",
268+
{
269+
OTEL_EXPORTER_OTLP_COMPRESSION: "deflate",
270+
OTEL_EXPORTER_OTLP_SPAN_COMPRESSION: "gzip",
271+
},
272+
):
273+
OTLPSpanExporter(insecure=True)
274+
mock_insecure_channel.assert_called_once_with(
275+
"localhost:4317", compression=Compression.Gzip
276+
)
277+
223278
@patch("opentelemetry.exporter.otlp.exporter.ssl_channel_credentials")
224279
@patch("opentelemetry.exporter.otlp.exporter.secure_channel")
225280
# pylint: disable=unused-argument

0 commit comments

Comments
 (0)