Skip to content

Commit 7c90cf4

Browse files
Alex Botensrikanthccvadriangblzchentigrannajaryan
authored
Add logging signal to main (#2251)
* Add initial overall structure and classes for logs sdk (#1894) * Add global LogEmitterProvider and convenience function get_log_emitter (#1901) * Add OTLPHandler for standard library logging module (#1903) * Add LogProcessors implementation (#1916) * Fix typos in test_handler.py (#1953) * Add support for OTLP Log exporter (#1943) * Add support for user defined attributes in OTLPHandler (#1952) * use timeout in force_flush (#2118) * use timeout in force_flush * fix lint * Update opentelemetry-sdk/src/opentelemetry/sdk/logs/export/__init__.py Co-authored-by: Srikanth Chekuri <[email protected]> * fix lint Co-authored-by: Srikanth Chekuri <[email protected]> * add a ConsoleExporter for logging (#2099) Co-authored-by: Srikanth Chekuri <[email protected]> * Update SDK docs and Add example with OTEL collector logging (debug) exporter (#2050) * Fix exception in severity number transformation (#2208) * Fix exception with warning message transformation * Fix lint * Fix lint * fstring * Demonstrate how to set the Resource for LogEmitterProvider (#2209) * Demonstrate how to set the Resource for LogEmitterProvider Added a Resource to the logs example to make it more complete. Previously it was using the built-in Resource. Now it adds the service.name and service.instance.id attributes. The resulting emitted log records look like this: ``` Resource labels: -> telemetry.sdk.language: STRING(python) -> telemetry.sdk.name: STRING(opentelemetry) -> telemetry.sdk.version: STRING(1.5.0) -> service.name: STRING(shoppingcart) -> service.instance.id: STRING(instance-12) InstrumentationLibraryLogs #0 InstrumentationLibrary __main__ 0.1 LogRecord #0 Timestamp: 2021-10-14 18:33:43.425820928 +0000 UTC Severity: ERROR ShortName: Body: Hyderabad, we have a major problem. Trace ID: ce1577e4a703f42d569e72593ad71888 Span ID: f8908ac4258ceff6 Flags: 1 ``` * Fix linting * Use batch processor in example (#2225) * move logs to _logs (#2240) * move logs to _logs * fix lint * move log_exporter to _log_exporter as it's still experimental (#2252) Co-authored-by: Srikanth Chekuri <[email protected]> Co-authored-by: Adrian Garcia Badaracco <[email protected]> Co-authored-by: Leighton Chen <[email protected]> Co-authored-by: Tigran Najaryan <[email protected]> Co-authored-by: Owais Lone <[email protected]>
1 parent 5bc91c8 commit 7c90cf4

File tree

22 files changed

+2530
-0
lines changed

22 files changed

+2530
-0
lines changed

CHANGELOG.md

+8
Original file line numberDiff line numberDiff line change
@@ -115,6 +115,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
115115
- Added dropped count to otlp, jaeger and zipkin exporters.
116116
([#1893](https://github.com/open-telemetry/opentelemetry-python/pull/1893))
117117

118+
### Added
119+
- Give OTLPHandler the ability to process attributes
120+
([#1952](https://github.com/open-telemetry/opentelemetry-python/pull/1952))
121+
- Add global LogEmitterProvider and convenience function get_log_emitter
122+
([#1901](https://github.com/open-telemetry/opentelemetry-python/pull/1901))
123+
- Add OTLPHandler for standard library logging module
124+
([#1903](https://github.com/open-telemetry/opentelemetry-python/pull/1903))
125+
118126
### Changed
119127
- Updated `opentelemetry-opencensus-exporter` to use `service_name` of spans instead of resource
120128
([#1897](https://github.com/open-telemetry/opentelemetry-python/pull/1897))

docs/examples/logs/README.rst

+75
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,75 @@
1+
OpenTelemetry Logs SDK
2+
======================
3+
4+
Start the Collector locally to see data being exported. Write the following file:
5+
6+
.. code-block:: yaml
7+
8+
# otel-collector-config.yaml
9+
receivers:
10+
otlp:
11+
protocols:
12+
grpc:
13+
14+
exporters:
15+
logging:
16+
17+
processors:
18+
batch:
19+
20+
Then start the Docker container:
21+
22+
.. code-block:: sh
23+
24+
docker run \
25+
-p 4317:4317 \
26+
-v $(pwd)/otel-collector-config.yaml:/etc/otel/config.yaml \
27+
otel/opentelemetry-collector-contrib:latest
28+
29+
.. code-block:: sh
30+
31+
$ python example.py
32+
33+
The resulting logs will appear in the output from the collector and look similar to this:
34+
35+
.. code-block:: sh
36+
37+
ResourceLog #0
38+
Resource labels:
39+
-> telemetry.sdk.language: STRING(python)
40+
-> telemetry.sdk.name: STRING(opentelemetry)
41+
-> telemetry.sdk.version: STRING(1.5.0.dev0)
42+
-> service.name: STRING(unknown_service)
43+
InstrumentationLibraryLogs #0
44+
InstrumentationLibrary __main__ 0.1
45+
LogRecord #0
46+
Timestamp: 2021-08-18 08:26:53.837349888 +0000 UTC
47+
Severity: ERROR
48+
ShortName:
49+
Body: Exception while exporting logs.
50+
ResourceLog #1
51+
Resource labels:
52+
-> telemetry.sdk.language: STRING(python)
53+
-> telemetry.sdk.name: STRING(opentelemetry)
54+
-> telemetry.sdk.version: STRING(1.5.0.dev0)
55+
-> service.name: STRING(unknown_service)
56+
InstrumentationLibraryLogs #0
57+
InstrumentationLibrary __main__ 0.1
58+
LogRecord #0
59+
Timestamp: 2021-08-18 08:26:53.842546944 +0000 UTC
60+
Severity: ERROR
61+
ShortName:
62+
Body: The five boxing wizards jump quickly.
63+
ResourceLog #2
64+
Resource labels:
65+
-> telemetry.sdk.language: STRING(python)
66+
-> telemetry.sdk.name: STRING(opentelemetry)
67+
-> telemetry.sdk.version: STRING(1.5.0.dev0)
68+
-> service.name: STRING(unknown_service)
69+
InstrumentationLibraryLogs #0
70+
InstrumentationLibrary __main__ 0.1
71+
LogRecord #0
72+
Timestamp: 2021-08-18 08:26:53.843979008 +0000 UTC
73+
Severity: ERROR
74+
ShortName:
75+
Body: Hyderabad, we have a major problem.

docs/examples/logs/example.py

+62
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
import logging
2+
3+
from opentelemetry import trace
4+
from opentelemetry.exporter.otlp.proto.grpc._log_exporter import (
5+
OTLPLogExporter,
6+
)
7+
from opentelemetry.sdk._logs import (
8+
LogEmitterProvider,
9+
OTLPHandler,
10+
set_log_emitter_provider,
11+
)
12+
from opentelemetry.sdk._logs.export import BatchLogProcessor
13+
from opentelemetry.sdk.resources import Resource
14+
from opentelemetry.sdk.trace import TracerProvider
15+
from opentelemetry.sdk.trace.export import (
16+
BatchSpanProcessor,
17+
ConsoleSpanExporter,
18+
)
19+
20+
trace.set_tracer_provider(TracerProvider())
21+
trace.get_tracer_provider().add_span_processor(
22+
BatchSpanProcessor(ConsoleSpanExporter())
23+
)
24+
25+
log_emitter_provider = LogEmitterProvider(
26+
resource=Resource.create(
27+
{
28+
"service.name": "shoppingcart",
29+
"service.instance.id": "instance-12",
30+
}
31+
),
32+
)
33+
set_log_emitter_provider(log_emitter_provider)
34+
35+
exporter = OTLPLogExporter(insecure=True)
36+
log_emitter_provider.add_log_processor(BatchLogProcessor(exporter))
37+
log_emitter = log_emitter_provider.get_log_emitter(__name__, "0.1")
38+
handler = OTLPHandler(level=logging.NOTSET, log_emitter=log_emitter)
39+
40+
# Attach OTLP handler to root logger
41+
logging.getLogger("root").addHandler(handler)
42+
43+
# Log directly
44+
logging.info("Jackdaws love my big sphinx of quartz.")
45+
46+
# Create different namespaced loggers
47+
logger1 = logging.getLogger("myapp.area1")
48+
logger2 = logging.getLogger("myapp.area2")
49+
50+
logger1.debug("Quick zephyrs blow, vexing daft Jim.")
51+
logger1.info("How quickly daft jumping zebras vex.")
52+
logger2.warning("Jail zesty vixen who grabbed pay from quack.")
53+
logger2.error("The five boxing wizards jump quickly.")
54+
55+
56+
# Trace context correlation
57+
tracer = trace.get_tracer(__name__)
58+
with tracer.start_as_current_span("foo"):
59+
# Do something
60+
logger2.error("Hyderabad, we have a major problem.")
61+
62+
log_emitter_provider.shutdown()
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
receivers:
2+
otlp:
3+
protocols:
4+
grpc:
5+
6+
exporters:
7+
logging:
8+
9+
processors:
10+
batch:

docs/sdk/logs.export.rst

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
opentelemetry.sdk._logs.export
2+
==============================
3+
4+
.. automodule:: opentelemetry.sdk._logs.export
5+
:members:
6+
:undoc-members:
7+
:show-inheritance:

docs/sdk/logs.rst

+22
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
opentelemetry.sdk._logs package
2+
===============================
3+
4+
.. warning::
5+
OpenTelemetry Python logs are in an experimental state. The APIs within
6+
:mod:`opentelemetry.sdk._logs` are subject to change in minor/patch releases and make no
7+
backward compatability guarantees at this time.
8+
9+
Once logs become stable, this package will be be renamed to ``opentelemetry.sdk.logs``.
10+
11+
Submodules
12+
----------
13+
14+
.. toctree::
15+
16+
logs.export
17+
logs.severity
18+
19+
.. automodule:: opentelemetry.sdk._logs
20+
:members:
21+
:undoc-members:
22+
:show-inheritance:

docs/sdk/logs.severity.rst

+7
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
opentelemetry.sdk._logs.severity
2+
================================
3+
4+
.. automodule:: opentelemetry.sdk._logs.severity
5+
:members:
6+
:undoc-members:
7+
:show-inheritance:

docs/sdk/sdk.rst

+1
Original file line numberDiff line numberDiff line change
@@ -8,5 +8,6 @@ OpenTelemetry Python SDK
88

99
resources
1010
trace
11+
logs
1112
error_handler
1213
environment_variables
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
# Copyright The OpenTelemetry Authors
2+
# Licensed under the Apache License, Version 2.0 (the "License");
3+
# you may not use this file except in compliance with the License.
4+
# You may obtain a copy of the License at
5+
#
6+
# http://www.apache.org/licenses/LICENSE-2.0
7+
#
8+
# Unless required by applicable law or agreed to in writing, software
9+
# distributed under the License is distributed on an "AS IS" BASIS,
10+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
11+
# See the License for the specific language governing permissions and
12+
# limitations under the License.
13+
14+
from typing import Optional, Sequence
15+
from grpc import ChannelCredentials, Compression
16+
from opentelemetry.exporter.otlp.proto.grpc.exporter import (
17+
OTLPExporterMixin,
18+
_translate_key_values,
19+
get_resource_data,
20+
_translate_value,
21+
)
22+
from opentelemetry.proto.collector.logs.v1.logs_service_pb2 import (
23+
ExportLogsServiceRequest,
24+
)
25+
from opentelemetry.proto.collector.logs.v1.logs_service_pb2_grpc import (
26+
LogsServiceStub,
27+
)
28+
from opentelemetry.proto.common.v1.common_pb2 import InstrumentationLibrary
29+
from opentelemetry.proto.logs.v1.logs_pb2 import (
30+
InstrumentationLibraryLogs,
31+
ResourceLogs,
32+
)
33+
from opentelemetry.proto.logs.v1.logs_pb2 import LogRecord as PB2LogRecord
34+
from opentelemetry.sdk._logs import LogRecord as SDKLogRecord
35+
from opentelemetry.sdk._logs import LogData
36+
from opentelemetry.sdk._logs.export import LogExporter, LogExportResult
37+
38+
39+
class OTLPLogExporter(
40+
LogExporter,
41+
OTLPExporterMixin[SDKLogRecord, ExportLogsServiceRequest, LogExportResult],
42+
):
43+
44+
_result = LogExportResult
45+
_stub = LogsServiceStub
46+
47+
def __init__(
48+
self,
49+
endpoint: Optional[str] = None,
50+
insecure: Optional[bool] = None,
51+
credentials: Optional[ChannelCredentials] = None,
52+
headers: Optional[Sequence] = None,
53+
timeout: Optional[int] = None,
54+
compression: Optional[Compression] = None,
55+
):
56+
super().__init__(
57+
**{
58+
"endpoint": endpoint,
59+
"insecure": insecure,
60+
"credentials": credentials,
61+
"headers": headers,
62+
"timeout": timeout,
63+
"compression": compression,
64+
}
65+
)
66+
67+
def _translate_name(self, log_data: LogData) -> None:
68+
self._collector_log_kwargs["name"] = log_data.log_record.name
69+
70+
def _translate_time(self, log_data: LogData) -> None:
71+
self._collector_log_kwargs[
72+
"time_unix_nano"
73+
] = log_data.log_record.timestamp
74+
75+
def _translate_span_id(self, log_data: LogData) -> None:
76+
self._collector_log_kwargs[
77+
"span_id"
78+
] = log_data.log_record.span_id.to_bytes(8, "big")
79+
80+
def _translate_trace_id(self, log_data: LogData) -> None:
81+
self._collector_log_kwargs[
82+
"trace_id"
83+
] = log_data.log_record.trace_id.to_bytes(16, "big")
84+
85+
def _translate_trace_flags(self, log_data: LogData) -> None:
86+
self._collector_log_kwargs["flags"] = int(
87+
log_data.log_record.trace_flags
88+
)
89+
90+
def _translate_body(self, log_data: LogData):
91+
self._collector_log_kwargs["body"] = _translate_value(
92+
log_data.log_record.body
93+
)
94+
95+
def _translate_severity_text(self, log_data: LogData):
96+
self._collector_log_kwargs[
97+
"severity_text"
98+
] = log_data.log_record.severity_text
99+
100+
def _translate_attributes(self, log_data: LogData) -> None:
101+
if log_data.log_record.attributes:
102+
self._collector_log_kwargs["attributes"] = []
103+
for key, value in log_data.log_record.attributes.items():
104+
try:
105+
self._collector_log_kwargs["attributes"].append(
106+
_translate_key_values(key, value)
107+
)
108+
except Exception: # pylint: disable=broad-except
109+
pass
110+
111+
def _translate_data(
112+
self, data: Sequence[LogData]
113+
) -> ExportLogsServiceRequest:
114+
# pylint: disable=attribute-defined-outside-init
115+
116+
sdk_resource_instrumentation_library_logs = {}
117+
118+
for log_data in data:
119+
resource = log_data.log_record.resource
120+
121+
instrumentation_library_logs_map = (
122+
sdk_resource_instrumentation_library_logs.get(resource, {})
123+
)
124+
if not instrumentation_library_logs_map:
125+
sdk_resource_instrumentation_library_logs[
126+
resource
127+
] = instrumentation_library_logs_map
128+
129+
instrumentation_library_logs = (
130+
instrumentation_library_logs_map.get(
131+
log_data.instrumentation_info
132+
)
133+
)
134+
if not instrumentation_library_logs:
135+
if log_data.instrumentation_info is not None:
136+
instrumentation_library_logs_map[
137+
log_data.instrumentation_info
138+
] = InstrumentationLibraryLogs(
139+
instrumentation_library=InstrumentationLibrary(
140+
name=log_data.instrumentation_info.name,
141+
version=log_data.instrumentation_info.version,
142+
)
143+
)
144+
else:
145+
instrumentation_library_logs_map[
146+
log_data.instrumentation_info
147+
] = InstrumentationLibraryLogs()
148+
149+
instrumentation_library_logs = (
150+
instrumentation_library_logs_map.get(
151+
log_data.instrumentation_info
152+
)
153+
)
154+
155+
self._collector_log_kwargs = {}
156+
157+
self._translate_name(log_data)
158+
self._translate_time(log_data)
159+
self._translate_span_id(log_data)
160+
self._translate_trace_id(log_data)
161+
self._translate_trace_flags(log_data)
162+
self._translate_body(log_data)
163+
self._translate_severity_text(log_data)
164+
self._translate_attributes(log_data)
165+
166+
self._collector_log_kwargs[
167+
"severity_number"
168+
] = log_data.log_record.severity_number.value
169+
170+
instrumentation_library_logs.logs.append(
171+
PB2LogRecord(**self._collector_log_kwargs)
172+
)
173+
174+
return ExportLogsServiceRequest(
175+
resource_logs=get_resource_data(
176+
sdk_resource_instrumentation_library_logs,
177+
ResourceLogs,
178+
"logs",
179+
)
180+
)
181+
182+
def export(self, batch: Sequence[LogData]) -> LogExportResult:
183+
return self._export(batch)
184+
185+
def shutdown(self) -> None:
186+
pass

exporter/opentelemetry-exporter-otlp-proto-grpc/tests/logs/__init__.py

Whitespace-only changes.

0 commit comments

Comments
 (0)