Skip to content

Commit 863a003

Browse files
alertedsnakealertedsnake
authored
Merge branch 'master' into grpc-streaming-bugfix
2 parents cc89f4b + cfaa2b7 commit 863a003

File tree

18 files changed

+680
-35
lines changed

18 files changed

+680
-35
lines changed

‎.github/workflows/test.yml

+1
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,7 @@ jobs:
7373
tool: pytest
7474
output-file-path: output.json
7575
github-token: ${{ secrets.GITHUB_TOKEN }}
76+
max-items-in-chart: 100
7677
# Alert with a commit comment on possible performance regression
7778
alert-threshold: 200%
7879
fail-on-alert: true

‎.pylintrc

+1-1
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ extension-pkg-whitelist=
77

88
# Add list of files or directories to be excluded. They should be base names, not
99
# paths.
10-
ignore=CVS,gen
10+
ignore=CVS,gen,Dockerfile,docker-compose.yml,README.md,requirements.txt,cortex-config.yml
1111

1212
# Add files or directories matching the regex patterns to be excluded. The
1313
# regex matches against base names, not paths.

‎CHANGELOG.md

+7-1
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77
## [Unreleased](https://github.com/open-telemetry/opentelemetry-python-contrib/compare/v0.16b1...HEAD)
88

99
### Added
10+
- `opentelemetry-instrumentation-celery` Add support for Celery version 5.x
11+
([#266](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/266))
1012
- `opentelemetry-instrumentation-urllib` Add urllib instrumentation
1113
([#222](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/222))
1214
- `opentelemetry-exporter-datadog` Add fields method
@@ -29,16 +31,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
2931
([#216](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/216))
3032
- `opentelemetry-instrumentation-grpc` Add tests for grpc span attributes, grpc `abort()` conditions
3133
([#236](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/236))
34+
- Add README and example app for Prometheus Remote Write Exporter
35+
([#227](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/227]))
3236

3337
### Changed
3438
- `opentelemetry-instrumentation-asgi`, `opentelemetry-instrumentation-wsgi` Return `None` for `CarrierGetter` if key not found
3539
([#1374](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/233))
3640
- `opentelemetry-instrumentation-grpc` Comply with updated spec, rework tests
3741
([#236](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/236))
3842
- `opentelemetry-instrumentation-asgi`, `opentelemetry-instrumentation-falcon`, `opentelemetry-instrumentation-flask`, `opentelemetry-instrumentation-pyramid`, `opentelemetry-instrumentation-wsgi` Renamed `host.port` attribute to `net.host.port`
39-
([#242](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/242))
43+
([#242](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/242))
4044
- `opentelemetry-instrumentation-flask` Do not emit a warning message for request contexts created with `app.test_request_context`
4145
([#253](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/253))
46+
- `opentelemetry-instrumentation-requests`, `opentelemetry-instrumentation-urllib` Fix span name callback parameters
47+
([#259](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/259))
4248
- `opentelemetry-instrumentation-grpc` Fix issue tracking child spans in streaming responses
4349
([#260](https://github.com/open-telemetry/opentelemetry-python-contrib/pull/260))
4450

‎exporter/opentelemetry-exporter-prometheus-remote-write/README.rst

+309-14
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,322 @@
11
OpenTelemetry Prometheus Remote Write Exporter
2-
==============================================
2+
=========================================================
33

4-
This library allows exporting metric data to `Prometheus Remote Write Integrated Backends
5-
<https://prometheus.io/docs/operating/integrations/>`_. Latest `types.proto
6-
<https://github.com/prometheus/prometheus/blob/master/prompb/types.proto>` and `remote.proto
7-
<https://github.com/prometheus/prometheus/blob/master/prompb/remote.proto>` Protocol Buffers
8-
used to create WriteRequest objects were taken from Prometheus repository. Development is
9-
currently in progress.
4+
This package contains an exporter to send `OTLP`_ metrics from the
5+
`OpenTelemetry Python SDK`_ directly to a `Prometheus Remote Write integrated backend`_
6+
(such as Cortex or Thanos) without having to run an instance of the
7+
Prometheus server. The latest `types.proto`_ and `remote.proto`_
8+
protocol buffers are used to create the WriteRequest. The image below shows the
9+
two Prometheus exporters in the OpenTelemetry Python SDK.
10+
11+
Pipeline 1 illustrates the setup required for a `Prometheus "pull" exporter`_.
12+
13+
Pipeline 2 illustrates the setup required for the Prometheus Remote
14+
Write exporter.
15+
16+
|Prometheus SDK pipelines|
17+
18+
The Prometheus Remote Write Exporter is a "push" based exporter and only
19+
works with the OpenTelemetry `push controller`_. The controller
20+
periodically collects data and passes it to the exporter. This exporter
21+
then converts the data into `timeseries`_ and sends it to the Remote
22+
Write integrated backend through HTTP POST requests. The metrics
23+
collection datapath is shown below:
24+
25+
26+
See the ``examples`` folder for a demo usage of this exporter
27+
28+
Table of Contents
29+
=================
30+
31+
- `Summary`_
32+
- `Table of Contents`_
33+
34+
- `Installation`_
35+
- `Quickstart`_
36+
- `Examples`_
37+
- `Configuring the Exporter`_
38+
- `Securing the Exporter`_
39+
40+
- `Authentication`_
41+
- `TLS`_
42+
43+
- `Supported Aggregators`_
44+
- `Error Handling`_
45+
- `Contributing`_
46+
47+
- `Design Doc`_
1048

1149
Installation
1250
------------
51+
Prerequisites
52+
~~~~~~~~~~~~~
53+
1. Install the snappy c-library
54+
**DEB**: ``sudo apt-get install libsnappy-dev``
1355

14-
::
56+
**RPM**: ``sudo yum install libsnappy-devel``
1557

16-
pip install opentelemetry-exporter-prometheus-remote-write
58+
**OSX/Brew**: ``brew install snappy``
1759

60+
**Windows**: ``pip install python_snappy-0.5-cp36-cp36m-win_amd64.whl``
1861

19-
.. _Prometheus: https://prometheus.io/
20-
.. _OpenTelemetry: https://github.com/open-telemetry/opentelemetry-python/
62+
Exporter
63+
~~~~~~~~
2164

65+
- To install from the latest PyPi release, run
66+
``pip install opentelemetry-exporter-prometheus-remote-write``
2267

23-
References
68+
69+
Quickstart
2470
----------
2571

26-
* `Prometheus <https://prometheus.io/>`_
27-
* `OpenTelemetry Project <https://opentelemetry.io/>`_
72+
.. code:: python
73+
74+
from opentelemetry import metrics
75+
from opentelemetry.sdk.metrics import MeterProvider
76+
from opentelemetry.exporter.prometheus_remote_write import (
77+
PrometheusRemoteWriteMetricsExporter
78+
)
79+
80+
# Sets the global MeterProvider instance
81+
metrics.set_meter_provider(MeterProvider())
82+
83+
# The Meter is responsible for creating and recording metrics. Each meter has a unique name, which we set as the module's name here.
84+
meter = metrics.get_meter(__name__)
85+
86+
exporter = PrometheusRemoteWriteMetricsExporter(endpoint="endpoint_here") # add other params as needed
87+
88+
metrics.get_meter_provider().start_pipeline(meter, exporter, 5)
89+
90+
91+
Examples
92+
--------
93+
94+
This example uses `Docker Compose`_ to set up:
95+
96+
1. A Python program that creates 5 instruments with 5 unique aggregators
97+
and a randomized load generator
98+
2. An instance of `Cortex`_ to recieve the metrics data
99+
3. An instance of `Grafana`_ to visualizse the exported data
100+
101+
Requirements
102+
~~~~~~~~~~~~
103+
104+
- Have Docker Compose `installed`_
105+
106+
*Users do not need to install Python as the app will be run in the
107+
Docker Container*
108+
109+
Instructions
110+
~~~~~~~~~~~~
111+
112+
1. Run ``docker-compose up -d`` in the the ``examples/`` directory
113+
114+
The ``-d`` flag causes all services to run in detached mode and frees up
115+
your terminal session. This also causes no logs to show up. Users can
116+
attach themselves to the service’s logs manually using
117+
``docker logs ${CONTAINER_ID} --follow``
118+
119+
2. Log into the Grafana instance at http://localhost:3000
120+
121+
- login credentials are ``username: admin`` and ``password: admin``
122+
- There may be an additional screen on setting a new password. This
123+
can be skipped and is optional
124+
125+
3. Navigate to the ``Data Sources`` page
126+
127+
- Look for a gear icon on the left sidebar and select
128+
``Data Sources``
129+
130+
4. Add a new Prometheus Data Source
131+
132+
- Use ``http://cortex:9009/api/prom`` as the URL
133+
- Set the scrape interval to ``2s`` to make updates
134+
appear quickly **(Optional)**
135+
- click ``Save & Test``
136+
137+
5. Go to ``Metrics Explore`` to query metrics
138+
139+
- Look for a compass icon on the left sidebar
140+
- click ``Metrics`` for a dropdown list of all the available metrics
141+
- Adjust time range by clicking the ``Last 6 hours``
142+
button on the upper right side of the graph **(Optional)**
143+
- Set up auto-refresh by selecting an option under the
144+
dropdown next to the refresh button on the upper right side of the
145+
graph **(Optional)**
146+
- Click the refresh button and data should show up on hte graph
147+
148+
6. Shutdown the services when finished
149+
150+
- Run ``docker-compose down`` in the examples directory
151+
152+
Configuring the Exporter
153+
------------------------
154+
155+
The exporter can be configured through parameters passed to the
156+
constructor. Here are all the options:
157+
158+
- ``endpoint``: url where data will be sent **(Required)**
159+
- ``basic_auth``: username and password for authentication
160+
**(Optional)**
161+
- ``headers``: additional headers for remote write request as
162+
determined by the remote write backend's API **(Optional)**
163+
- ``timeout``: timeout for requests to the remote write endpoint in
164+
seconds **(Optional)**
165+
- ``proxies``: dict mapping request proxy protocols to proxy urls
166+
**(Optional)**
167+
- ``tls_config``: configuration for remote write TLS settings
168+
**(Optional)**
169+
170+
Example with all the configuration options:
171+
172+
.. code:: python
173+
174+
exporter = PrometheusRemoteWriteMetricsExporter(
175+
endpoint="http://localhost:9009/api/prom/push",
176+
timeout=30,
177+
basic_auth={
178+
"username": "user",
179+
"password": "pass123",
180+
},
181+
headers={
182+
"X-Scope-Org-ID": "5",
183+
"Authorization": "Bearer mytoken123",
184+
},
185+
proxies={
186+
"http": "http://10.10.1.10:3000",
187+
"https": "http://10.10.1.10:1080",
188+
},
189+
tls_config={
190+
"cert_file": "path/to/file",
191+
"key_file": "path/to/file",
192+
"ca_file": "path_to_file",
193+
"insecure_skip_verify": true, # for developing purposes
194+
}
195+
)
196+
197+
Securing the Exporter
198+
---------------------
199+
200+
Authentication
201+
~~~~~~~~~~~~~~
202+
203+
The exporter provides two forms of authentication which are shown below.
204+
Users can add their own custom authentication by setting the appropriate
205+
values in the ``headers`` dictionary
206+
207+
1. Basic Authentication Basic authentication sets a HTTP Authorization
208+
header containing a base64 encoded username/password pair. See `RFC
209+
7617`_ for more information. This
210+
211+
.. code:: python
212+
213+
exporter = PrometheusRemoteWriteMetricsExporter(
214+
basic_auth={"username": "base64user", "password": "base64pass"}
215+
)
216+
217+
2. Bearer Token Authentication This custom configuration can be achieved
218+
by passing in a custom ``header`` to the constructor. See `RFC 6750`_
219+
for more information.
220+
221+
.. code:: python
222+
223+
header = {
224+
"Authorization": "Bearer mytoken123"
225+
}
226+
227+
TLS
228+
~~~
229+
230+
Users can add TLS to the exporter's HTTP Client by providing certificate
231+
and key files in the ``tls_config`` parameter.
232+
233+
Supported Aggregators
234+
---------------------
235+
Behaviour of these aggregators is outlined in the `OpenTelemetry Specification <https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/metrics/api.md#aggregations>`_.
236+
All aggregators are converted into the `timeseries`_ data format. However, method in
237+
which they are converted `differs <https://github.com/open-telemetry/opentelemetry-python-contrib/blob/master/exporter/opentelemetry-exporter-prometheus-remote-write/src/opentelemetry/exporter/prometheus_remote_write/__init__.py#L196>`_ from aggregator to aggregator. A
238+
map of the conversion methods can be found `here <https://github.com/open-telemetry/opentelemetry-python-contrib/blob/master/exporter/opentelemetry-exporter-prometheus-remote-write/src/opentelemetry/exporter/prometheus_remote_write/__init__.py#L75>`_.
239+
240+
+------------------------------+-------------------------------------+------------------------------------------------------------------------------------------------------------+
241+
| **OpenTelemetry Aggregator** | **Equivalent Prometheus Data Type** | **Behaviour** |
242+
+------------------------------+-------------------------------------+------------------------------------------------------------------------------------------------------------+
243+
| Sum | Counter | Metric value can only go up or be reset to 0 |
244+
+------------------------------+-------------------------------------+------------------------------------------------------------------------------------------------------------+
245+
| MinMaxSumCount | Gauge | Metric value can arbitrarily increment or decrement |
246+
+------------------------------+-------------------------------------+------------------------------------------------------------------------------------------------------------+
247+
| Histogram | Histogram | Unlike the Prometheus histogram, the OpenTelemetry Histogram does not provide a sum of all observed values |
248+
+------------------------------+-------------------------------------+------------------------------------------------------------------------------------------------------------+
249+
| LastValue | N/A | Metric only contains the most recently observed value |
250+
+------------------------------+-------------------------------------+------------------------------------------------------------------------------------------------------------+
251+
| ValueObserver | N/A | Similar to MinMaxSumCount but also contains LastValue |
252+
+------------------------------+-------------------------------------+------------------------------------------------------------------------------------------------------------+
253+
254+
255+
Error Handling
256+
--------------
257+
258+
In general, errors are raised by the calling function. The exception is
259+
for failed requests where any error status code is logged as a warning
260+
instead.
261+
262+
This is because the exporter does not implement any retry logic as data that
263+
failed to export will be dropped.
264+
265+
For example, consider a situation where a user increments a Counter
266+
instrument 5 times and an export happens between each increment. If the
267+
exports happen like so:
268+
269+
::
270+
271+
SUCCESS FAIL FAIL SUCCESS SUCCESS
272+
1 2 3 4 5
273+
274+
Then the received data will be:
275+
276+
::
277+
278+
1 4 5
279+
280+
Contributing
281+
------------
282+
283+
If you would like to learn more about the exporter's structure and
284+
design decisions please view the design document below
285+
286+
Design Doc
287+
~~~~~~~~~~
288+
289+
`Design Document`_
290+
291+
This document is stored elsewhere as it contains large images which will
292+
significantly increase the size of this repo.
293+
294+
.. _Summary: #opentelemetry-python-sdk-prometheus-remote-write-exporter
295+
.. _Table of Contents: #table-of-contents
296+
.. _Installation: #installation
297+
.. _Quickstart: #quickstart
298+
.. _Examples: #examples
299+
.. _Configuring the Exporter: #configuring-the-exporter
300+
.. _Securing the Exporter: #securing-the-exporter
301+
.. _Authentication: #authentication
302+
.. _TLS: #tls
303+
.. _Supported Aggregators: #supported-aggregators
304+
.. _Error Handling: #error-handling
305+
.. _Contributing: #contributing
306+
.. _Design Doc: #design-doc
307+
.. |Prometheus SDK pipelines| image:: https://user-images.githubusercontent.com/20804975/100285430-e320fd80-2f3e-11eb-8217-a562c559153c.png
308+
.. _RFC 7617: https://tools.ietf.org/html/rfc7617
309+
.. _RFC 6750: https://tools.ietf.org/html/rfc6750
310+
.. _Design Document: https://github.com/open-o11y/docs/blob/master/python-prometheus-remote-write/design-doc.md
311+
.. _OTLP: https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/protocol/otlp.md
312+
.. _OpenTelemetry Python SDK: https://github.com/open-telemetry/opentelemetry-python
313+
.. _Prometheus "pull" exporter: https://github.com/open-telemetry/opentelemetry-python/tree/master/exporter/opentelemetry-exporter-prometheus
314+
.. _Prometheus Remote Write integrated backend: https://prometheus.io/docs/operating/integrations/
315+
.. _types.proto: https://github.com/prometheus/prometheus/blob/master/prompb/types.proto
316+
.. _remote.proto: https://github.com/prometheus/prometheus/blob/master/prompb/remote.proto
317+
.. _push controller: https://github.com/open-telemetry/opentelemetry-python/blob/master/opentelemetry-sdk/src/opentelemetry/sdk/metrics/export/controller.py#L22
318+
.. _timeseries: https://prometheus.io/docs/concepts/data_model/
319+
.. _Docker Compose: https://docs.docker.com/compose/
320+
.. _Cortex: https://cortexmetrics.io/
321+
.. _Grafana: https://grafana.com/
322+
.. _installed: https://docs.docker.com/compose/install/

‎exporter/opentelemetry-exporter-prometheus-remote-write/examples/Dockerfile

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
FROM python:3.7
2+
WORKDIR /code
3+
4+
COPY . .
5+
RUN apt-get update -y && apt-get install libsnappy-dev -y
6+
RUN pip install -e .
7+
RUN pip install -r ./examples/requirements.txt
8+
CMD ["python", "./examples/sampleapp.py"]

0 commit comments

Comments
 (0)