Add an autoinstrumentation mechanism and an instrumentor for Flask (#327)

Adding an autoinstrumentation mechanism and a Flask instrumentor (an instrumentor is a class that implements the _instrument and _uninstrument methods).

It works like this:

A console command is defined. This makes it possible to run a command named opentelemetry-auto-instrumentation that will execute this function.

When the opentelemetry-auto-instrumentation command is executed, then the instrument method of the different instrumentors is called, which are made available via an entry-point.

2.In the case of the Flask instrumentor, the original flask.Flask gets replaced with _InstrumentedFlask (in this case, the Flask instrumentor uses monkey patching to perform the instrumentation, nevertheless, monkey patching is not always the method used to do this, so the name instrumentor is preferred over patcher).

Once all instrumentation is enabled, the app is executed.

Co-Authored-By: Mauricio Vásquez <[email protected]>
Co-authored-by: Chris Kleinknecht <[email protected]>

Author: 3 people

docs/auto_instrumentation/auto_instrumentation.rst

@@ -0,0 +1,7 @@
+OpenTelemetry Python Autoinstrumentation
+========================================
+
+.. toctree::
+    :maxdepth: 1
+
+    instrumentor

docs/auto_instrumentation/instrumentor.rst

@@ -0,0 +1,7 @@
+opentelemetry.auto_instrumentation.instrumentor package
+=======================================================
+
+.. automodule:: opentelemetry.auto_instrumentation.instrumentor
+    :members:
+    :undoc-members:
+    :show-inheritance:

docs/conf.py

@@ -18,6 +18,7 @@
 source_dirs = [
     os.path.abspath("../opentelemetry-api/src/"),
     os.path.abspath("../opentelemetry-sdk/src/"),
+    os.path.abspath("../opentelemetry-auto-instrumentation/src/"),
 ]
 
 ext = "../ext"

docs/examples/auto-instrumentation/README.md

@@ -0,0 +1,112 @@
+# Overview
+
+This example shows how to use auto-instrumentation in OpenTelemetry. This example is also based on a previous example
+for OpenTracing that can be found [here](https://github.com/yurishkuro/opentracing-tutorial/tree/master/python).
+
+This example uses 2 scripts whose main difference is they being instrumented manually or not:
+
+1. `server_instrumented.py` which has been instrumented manually
+2. `server_uninstrumented.py` which has not been instrumented manually
+
+The former will be run without the automatic instrumentation agent and the latter with the automatic instrumentation
+agent. They should produce the same result, showing that the automatic instrumentation agent does the equivalent
+of what manual instrumentation does.
+
+In order to understand this better, here is the relevant part of both scripts:
+
+## Manually instrumented server
+
+`server_instrumented.py`
+
+```python
+@app.route("/server_request")
+def server_request():
+    with tracer.start_as_current_span(
+        "server_request",
+        parent=propagators.extract(
+            lambda dict_, key: dict_.get(key, []), request.headers
+        )["current-span"],
+    ):
+        print(request.args.get("param"))
+        return "served"
+```
+
+## Publisher not instrumented manually
+
+`server_uninstrumented.py`
+
+```python
+@app.route("/server_request")
+def server_request():
+    print(request.args.get("param"))
+    return "served"
+```
+
+# Preparation
+
+This example will be executed in a separate virtual environment:
+
+```sh
+$ mkdir auto_instrumentation
+$ virtualenv auto_instrumentation
+$ source auto_instrumentation/bin/activate
+```
+
+# Installation
+
+```sh
+$ pip install opentelemetry-api
+$ pip install opentelemetry-sdk
+$ pip install opentelemetry-auto-instrumentation
+$ pip install ext/opentelemetry-ext-flask
+$ pip install flask
+$ pip install requests
+```
+
+# Execution
+
+## Execution of the manually instrumented server
+
+This is done in 2 separate consoles, one to run each of the scripts that make up this example:
+
+```sh
+$ source auto_instrumentation/bin/activate
+$ python opentelemetry-python/opentelemetry-auto-instrumentation/example/server_instrumented.py
+```
+
+```sh
+$ source auto_instrumentation/bin/activate
+$ python opentelemetry-python/opentelemetry-auto-instrumentation/example/client.py testing
+```
+
+The execution of `server_instrumented.py` should return an output similar to:
+
+```sh
+Hello, testing!
+Span(name="serv_request", context=SpanContext(trace_id=0x9c0e0ce8f7b7dbb51d1d6e744a4dad49, span_id=0xd1ba3ec4c76a0d7f, trace_state={}), kind=SpanKind.INTERNAL, parent=None, start_time=2020-03-19T00:06:31.275719Z, end_time=2020-03-19T00:06:31.275920Z)
+127.0.0.1 - - [18/Mar/2020 18:06:31] "GET /serv_request?helloStr=Hello%2C+testing%21 HTTP/1.1" 200 -
+```
+
+## Execution of an automatically instrumented server
+
+Now, kill the execution of `server_instrumented.py` with `ctrl + c` and run this instead:
+
+```sh
+$ opentelemetry-auto-instrumentation opentelemetry-python/opentelemetry-auto-instrumentation/example/server_uninstrumented.py
+```
+
+In the console where you previously executed `client.py`, run again this again:
+
+```sh
+$ python opentelemetry-python/opentelemetry-auto-instrumentation/example/client.py testing
+```
+
+The execution of `server_uninstrumented.py` should return an output similar to:
+
+```sh
+Hello, testing!
+Span(name="serv_request", context=SpanContext(trace_id=0xf26b28b5243e48f5f96bfc753f95f3f0, span_id=0xbeb179a095d087ed, trace_state={}), kind=SpanKind.SERVER, parent=<opentelemetry.trace.DefaultSpan object at 0x7f1a20a54908>, start_time=2020-03-19T00:24:18.828561Z, end_time=2020-03-19T00:24:18.845127Z)
+127.0.0.1 - - [18/Mar/2020 18:24:18] "GET /serv_request?helloStr=Hello%2C+testing%21 HTTP/1.1" 200 -
+```
+
+As you can see, both outputs are equivalentsince the automatic instrumentation does what the manual instrumentation does too.

docs/examples/auto-instrumentation/client.py

@@ -0,0 +1,50 @@
+# Copyright The OpenTelemetry Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from sys import argv
+
+from flask import Flask
+from requests import get
+
+from opentelemetry import propagators, trace
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import (
+    ConsoleSpanExporter,
+    SimpleExportSpanProcessor,
+)
+
+app = Flask(__name__)
+
+trace.set_tracer_provider(TracerProvider())
+tracer = trace.get_tracer_provider().get_tracer(__name__)
+
+trace.get_tracer_provider().add_span_processor(
+    SimpleExportSpanProcessor(ConsoleSpanExporter())
+)
+
+
+assert len(argv) == 2
+
+with tracer.start_as_current_span("client"):
+
+    with tracer.start_as_current_span("client-server"):
+        headers = {}
+        propagators.inject(dict.__setitem__, headers)
+        requested = get(
+            "http://localhost:8082/server_request",
+            params={"param": argv[1]},
+            headers=headers,
+        )
+
+        assert requested.status_code == 200

docs/examples/auto-instrumentation/server_instrumented.py

@@ -0,0 +1,47 @@
+# Copyright The OpenTelemetry Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from flask import Flask, request
+
+from opentelemetry import propagators, trace
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import (
+    ConsoleSpanExporter,
+    SimpleExportSpanProcessor,
+)
+
+app = Flask(__name__)
+
+trace.set_tracer_provider(TracerProvider())
+tracer = trace.get_tracer_provider().get_tracer(__name__)
+
+trace.get_tracer_provider().add_span_processor(
+    SimpleExportSpanProcessor(ConsoleSpanExporter())
+)
+
+
+@app.route("/server_request")
+def server_request():
+    with tracer.start_as_current_span(
+        "server_request",
+        parent=propagators.extract(
+            lambda dict_, key: dict_.get(key, []), request.headers
+        )["current-span"],
+    ):
+        print(request.args.get("param"))
+        return "served"
+
+
+if __name__ == "__main__":
+    app.run(port=8082)

docs/examples/auto-instrumentation/server_uninstrumented.py

@@ -0,0 +1,40 @@
+# Copyright The OpenTelemetry Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from flask import Flask, request
+
+from opentelemetry import trace
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import (
+    ConsoleSpanExporter,
+    SimpleExportSpanProcessor,
+)
+
+app = Flask(__name__)
+
+trace.set_tracer_provider(TracerProvider())
+
+trace.get_tracer_provider().add_span_processor(
+    SimpleExportSpanProcessor(ConsoleSpanExporter())
+)
+
+
+@app.route("/server_request")
+def server_request():
+    print(request.args.get("param"))
+    return "served"
+
+
+if __name__ == "__main__":
+    app.run(port=8082)

docs/examples/opentelemetry-example-app/src/opentelemetry_example_app/flask_example.py

@@ -21,7 +21,7 @@
 
 import opentelemetry.ext.http_requests
 from opentelemetry import trace
-from opentelemetry.ext.flask import instrument_app
+from opentelemetry.ext.flask import FlaskInstrumentor
 from opentelemetry.sdk.trace import TracerProvider
 from opentelemetry.sdk.trace.export import (
     ConsoleSpanExporter,
@@ -33,9 +33,9 @@
     SimpleExportSpanProcessor(ConsoleSpanExporter())
 )
 
+FlaskInstrumentor().instrument()
 app = flask.Flask(__name__)
 opentelemetry.ext.http_requests.enable(trace.get_tracer_provider())
-instrument_app(app)
 
 
 @app.route("/")

docs/index.rst

@@ -61,14 +61,14 @@ install <https://pip.pypa.io/en/stable/reference/pip_install/#editable-installs>
 
     getting-started
 
-
 .. toctree::
     :maxdepth: 1
     :caption: OpenTelemetry Python Packages
     :name: packages
 
     api/api
     sdk/sdk
+    auto_instrumentation/auto_instrumentation
 
 .. toctree::
     :maxdepth: 2

ext/opentelemetry-ext-flask/setup.py

@@ -23,4 +23,11 @@
 with open(VERSION_FILENAME) as f:
     exec(f.read(), PACKAGE_INFO)
 
-setuptools.setup(version=PACKAGE_INFO["__version__"])
+setuptools.setup(
+    version=PACKAGE_INFO["__version__"],
+    entry_points={
+        "opentelemetry_instrumentor": [
+            "flask = opentelemetry.ext.flask:FlaskInstrumentor"
+        ]
+    },
+)