Automatic exporter/provider setup for opentelemetry-instrument command. #1036

This commit adds support to the opentelemetry-instrument command to
automatically configure a tracer provider and exporter. By default,
it configures the OTLP exporter (like other Otel auto-instrumentations.
e.g, Java: https://github.com/open-telemetry/opentelemetry-java-instrumentation#getting-started).
It also allows using a different in-built or 3rd party via a CLI argument or env variable.

Details can be found on opentelemetry-instrumentation's README package.

Fixes #663

Author: owais

opentelemetry-instrumentation/README.rst

@@ -16,30 +16,92 @@ Installation
 
 This package provides a couple of commands that help automatically instruments a program:
 
+
+opentelemetry-bootstrap
+-----------------------
+
+::
+
+    opentelemetry-bootstrap --action=install|requirements
+
+This commands inspects the active Python site-packages and figures out which
+instrumentation packages the user might want to install. By default it prints out
+a list of the suggested instrumentation packages which can be added to a requirements.txt
+file. It also supports installing the suggested packages when run with :code:`--action=install`
+flag.
+
+The command also installs the OTLP exporter by default. This can be overriden by specifying another
+exporter using the `--exporter` or `-e` CLI flag. Multiple exporters can be installed by specifying
+the flag more than once. Run `opentelemetry-bootstrap --help` to list down all supported exporters.
+
+Manually specifying exporters to install:
+
+::
+
+    opentelemetry-bootstrap -e=otlp -e=zipkin
+
+
 opentelemetry-instrument
 ------------------------
 
 ::
 
     opentelemetry-instrument python program.py
 
+The instrument command will try to automatically detect packages used by your python program
+and when possible, apply automatic tracing instrumentation on them. This means your program
+will get automatic distrubuted tracing for free without having to make any code changes
+at all. This will also configure a global tracer and tracing exporter without you having to
+make any code changes. By default, the instrument command will use the OTLP exporter but
+this can be overrided when needed.
+
+The command supports the following configuration options as CLI arguments and environments vars:
+
+
+* ``--trace-exporter`` or ``OTEL_TRACE_EXPORTER``
+
+Used to specify which trace exporter to use. Can be set to one of the well-known
+exporter names (see below) or a fully qualified Python import path to a trace
+exporter implementation.
+
+    - Defaults to `otlp`.
+    - Can be set to `none` to disbale automatic tracer initialization. 
+
+Well known trace exporter names:
+
+    - datadog
+    - jaeger
+    - opencensus
+    - otlp
+    - zipkin
+
+* ``--tracer-provider`` or ``OTEL_TRACER_PROVIDER``
+
+Must be a fully qualified Python import path to a Tracer Provider implementation or
+a callable that returns a tracer provider instance.
+
+Defaults to `opentelemetry.sdk.trace.TracerProvider`
+
+
+* ``--service-name`` or ``OTEL_SERVICE_NAME``
+
+When present the value is passed on to the relevant exporter initializer as ``service_name`` argument.
+
 The code in ``program.py`` needs to use one of the packages for which there is
 an OpenTelemetry integration. For a list of the available integrations please
 check `here <https://opentelemetry-python.readthedocs.io/en/stable/index.html#integrations>`_
 
+Passing arguments to program
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
-opentelemetry-bootstrap
------------------------
+Any arguments passed apply to the instrument command by default. You can still pass arguments to your program by
+separating them from the rest of the command with ``--``. For example, 
 
 ::
 
-    opentelemetry-bootstrap --action=install|requirements
+    opentelemetry-instrument -e otlp flask run -- --port=3000
 
-This commands inspects the active Python site-packages and figures out which
-instrumentation packages the user might want to install. By default it prints out
-a list of the suggested instrumentation packages which can be added to a requirements.txt
-file. It also supports installing the suggested packages when run with :code:`--action=install`
-flag.
+The above command will pass ``-e otlp` to the instrument command and ``--port=3000`` to ``flask run``.
 
 References
 ----------

opentelemetry-instrumentation/src/opentelemetry/instrumentation/auto_instrumentation/__init__.py

@@ -14,16 +14,80 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import argparse
 from logging import getLogger
 from os import environ, execl, getcwd
 from os.path import abspath, dirname, pathsep
 from shutil import which
-from sys import argv
+
+from opentelemetry.instrumentation import symbols
 
 logger = getLogger(__file__)
 
 
+def parse_args():
+    parser = argparse.ArgumentParser(
+        description="""
+        opentelemetry-instrument automatically instruments a Python
+        program and it's dependencies and then runs the program.
+        """
+    )
+
+    parser.add_argument(
+        "-tp",
+        "--tracer-provider",
+        required=False,
+        help="""
+        Uses the specified tracer provider.
+        Must be a fully qualified python import path to a tracer
+        provider implementation or a callable that returns a new
+        instance of a tracer provider.
+        """,
+    )
+
+    parser.add_argument(
+        "-e",
+        "--exporter",
+        required=False,
+        help="""
+        Uses the specified exporter to export spans.
+
+        Must be one of the following:
+            - Name of a well-known trace exporter. Choices are:
+                {0}
+            - A fully qualified python import path to a trace exporter
+              implementation or a callable that returns a new instance
+              of a trace exporter.
+        """.format(
+            symbols.trace_exporters
+        ),
+    )
+
+    parser.add_argument(
+        "-s",
+        "--service-name",
+        required=False,
+        help="""
+        The service name that should be passed to a trace exporter.
+        """,
+    )
+
+    parser.add_argument("command", nargs="+")
+    return parser.parse_args()
+
+
+def set_default_env_vars(args):
+    if args.exporter:
+        environ["OTEL_TRACE_EXPORTER"] = args.exporter
+    if args.tracer_provider:
+        environ["OTEL_TRACE_PROVIDER"] = args.tracer_provider
+    if args.service_name:
+        environ["OTEL_SERVICE_NAME"] = args.service_name
+
+
 def run() -> None:
+    args = parse_args()
+    set_default_env_vars(args)
 
     python_path = environ.get("PYTHONPATH")
 
@@ -49,6 +113,6 @@ def run() -> None:
 
     environ["PYTHONPATH"] = pathsep.join(python_path)
 
-    executable = which(argv[1])
-
-    execl(executable, executable, *argv[2:])
+    command = args.command
+    executable = which(command[0])
+    execl(executable, executable, *command[1:])

opentelemetry-instrumentation/src/opentelemetry/instrumentation/auto_instrumentation/sitecustomize.py

@@ -16,13 +16,22 @@
 
 from pkg_resources import iter_entry_points
 
+from opentelemetry.instrumentation.auto_instrumentation.tracing import (
+    initialize_tracing,
+)
+
 logger = getLogger(__file__)
 
 
-for entry_point in iter_entry_points("opentelemetry_instrumentor"):
-    try:
-        entry_point.load()().instrument()  # type: ignore
-        logger.debug("Instrumented %s", entry_point.name)
+def auto_instrument():
+    for entry_point in iter_entry_points("opentelemetry_instrumentor"):
+        try:
+            entry_point.load()().instrument()  # type: ignore
+            logger.debug("Instrumented %s", entry_point.name)
+
+        except Exception:  # pylint: disable=broad-except
+            logger.exception("Instrumenting of %s failed", entry_point.name)
+
 
-    except Exception:  # pylint: disable=broad-except
-        logger.exception("Instrumenting of %s failed", entry_point.name)
+initialize_tracing()
+auto_instrument()

opentelemetry-instrumentation/src/opentelemetry/instrumentation/auto_instrumentation/tracing.py

@@ -0,0 +1,152 @@
+# Copyright The OpenTelemetry Authors
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from collections import defaultdict
+from functools import partial
+from logging import getLogger
+
+from opentelemetry import trace
+from opentelemetry.configuration import Configuration
+from opentelemetry.instrumentation import symbols
+from opentelemetry.sdk.resources import Resource
+
+logger = getLogger(__file__)
+
+
+# Defaults
+_DEFAULT_TRACE_EXPORTER = symbols.exporter_otlp
+_DEFAULT_TRACER_PROVIDER = "opentelemetry.sdk.trace.TracerProvider"
+_DEFAULT_SPAN_PROCESSOR = (
+    "opentelemetry.sdk.trace.export.BatchExportSpanProcessor"
+)
+
+
+_trace_exporter_classes = {
+    symbols.exporter_dd: (
+        "opentelemetry.exporter.datadog.DatadogSpanExporter"
+    ),
+    symbols.exporter_oc: (
+        "opentelemetry.exporter.opencensus"
+        ".trace_exporter.OpenCensusSpanExporter"
+    ),
+    symbols.exporter_otlp: (
+        "opentelemetry.exporter.otlp.trace_exporter.OTLPSpanExporter"
+    ),
+    symbols.exporter_jaeger: (
+        "opentelemetry.exporter.jaeger.JaegerSpanExporter"
+    ),
+    symbols.exporter_zipkin: (
+        "opentelemetry.exporter.zipkin.ZipkinSpanExporter"
+    ),
+}
+
+_span_processors_by_exporter = defaultdict(
+    lambda: _DEFAULT_SPAN_PROCESSOR,
+    {
+        symbols.exporter_dd: (
+            "opentelemetry.exporter.datadog.DatadogExportSpanProcessor"
+        )
+    },
+)
+
+
+def get_service_name():
+    return Configuration().SERVICE_NAME or ""
+
+
+def get_tracer_provider():
+    return Configuration().TRACER_PROVIDER or _DEFAULT_TRACER_PROVIDER
+
+
+def get_exporter_name():
+    return Configuration().TRACE_EXPORTER or _DEFAULT_TRACE_EXPORTER
+
+
+def _trace_init(
+    trace_exporter, tracer_provider, span_processor,
+):
+    exporter = trace_exporter()
+    processor = span_processor(exporter)
+    provider = tracer_provider()
+    trace.set_tracer_provider(provider)
+    provider.add_span_processor(processor)
+
+
+def _default_trace_init(exporter, provider, processor):
+    service_name = get_service_name()
+    if service_name:
+        exporter = partial(exporter, service_name=get_service_name())
+    _trace_init(exporter, provider, processor)
+
+
+def _otlp_trace_init(exporter, provider, processor):
+    resource = Resource(labels={"service_name": get_service_name()})
+    provider = partial(provider, resource=resource)
+    _trace_init(exporter, provider, processor)
+
+
+def _dd_trace_init(exporter, provider, processor):
+    exporter = partial(exporter, service=get_service_name())
+    _trace_init(exporter, provider, processor)
+
+
+_initializers = defaultdict(
+    lambda: _default_trace_init,
+    {
+        symbols.exporter_dd: _dd_trace_init,
+        symbols.exporter_otlp: _otlp_trace_init,
+    },
+)
+
+
+def _import(import_path):
+    split_path = import_path.rsplit(".", 1)
+    if len(split_path) < 2:
+        raise ImportError(
+            "could not import module or class: {0}".format(import_path)
+        )
+    module, class_name = split_path
+    mod = __import__(module, fromlist=[class_name])
+    return getattr(mod, class_name)
+
+
+def _load_component(components, name):
+    if name.lower() == "none":
+        return
+
+    component = components.get(name.lower(), name)
+    if not component:
+        logger.info("component not found with name: %s", format(name))
+        return
+
+    if isinstance(component, str):
+        try:
+            return _import(component)
+        except ImportError as exc:
+            logger.error(exc.msg)
+            return
+    return component
+
+
+def initialize_tracing():
+    exporter_name = get_exporter_name()
+    trace_exporter = _load_component(_trace_exporter_classes, exporter_name)
+    if trace_exporter is None:
+        logger.info("not using any trace exporter")
+        return
+
+    tracer_provider = _load_component({}, get_tracer_provider())
+    span_processor = _import(_span_processors_by_exporter[exporter_name])
+    initializer = _initializers[exporter_name]
+    initializer(trace_exporter, tracer_provider, span_processor)