Skip to content

Prometheus metric target_info with SERVICE_NAME contains a dot #3571

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
kornerc opened this issue Dec 5, 2023 · 2 comments · Fixed by #3572
Closed

Prometheus metric target_info with SERVICE_NAME contains a dot #3571

kornerc opened this issue Dec 5, 2023 · 2 comments · Fixed by #3572
Labels
bug Something isn't working

Comments

@kornerc
Copy link
Contributor

kornerc commented Dec 5, 2023

Describe your environment

  • Python = 3.11.4
  • opentelemetry-api = 1.21.0
  • opentelemetry-sdk = 1.21.0
  • opentelemetry-exporter-prometheus = 0.42b0
  • prometheus-client = 0.19.0
  • Prometheus = 2.48.0
  • Windows

Steps to reproduce
When using the Prometheus exporter and initializing the Resource with the attribute opentelemetry.sdk.resources.SERVICE_NAME as suggested in the docs, a metric target_info gets exported with the attribute service.name.
Prometheus does not like the dot . in the attribute name and fails scraping the target.

Minimal code to reproduce the issue:

import time

from prometheus_client import start_http_server

from opentelemetry import metrics
from opentelemetry.exporter.prometheus import PrometheusMetricReader
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.resources import SERVICE_NAME, Resource

# Service name is required for most backends
resource = Resource(attributes={
    SERVICE_NAME: "your-service-name"
})

# Start Prometheus client
start_http_server(port=8000, addr="localhost")
# Initialize PrometheusMetricReader which pulls metrics from the SDK
# on-demand to respond to scrape requests
reader = PrometheusMetricReader()
provider = MeterProvider(resource=resource, metric_readers=[reader])
metrics.set_meter_provider(provider)

# Creates a meter from the global meter provider
meter = metrics.get_meter("my.meter.name")

work_counter = meter.create_counter(
    "work.counter", unit="1", description="Counts the amount of work done"
)

while True:
    work_counter.add(1, {"work.type": "Debug"})
    print("doing some work...")
    time.sleep(1)

When browsing to localhost:8000/metrics these are the metrics:

# HELP python_gc_objects_collected_total Objects collected during gc
# TYPE python_gc_objects_collected_total counter
python_gc_objects_collected_total{generation="0"} 356.0
python_gc_objects_collected_total{generation="1"} 289.0
python_gc_objects_collected_total{generation="2"} 0.0
# HELP python_gc_objects_uncollectable_total Uncollectable objects found during GC
# TYPE python_gc_objects_uncollectable_total counter
python_gc_objects_uncollectable_total{generation="0"} 0.0
python_gc_objects_uncollectable_total{generation="1"} 0.0
python_gc_objects_uncollectable_total{generation="2"} 0.0
# HELP python_gc_collections_total Number of times this generation was collected
# TYPE python_gc_collections_total counter
python_gc_collections_total{generation="0"} 55.0
python_gc_collections_total{generation="1"} 4.0
python_gc_collections_total{generation="2"} 0.0
# HELP python_info Python platform information
# TYPE python_info gauge
python_info{implementation="CPython",major="3",minor="11",patchlevel="4",version="3.11.4"} 1.0
# HELP target_info Target metadata
# TYPE target_info gauge
target_info{service.name="your-service-name"} 1.0
# HELP work_counter_1_total Counts the amount of work done
# TYPE work_counter_1_total counter
work_counter_1_total{work_type="Debug"} 9.0

When checking the status of the target in the Prometheus UI, the following error is displayed: expected equal, got "." ("INVALID") while parsing: "target_info{service." because of the dot ..

What is the expected behavior?
The attribute of the metric target_info should not have a dot . in its name.
E.g. service_name.

What is the actual behavior?
The attribute of the metric target_info has a dot . in its name service.name (target_info{service.name="your-service-name"} 1.0).

@kornerc kornerc added the bug Something isn't working label Dec 5, 2023
@kornerc
Copy link
Contributor Author

kornerc commented Dec 7, 2023

I was able to find the cause of the issue, it is this line

info.add_metric(labels=list(attributes.keys()), value=attributes)

for each key of attributes the method _sanitize has to be called

I'll write a pull request to fix this issue.

@Jdsleppy
Copy link

Jdsleppy commented Dec 7, 2023

Linking to duplicate #3563 to close it when your PR is merged. Thank you for the PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants