Skip to content

PeriodicMetricExporter does not export periodically for either Delta or Cumulative temporality ExponentialHistogram #3971

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alexchowle opened this issue Jun 14, 2024 · 1 comment
Labels
bug Something isn't working

Comments

@alexchowle
Copy link

Describe your environment

OS: Centos 7
Python version: 3.11.1
SDK version: 1.25.0
API version: 1.25.0

What happened?

I have a low-throughput service and have implemented my own Histogram to measure the duration of an operation. This uses a PeriodicMetricExporter which is configured to export every 60 seconds. When I use Delta temporality, I get an export within 60 seconds of the last histogram recording. I then have no exports until the next observation is made.

When changing to Cumulative temporality I have the exact same experience. For comparison purposes, when using the Golang SDK I get the same behaviour as Python's Delta but, when using Cumulative, I get a metronomic export every 60 seconds regardless of any observations being recorded..

Should the Python PeriodicMetricExporter export every 60 seconds or not, and how does temporality setting play into it? At the moment it causes me a problem because my service has low throughput. My observability backend (New Relic) uses the StartTimeUnixNano value which can be hours in the past. A constant periodic export would help to correct this as the StartTimeUnixNano would be at most 60 seconds in the past.

Steps to Reproduce

metrics_temporality = {
        Histogram: AggregationTemporality.DELTA
    }
otlp_metric_exporter = OTLPMetricExporter(endpoint=grpc_collector_address, insecure=True,
                                              preferred_temporality=metrics_temporality, timeout=30_000)
metric_reader = PeriodicExportingMetricReader(otlp_metric_exporter,
                                                  export_interval_millis=60_000))
meter_provider = MeterProvider(metric_readers=[metric_reader], resource=_create_resources(),
                                   views=[View(instrument_type=Histogram,
                                               aggregation=ExponentialBucketHistogramAggregation())])
metrics.set_meter_provider(meter_provider)

meter = metrics.get_meter(__name__)

h = meter.create_histogram(
        name='my_histogram', unit='s', description='just my histogram'
    )

h.record(1.0)
time.sleep(120)
h.record(2.0)
    

Expected Result

I expected to see periodic metric exports for the histogram every 60 seconds, regardless of chosen temporality

Actual Result

A metric export within 60 seconds of the last observation.

Additional context

N/A

Would you like to implement a fix?

None

@alexchowle alexchowle added the bug Something isn't working label Jun 14, 2024
@aabmass
Copy link
Member

aabmass commented Jun 14, 2024

@ocelotl

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants