Skip to content

Commit c5a026f

Browse files
committed
fix kafka: wait for metadata
Kafka's instance metadata could be unavailable (because it's being filled asynchronously). extract_send_partition() is based on a metadata, so it may return `None` for partition and later cause all type of warning messages (e.g. `Invalid type NoneType for attribute value. Expected one of ['bool', 'str', 'bytes', 'int', 'float'] or a sequence of those types`). The proposed fix makes sure metadata is pre-populated (based on https://github.com/dpkp/kafka-python/blob/4d598055dab7da99e41bfcceffa8462b32931cdd/kafka/producer/kafka.py#L579). I'm just not sure if we should wrap `_wait_on_metadata` into try\except, maybe just passing Exception to the caller would be a better idea...
1 parent 8107ad4 commit c5a026f

File tree

1 file changed

+5
-0
lines changed
  • instrumentation/opentelemetry-instrumentation-kafka-python/src/opentelemetry/instrumentation/kafka

1 file changed

+5
-0
lines changed

instrumentation/opentelemetry-instrumentation-kafka-python/src/opentelemetry/instrumentation/kafka/utils.py

+5
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
from logging import getLogger
33
from typing import Callable, Dict, List, Optional
44

5+
from kafka.errors import errors as KafkaErrors
56
from kafka.record.abc import ABCRecord
67

78
from opentelemetry import context, propagate, trace
@@ -146,6 +147,10 @@ def _traced_send(func, instance, args, kwargs):
146147
kwargs["headers"] = headers
147148

148149
topic = KafkaPropertiesExtractor.extract_send_topic(args, kwargs)
150+
try:
151+
instance._wait_on_metadata(topic, instance.config['max_block_ms'] / 1000.0)
152+
except KafkaErrors.BrokerResponseError as kafka_exception:
153+
_LOG.exception(kafka_exception)
149154
bootstrap_servers = KafkaPropertiesExtractor.extract_bootstrap_servers(
150155
instance
151156
)

0 commit comments

Comments
 (0)