Monitoring Listener Performance with multiple MeterRegistry beans

[garyrussell]

Discussed in #2105

^{Originally posted by alexanderad February 11, 2022}
Hi!

Recently I've been looking for a way to monitor listener specific metrics and found a corresponding section in the docs (Monitoring Listener Performance) saying

... the listener container will automatically create and update Micrometer Timers for the listener, if Micrometer is detected on the class path, and a single MeterRegistry is present in the application context ...

This way of collecting listener metrics is particulary useful given that it is able to account for multiple types of listeners (e.g. declared with @KafkaListener, or @KafkaHandler, or being batch listeners, or hidden behind custom factories for consumers, etc); as well as aware of concurrent listeners, and templates.

The corresponding implementation to be found in:

spring-kafka/spring-kafka/src/main/java/org/springframework/kafka/support/micrometer/MicrometerHolder.java

Lines 61 to 79 in 7c2178a

	public MicrometerHolder(@Nullable ApplicationContext context, String name,
	String timerName, String timerDesc, Map<String, String> tags) {
	if (context == null) {
	throw new IllegalStateException("No micrometer registry present");
	}
	Map<String, MeterRegistry> registries = context.getBeansOfType(MeterRegistry.class, false, false);
	this.timerName = timerName;
	this.timerDesc = timerDesc;
	this.name = name;
	this.tags = tags;
	if (registries.size() == 1) {
	this.registry = registries.values().iterator().next();
	buildTimer(NONE_EXCEPTION_METERS_KEY);
	}
	else {
	throw new IllegalStateException("No micrometer registry present (or more than one)");
	}
	}

The problem I faced is that in case when multiple registries are auto-configured for the application context, e.g. Prometheus + Graphite timers for metrics do not register themselves.This applies to any combination of monitoring systems providing more that one MeterRegistry.class bean visible via context.getBeansOfType(MeterRegistry.class, ...). KafkaMessageListenerContainer silently rejects to collect these metrics, and it seems there is no approachable way to configure alternative behavior, e.g.:

• prefer a specific registry (say, provided via containerProperties)
• combine discovered registries into CompositeMeterRegistry

I'm looking for an advice here. Is there anything obvious I'm missing? Is the only one way reasonable in here would be to patch spring-kafka itself to achieve metrics collection for the case of multiple metric registries in use?

Thanks!