You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Operator should first deploy two replicas of a tomcat instance and as soon as they become ready deploy the webapp.
What did you see instead? Under which circumstances?
Operator deploys two replicas of a tomcat instance, both became ready, then the WebappReconciler fails with the following exception:
13:40:53.414 ERROR io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:64) - Error during event processing ExecutionScope{ resource id: ResourceID{name='sample-webapp1', namespace='operator-sample-tomcat-issue'}, version: 498543} failed.
java.lang.IllegalStateException: Cannot find Tomcat test-tomcat1 for Webapp sample-webapp1 in namespace operator-sample-tomcat-issue
at io.javaoperatorsdk.operator.sample.WebappReconciler.lambda$reconcile$2(WebappReconciler.java:84) ~[classes/:?]
at java.util.Optional.orElseThrow(Optional.java:403) ~[?:?]
at io.javaoperatorsdk.operator.sample.WebappReconciler.reconcile(WebappReconciler.java:81) ~[classes/:?]
at io.javaoperatorsdk.operator.sample.WebappReconciler.reconcile(WebappReconciler.java:35) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:138) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:96) ~[classes/:?]
at io.javaoperatorsdk.operator.api.monitoring.Metrics.timeControllerExecution(Metrics.java:219) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller.reconcile(Controller.java:95) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.reconcileExecution(ReconciliationDispatcher.java:139) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleReconcile(ReconciliationDispatcher.java:119) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleDispatch(ReconciliationDispatcher.java:89) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:62) [classes/:?]
at io.javaoperatorsdk.operator.processing.event.EventProcessor$ReconcilerExecutor.run(EventProcessor.java:414) [classes/:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:831) [?:?]
This is due to the fact, that the primaryToSecondaryIndex in io.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource is empty until the operator restarts.
Just a workaround: when restarting the operator, everything works fine as long as no new tomcat gets deployed.
For each new tomcat and webapp the same exception rises again until the operator gets restarted.
Additional context
Full log:
13:40:03.236 WARN io.javaoperatorsdk.operator.api.config.BaseConfigurationService.logMissingReconcilerWarning(BaseConfigurationService.java:61) - Configuration for reconciler 'tomcatreconciler' was not found. Known reconcilers: None.
13:40:03.267 INFO io.javaoperatorsdk.operator.api.config.BaseConfigurationService.getConfigurationFor(BaseConfigurationService.java:82) - Created configuration for reconciler io.javaoperatorsdk.operator.sample.TomcatReconciler with name tomcatreconciler
13:40:03.373 INFO io.javaoperatorsdk.operator.Operator.register(Operator.java:223) - Registered reconciler: 'tomcatreconciler' for resource: 'class io.javaoperatorsdk.operator.sample.Tomcat' for namespace(s): [all namespaces]
13:40:03.376 WARN io.javaoperatorsdk.operator.api.config.BaseConfigurationService.logMissingReconcilerWarning(BaseConfigurationService.java:61) - Configuration for reconciler 'webappreconciler' was not found. Known reconcilers: tomcatreconciler.
13:40:03.377 INFO io.javaoperatorsdk.operator.api.config.BaseConfigurationService.getConfigurationFor(BaseConfigurationService.java:82) - Created configuration for reconciler io.javaoperatorsdk.operator.sample.WebappReconciler with name webappreconciler
13:40:03.380 INFO io.javaoperatorsdk.operator.Operator.register(Operator.java:223) - Registered reconciler: 'webappreconciler' for resource: 'class io.javaoperatorsdk.operator.sample.Webapp' for namespace(s): [all namespaces]
13:40:03.382 INFO io.javaoperatorsdk.operator.Operator.start(Operator.java:129) - Operator SDK 4.3.1-SNAPSHOT (commit: 2c51ca7) built on 2023-06-16T13:35:52.000+0200 starting...
13:40:03.383 INFO io.javaoperatorsdk.operator.Operator.start(Operator.java:136) - Client version: 6.5.1
13:40:03.386 INFO io.javaoperatorsdk.operator.processing.Controller.start(Controller.java:320) - Starting 'webappreconciler' controller for reconciler: io.javaoperatorsdk.operator.sample.WebappReconciler, resource: io.javaoperatorsdk.operator.sample.Webapp
13:40:03.386 INFO io.javaoperatorsdk.operator.processing.Controller.start(Controller.java:320) - Starting 'tomcatreconciler' controller for reconciler: io.javaoperatorsdk.operator.sample.TomcatReconciler, resource: io.javaoperatorsdk.operator.sample.Tomcat
13:40:04.686 INFO io.javaoperatorsdk.operator.processing.Controller.start(Controller.java:332) - 'webappreconciler' controller started
13:40:05.020 INFO io.javaoperatorsdk.operator.processing.Controller.start(Controller.java:332) - 'tomcatreconciler' controller started
13:40:33.619 INFO io.javaoperatorsdk.operator.processing.dependent.AbstractDependentResource.logForOperation(AbstractDependentResource.java:110) - Creating 'test-tomcat1' Service for primary ResourceID{name='test-tomcat1', namespace='operator-sample-tomcat-issue'}
13:40:33.679 INFO io.javaoperatorsdk.operator.processing.dependent.AbstractDependentResource.logForOperation(AbstractDependentResource.java:110) - Creating 'test-tomcat1' Deployment for primary ResourceID{name='test-tomcat1', namespace='operator-sample-tomcat-issue'}
13:40:33.830 INFO io.javaoperatorsdk.operator.sample.TomcatReconciler.lambda$reconcile$0(TomcatReconciler.java:33) - Updating status of Tomcat test-tomcat1 in namespace operator-sample-tomcat-issue to 0 ready replicas
13:40:33.974 INFO io.javaoperatorsdk.operator.sample.TomcatReconciler.lambda$reconcile$0(TomcatReconciler.java:33) - Updating status of Tomcat test-tomcat1 in namespace operator-sample-tomcat-issue to 0 ready replicas
13:40:35.887 INFO io.javaoperatorsdk.operator.sample.TomcatReconciler.lambda$reconcile$0(TomcatReconciler.java:33) - Updating status of Tomcat test-tomcat1 in namespace operator-sample-tomcat-issue to 1 ready replicas
13:40:35.952 INFO io.javaoperatorsdk.operator.sample.TomcatReconciler.lambda$reconcile$0(TomcatReconciler.java:33) - Updating status of Tomcat test-tomcat1 in namespace operator-sample-tomcat-issue to 2 ready replicas
13:40:53.414 ERROR io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:64) - Error during event processing ExecutionScope{ resource id: ResourceID{name='sample-webapp1', namespace='operator-sample-tomcat-issue'}, version: 498543} failed.
java.lang.IllegalStateException: Cannot find Tomcat test-tomcat1 for Webapp sample-webapp1 in namespace operator-sample-tomcat-issue
at io.javaoperatorsdk.operator.sample.WebappReconciler.lambda$reconcile$2(WebappReconciler.java:84) ~[classes/:?]
at java.util.Optional.orElseThrow(Optional.java:403) ~[?:?]
at io.javaoperatorsdk.operator.sample.WebappReconciler.reconcile(WebappReconciler.java:81) ~[classes/:?]
at io.javaoperatorsdk.operator.sample.WebappReconciler.reconcile(WebappReconciler.java:35) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:138) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:96) ~[classes/:?]
at io.javaoperatorsdk.operator.api.monitoring.Metrics.timeControllerExecution(Metrics.java:219) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller.reconcile(Controller.java:95) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.reconcileExecution(ReconciliationDispatcher.java:139) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleReconcile(ReconciliationDispatcher.java:119) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleDispatch(ReconciliationDispatcher.java:89) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:62) [classes/:?]
at io.javaoperatorsdk.operator.processing.event.EventProcessor$ReconcilerExecutor.run(EventProcessor.java:414) [classes/:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:831) [?:?]
13:40:55.455 ERROR io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:64) - Error during event processing ExecutionScope{ resource id: ResourceID{name='sample-webapp1', namespace='operator-sample-tomcat-issue'}, version: 498543} failed.
java.lang.IllegalStateException: Cannot find Tomcat test-tomcat1 for Webapp sample-webapp1 in namespace operator-sample-tomcat-issue
at io.javaoperatorsdk.operator.sample.WebappReconciler.lambda$reconcile$2(WebappReconciler.java:84) ~[classes/:?]
at java.util.Optional.orElseThrow(Optional.java:403) ~[?:?]
at io.javaoperatorsdk.operator.sample.WebappReconciler.reconcile(WebappReconciler.java:81) ~[classes/:?]
at io.javaoperatorsdk.operator.sample.WebappReconciler.reconcile(WebappReconciler.java:35) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:138) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:96) ~[classes/:?]
at io.javaoperatorsdk.operator.api.monitoring.Metrics.timeControllerExecution(Metrics.java:219) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller.reconcile(Controller.java:95) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.reconcileExecution(ReconciliationDispatcher.java:139) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleReconcile(ReconciliationDispatcher.java:119) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleDispatch(ReconciliationDispatcher.java:89) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:62) [classes/:?]
at io.javaoperatorsdk.operator.processing.event.EventProcessor$ReconcilerExecutor.run(EventProcessor.java:414) [classes/:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:831) [?:?]
13:40:57.465 ERROR io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:64) - Error during event processing ExecutionScope{ resource id: ResourceID{name='sample-webapp1', namespace='operator-sample-tomcat-issue'}, version: 498543} failed.
java.lang.IllegalStateException: Cannot find Tomcat test-tomcat1 for Webapp sample-webapp1 in namespace operator-sample-tomcat-issue
at io.javaoperatorsdk.operator.sample.WebappReconciler.lambda$reconcile$2(WebappReconciler.java:84) ~[classes/:?]
at java.util.Optional.orElseThrow(Optional.java:403) ~[?:?]
at io.javaoperatorsdk.operator.sample.WebappReconciler.reconcile(WebappReconciler.java:81) ~[classes/:?]
at io.javaoperatorsdk.operator.sample.WebappReconciler.reconcile(WebappReconciler.java:35) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:138) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:96) ~[classes/:?]
at io.javaoperatorsdk.operator.api.monitoring.Metrics.timeControllerExecution(Metrics.java:219) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller.reconcile(Controller.java:95) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.reconcileExecution(ReconciliationDispatcher.java:139) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleReconcile(ReconciliationDispatcher.java:119) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleDispatch(ReconciliationDispatcher.java:89) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:62) [classes/:?]
at io.javaoperatorsdk.operator.processing.event.EventProcessor$ReconcilerExecutor.run(EventProcessor.java:414) [classes/:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:831) [?:?]
13:41:00.564 ERROR io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:64) - Error during event processing ExecutionScope{ resource id: ResourceID{name='sample-webapp1', namespace='operator-sample-tomcat-issue'}, version: 498543} failed.
java.lang.IllegalStateException: Cannot find Tomcat test-tomcat1 for Webapp sample-webapp1 in namespace operator-sample-tomcat-issue
at io.javaoperatorsdk.operator.sample.WebappReconciler.lambda$reconcile$2(WebappReconciler.java:84) ~[classes/:?]
at java.util.Optional.orElseThrow(Optional.java:403) ~[?:?]
at io.javaoperatorsdk.operator.sample.WebappReconciler.reconcile(WebappReconciler.java:81) ~[classes/:?]
at io.javaoperatorsdk.operator.sample.WebappReconciler.reconcile(WebappReconciler.java:35) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:138) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:96) ~[classes/:?]
at io.javaoperatorsdk.operator.api.monitoring.Metrics.timeControllerExecution(Metrics.java:219) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller.reconcile(Controller.java:95) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.reconcileExecution(ReconciliationDispatcher.java:139) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleReconcile(ReconciliationDispatcher.java:119) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleDispatch(ReconciliationDispatcher.java:89) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:62) [classes/:?]
at io.javaoperatorsdk.operator.processing.event.EventProcessor$ReconcilerExecutor.run(EventProcessor.java:414) [classes/:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:831) [?:?]
13:41:05.098 ERROR io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:64) - Error during event processing ExecutionScope{ resource id: ResourceID{name='sample-webapp1', namespace='operator-sample-tomcat-issue'}, version: 498543} failed.
java.lang.IllegalStateException: Cannot find Tomcat test-tomcat1 for Webapp sample-webapp1 in namespace operator-sample-tomcat-issue
at io.javaoperatorsdk.operator.sample.WebappReconciler.lambda$reconcile$2(WebappReconciler.java:84) ~[classes/:?]
at java.util.Optional.orElseThrow(Optional.java:403) ~[?:?]
at io.javaoperatorsdk.operator.sample.WebappReconciler.reconcile(WebappReconciler.java:81) ~[classes/:?]
at io.javaoperatorsdk.operator.sample.WebappReconciler.reconcile(WebappReconciler.java:35) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:138) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:96) ~[classes/:?]
at io.javaoperatorsdk.operator.api.monitoring.Metrics.timeControllerExecution(Metrics.java:219) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller.reconcile(Controller.java:95) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.reconcileExecution(ReconciliationDispatcher.java:139) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleReconcile(ReconciliationDispatcher.java:119) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleDispatch(ReconciliationDispatcher.java:89) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:62) [classes/:?]
at io.javaoperatorsdk.operator.processing.event.EventProcessor$ReconcilerExecutor.run(EventProcessor.java:414) [classes/:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:831) [?:?]
13:41:11.869 ERROR io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:64) - Error during event processing ExecutionScope{ resource id: ResourceID{name='sample-webapp1', namespace='operator-sample-tomcat-issue'}, version: 498543} failed.
java.lang.IllegalStateException: Cannot find Tomcat test-tomcat1 for Webapp sample-webapp1 in namespace operator-sample-tomcat-issue
at io.javaoperatorsdk.operator.sample.WebappReconciler.lambda$reconcile$2(WebappReconciler.java:84) ~[classes/:?]
at java.util.Optional.orElseThrow(Optional.java:403) ~[?:?]
at io.javaoperatorsdk.operator.sample.WebappReconciler.reconcile(WebappReconciler.java:81) ~[classes/:?]
at io.javaoperatorsdk.operator.sample.WebappReconciler.reconcile(WebappReconciler.java:35) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:138) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller$1.execute(Controller.java:96) ~[classes/:?]
at io.javaoperatorsdk.operator.api.monitoring.Metrics.timeControllerExecution(Metrics.java:219) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.Controller.reconcile(Controller.java:95) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.reconcileExecution(ReconciliationDispatcher.java:139) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleReconcile(ReconciliationDispatcher.java:119) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleDispatch(ReconciliationDispatcher.java:89) ~[classes/:?]
at io.javaoperatorsdk.operator.processing.event.ReconciliationDispatcher.handleExecution(ReconciliationDispatcher.java:62) [classes/:?]
at io.javaoperatorsdk.operator.processing.event.EventProcessor$ReconcilerExecutor.run(EventProcessor.java:414) [classes/:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1135) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) [?:?]
at java.lang.Thread.run(Thread.java:831) [?:?]
13:41:11.871 ERROR io.javaoperatorsdk.operator.processing.event.EventProcessor.lambda$handleRetryOnException$6(EventProcessor.java:320) - Exhausted retries for ExecutionScope{ resource id: ResourceID{name='sample-webapp1', namespace='operator-sample-tomcat-issue'}, version: 498543}
The text was updated successfully, but these errors were encountered:
Bug Report
What did you do?
What did you expect to see?
Operator should first deploy two replicas of a tomcat instance and as soon as they become ready deploy the webapp.
What did you see instead? Under which circumstances?
Operator deploys two replicas of a tomcat instance, both became ready, then the WebappReconciler fails with the following exception:
This is due to the fact, that the
primaryToSecondaryIndex
inio.javaoperatorsdk.operator.processing.event.source.informer.InformerEventSource
is empty until the operator restarts.Environment
Kubernetes cluster type:
$ Mention java-operator-sdk version from pom.xml file
$ java -version
$ kubectl version --output=yaml
Possible Solution
Just a workaround: when restarting the operator, everything works fine as long as no new tomcat gets deployed.
For each new tomcat and webapp the same exception rises again until the operator gets restarted.
Additional context
Full log:
The text was updated successfully, but these errors were encountered: