You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Describe the bug a clear and concise description of what the bug is.
We're currently in the process of upgrading the Prometheus-Stack to the latest version, but in our test environment, we noticed a few issues with the API-Server Monitoring.
In the Dashboard "Kubernetes / API server", one can see that several Panels show no data. While the "Write Availability (30d)" works fine, the following panels show "No Data":
Availability (30d) > 99.000%
ErrorBudget (30d) > 99.000%
Read Availability (30d)
In my tests, I noticed that for example the Read Availability works if I add the or vector(0) fallback to the sums: Example Grafana Explore View (you need to have a Port-forward to Grafana on your local Port 3000, i.e. via kubectl port-forward svc/prometheus-grafana 3000:80)
Client Version: v1.32.2 Kustomize Version: v5.5.0 Server Version: v1.32.2
Which chart?
kube-prometheus-stack
What's the chart version?
69.7.3
What happened?
After upgrading to the latest version of kube-prometheus-stack (69.7.3), the API-Server Availability Dashboard is broken. Several Panels show "No Data" when they should show an availability of up to 100%.
What you expected to happen?
The Panels should show an availability of 100% or less.
Log in using default credentials (admin - prom-operator)
Open the Dashboard "Kubernetes / API server"
Wait a few minutes for Prometheus to gather data and calculate the rules
Enter the changed values of values.yaml?
NONE
Enter the command that you execute and failing/misfunctioning.
The broken Panels use the Prometheus-Recording-Rules for apiserver_request:availability30d with the verbs read and all.
Anything else we need to know?
Background: After upgrading from kube-prometheus-stack from 62.6.0 to 69.7.2, we've noticed alerts about alertname: KubeAPIErrorBudgetBurn. Checking out the API-Server Dashboard revealed that the availability Panels showed more than 100% (i.e. 5000% and growing).
As the Upgrade to 68 has updated the behaviour of different API-Server metrics, we've used the Admin-API to delete older Metrics. But then some panels could not be shown anymore.
The same issue can be reproduced in a freshly installed Installation of the Kube-Prometheus-Stack.
Edit: Improve comprehensibility of Link "Example Grafana Explore View"
The text was updated successfully, but these errors were encountered:
Uh oh!
There was an error while loading. Please reload this page.
Describe the bug a clear and concise description of what the bug is.
We're currently in the process of upgrading the Prometheus-Stack to the latest version, but in our test environment, we noticed a few issues with the API-Server Monitoring.
In the Dashboard "Kubernetes / API server", one can see that several Panels show no data. While the "Write Availability (30d)" works fine, the following panels show "No Data":
In my tests, I noticed that for example the Read Availability works if I add the
or vector(0)
fallback to the sums:Example Grafana Explore View (you need to have a Port-forward to Grafana on your local Port 3000, i.e. via
kubectl port-forward svc/prometheus-grafana 3000:80
)What's your helm version?
version.BuildInfo{Version:"v3.17.1", GitCommit:"980d8ac1939e39138101364400756af2bdee1da5", GitTreeState:"clean", GoVersion:"go1.23.6"}
What's your kubectl version?
Client Version: v1.32.2 Kustomize Version: v5.5.0 Server Version: v1.32.2
Which chart?
kube-prometheus-stack
What's the chart version?
69.7.3
What happened?
After upgrading to the latest version of
kube-prometheus-stack
(69.7.3), the API-Server Availability Dashboard is broken. Several Panels show "No Data" when they should show an availability of up to 100%.What you expected to happen?
The Panels should show an availability of 100% or less.
How to reproduce it?
kube-prometheus-stack
via Helmkubectl port-forward svc/prometheus-grafana 3000:80
Enter the changed values of values.yaml?
NONE
Enter the command that you execute and failing/misfunctioning.
The broken Panels use the Prometheus-Recording-Rules for
apiserver_request:availability30d
with the verbsread
andall
.Anything else we need to know?
Background: After upgrading from
kube-prometheus-stack
from62.6.0
to69.7.2
, we've noticed alerts aboutalertname: KubeAPIErrorBudgetBurn
. Checking out the API-Server Dashboard revealed that the availability Panels showed more than 100% (i.e. 5000% and growing).As the Upgrade to 68 has updated the behaviour of different API-Server metrics, we've used the Admin-API to delete older Metrics. But then some panels could not be shown anymore.
The same issue can be reproduced in a freshly installed Installation of the Kube-Prometheus-Stack.
Edit: Improve comprehensibility of Link "Example Grafana Explore View"
The text was updated successfully, but these errors were encountered: