"Failed to list" errors logged using default ClusterRole "prometheus-k8s" #483
Comments
|
The reason for this, it seems, is that those resources and verbs are granted on a per-namespace basis, not as part of the ClusterRole. You can therefore update the array of namespaces which these permissions should be extended to, or amend the ClusterRole to add in those permissions. I'm going to do the latter, and I'll add a comment to this to demonstrate how I did it |
|
I updated the local k = import 'ksonnet/ksonnet.beta.3/k.libsonnet';
local clusterRole = k.rbac.v1.clusterRole;
local policyRule = clusterRole.rulesType;
local extra_cluster_role_resources = policyRule.new() +
policyRule.withApiGroups(['']) +
policyRule.withResources(['services','pods','endpoints']) +
policyRule.withVerbs(['get','list','watch']);
local kp =
(import 'kube-prometheus/kube-prometheus.libsonnet') +
{
prometheus+:: {
clusterRole+: {
rules+: [extra_cluster_role_resources],
},
},
};These changes resulted in my ClusterRole looking like this: apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: prometheus-k8s
rules:
- apiGroups:
- ""
resources:
- nodes/metrics
verbs:
- get
- nonResourceURLs:
- /metrics
verbs:
- get
- apiGroups:
- ""
resources:
- services
- pods
- endpoints
verbs:
- get
- list
- watch |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Prometheus is spamming our logs with messages about it not having permissions to
listthe resourcesservices,endpointsandpodsin our cluster. The error messages look like this:Amending the included
ClusterRoleto include the resources and thelistverb stops the errors. However, I notice that in all of the releases in this repo, none of them have these permissions included on the ClusterRole:Looking at the Prometheus docs, their ClusterRole does include these permissions in their docs.
What am I missing? I notice that in this issue the same errors are reported, but the answer is that they should be using the release-0.1 branch.
We're running Kubernetes 1.16.5 with the
release-0.3branch of kube-prometheus.Thanks in advance!
The text was updated successfully, but these errors were encountered: