Cluster Member can't see/use Grafana or monitoring stuff

Rancher 2.x
#1

I have a running Rancher (v2.3.2) setup with enabled cluster monitoring via prometheus/grafana.
As admin user I can see/use those cluster metrics in Rancher UI (v2.3.22) and in running Grafana instance.

Adding new users to the cluster - but even though they are now “Cluster Member” they can’t see/use monitoring stuff.

E.g. in cluster monitoring overview page in Rancher UI there are no Grafana icons and opening a URL that works for admin user fails for those users with

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
    
  },
  "status": "Failure",
  "message": "services \"http:access-grafana:80\" is forbidden: User \"u-qddg6\" cannot get resource \"services/proxy\" in API group \"\" in the namespace \"cattle-prometheus\"",
  "reason": "Forbidden",
  "details": {
    "name": "http:access-grafana:80",
    "kind": "services"
  },
  "code": 403
}

What Role(s) does a user have to be to be able to use prometheus/grafana with metrics in Rancher ?

Rancher 25 Prometheus for cluster/project owners
Rancher 25 Prometheus for cluster/project owners
#2

Hey Manuel,

Here is how I fixed it :

  • In your cluster, go to Global > Security > Roles
  • Cick the “Projects” Tab
  • Click the “Add Project Role” button
  • Name you new role like “Grafana Access”
  • Click the “Add Reource” button in “Grant Resources” Table
  • Check the “Get”, “List” checkboxes and input “services/proxy” in the Resource field
  • Click the “Save” button
  • Go to your cluster name “xxxxxx” > System project
  • Click the “Members” tab
  • Click “Add Member” button
  • Input the name of the user you want to grant access and check the “Grafana Access” Role ( or name you input during creation)
  • Click the “Create” button

Tadaaa

3 Likes
#3

Thank you for that hint - my teammates can now use Grafana !

#4

Hi @quick691

To first thank you for your solution which was useful to me.

I’m working on Rancher rights, my goal is to create a limited project level profile that allows to see only workloads (see events, metrics) and config maps.

Since I’m on Rancher 2.6 for metrics there is now a dedicated role monitoring-ui-view (see Rancher Docs: Role-based Access Control) which works well.

With that my users have access to grafana but there are two problems:

1/ As I explain in my ticket Advanced View Workloads Role I would like my users to be able to directly see the workloads-metric tab in Rancher; Do you have an idea?

deployment-metrics
deployment-metrics567×652 30.5 KB

2/ The error message services "http:rancher-monitoring-grafana:80" is forbidden: User "u-fhws6" cannot create resource "services/proxy" in API group ... remains but only for a particular path /k8s/clusters/c-t4vjj/api/v1/namespaces/cattle-monitoring- system/services/http:rancher-monitoring-grafana:80/proxy/api/frontend-metrics. I guess I need to give cluster level rights on some resource other than workloads, I haven’t found what yet, do you think I’m on the right direction?

Thanks