Cluster Member can't see/use Grafana or monitoring stuff

I have a running Rancher (v2.3.2) setup with enabled cluster monitoring via prometheus/grafana.
As admin user I can see/use those cluster metrics in Rancher UI (v2.3.22) and in running Grafana instance.

Adding new users to the cluster - but even though they are now “Cluster Member” they can’t see/use monitoring stuff.

E.g. in cluster monitoring overview page in Rancher UI there are no Grafana icons and opening a URL that works for admin user fails for those users with

  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
  "status": "Failure",
  "message": "services \"http:access-grafana:80\" is forbidden: User \"u-qddg6\" cannot get resource \"services/proxy\" in API group \"\" in the namespace \"cattle-prometheus\"",
  "reason": "Forbidden",
  "details": {
    "name": "http:access-grafana:80",
    "kind": "services"
  "code": 403

What Role(s) does a user have to be to be able to use prometheus/grafana with metrics in Rancher ?

Hey Manuel,

Here is how I fixed it :

  • In your cluster, go to Global > Security > Roles
  • Cick the “Projects” Tab
  • Click the “Add Project Role” button
  • Name you new role like “Grafana Access”
  • Click the “Add Reource” button in “Grant Resources” Table
  • Check the “Get”, “List” checkboxes and input “services/proxy” in the Resource field
  • Click the “Save” button
  • Go to your cluster name “xxxxxx” > System project
  • Click the “Members” tab
  • Click “Add Member” button
  • Input the name of the user you want to grant access and check the “Grafana Access” Role ( or name you input during creation)
  • Click the “Create” button



Thank you for that hint - my teammates can now use Grafana !