Cluster Member can't see/use Grafana or monitoring stuff

I have a running Rancher (v2.3.2) setup with enabled cluster monitoring via prometheus/grafana.
As admin user I can see/use those cluster metrics in Rancher UI (v2.3.22) and in running Grafana instance.

Adding new users to the cluster - but even though they are now “Cluster Member” they can’t see/use monitoring stuff.

E.g. in cluster monitoring overview page in Rancher UI there are no Grafana icons and opening a URL that works for admin user fails for those users with

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {
    
  },
  "status": "Failure",
  "message": "services \"http:access-grafana:80\" is forbidden: User \"u-qddg6\" cannot get resource \"services/proxy\" in API group \"\" in the namespace \"cattle-prometheus\"",
  "reason": "Forbidden",
  "details": {
    "name": "http:access-grafana:80",
    "kind": "services"
  },
  "code": 403
}

What Role(s) does a user have to be to be able to use prometheus/grafana with metrics in Rancher ?

Hey Manuel,

Here is how I fixed it :

  • In your cluster, go to Global > Security > Roles
  • Cick the “Projects” Tab
  • Click the “Add Project Role” button
  • Name you new role like “Grafana Access”
  • Click the “Add Reource” button in “Grant Resources” Table
  • Check the “Get”, “List” checkboxes and input “services/proxy” in the Resource field
  • Click the “Save” button
  • Go to your cluster name “xxxxxx” > System project
  • Click the “Members” tab
  • Click “Add Member” button
  • Input the name of the user you want to grant access and check the “Grafana Access” Role ( or name you input during creation)
  • Click the “Create” button

Tadaaa

1 Like

Thank you for that hint - my teammates can now use Grafana !

Hi @quick691

To first thank you for your solution which was useful to me.

I’m working on Rancher rights, my goal is to create a limited project level profile that allows to see only workloads (see events, metrics) and config maps.

Since I’m on Rancher 2.6 for metrics there is now a dedicated role monitoring-ui-view (see Rancher Docs: Role-based Access Control) which works well.

With that my users have access to grafana but there are two problems:

1/ As I explain in my ticket Advanced View Workloads Role I would like my users to be able to directly see the workloads-metric tab in Rancher; Do you have an idea?

2/ The error message services "http:rancher-monitoring-grafana:80" is forbidden: User "u-fhws6" cannot create resource "services/proxy" in API group ... remains but only for a particular path /k8s/clusters/c-t4vjj/api/v1/namespaces/cattle-monitoring- system/services/http:rancher-monitoring-grafana:80/proxy/api/frontend-metrics. I guess I need to give cluster level rights on some resource other than workloads, I haven’t found what yet, do you think I’m on the right direction?

Thanks

Hello!

R 2.7.7.

Grafana access for new users without any admin rights.

Create the custom role:

1.1 Click ☰ > Users & Authentication > Roles.

1.2 Select the appropriate tab, e.g., Cluster role. Then click Create Cluster Role.

1.3 In the Name field, create a custom role such as View Monitoring, Edit Monitoring, or Admin Monitoring.

1.4 Click Inherit From > Add Resource, then select the Kubernetes role, as applicable, from the dropdown.

1.5 Click Create.

Assign the custom role to a new user:

2.1 Click ☰ > Cluster Management > Cluster Explore > Cluster > Cluster Members > Add.

2.2 Search for your new user name from Select Member options displayed.

2.3 Assign the new custom role from Cluster Permissions to the new user.

2.4 Click Create.

not worked, new user steel cant see monitoring tools.

kubectl create clusterrolebinding grafana-bind-cls-test --clusterrole=monitoring-ui-view --user='myuser'

Gives nothing too.

Can someone provide full list of actions to allow new users(not admins) see monitoring menu in left pane and have access to grafana.

1 Like

I’m in the same situation with Rancher 2.7.9
Does anyone provide a step by step for not admins users?
Thanks in advanced