just putting this out there in case it might help someone in the future…
So after a successful upgrade to SES5 we saw false/unexpected results in “Ceph - Cluster” dashboard in Grafana.
Health Status showed “Warn” and sometimes a handful OSDs showed as down but running cluster checks in the terminal everything was up and healthy, no problems…
Did some troubleshooting and eventually landed at Prometheus (admin node on port 9090), there it clearly stated that there was time difference between my browser and the server, very true, clock on my machine was 2 hours off for some reason.
So I did a timesync and reloaded openattic and now everything is SUSE Green, no errors.