Rancher-token not found after upgrade

Rancher 2.x
1

I’m trying to upgrade an existing Rancher v2.5.11 deployment to v2.6.3 however the new rancher pod which starts up just goes into a crash loop

If I check the logs this is what it shows

2021/12/22 04:52:04 [INFO] Rancher version v2.6.3 (3c1d5fac3) is starting
2021/12/22 04:52:04 [INFO] Rancher arguments {ACMEDomains:[] AddLocal:true Embedded:false BindHost: HTTPListenPort:80 HTTPSListenPort:443 K8sMode:auto Debug:false Trace:false NoCACerts:true AuditLogPath:/var/log/auditlog/rancher-api-audit.log AuditLogMaxage:10 AuditLogMaxsize:100 AuditLogMaxbackup:10 AuditLevel:0 Features: ClusterRegistry:}
2021/12/22 04:52:04 [INFO] Listening on /tmp/log.sock
2021/12/22 04:52:04 [FATAL] secrets "rancher-token-gpggv" not found

There is no secret with the name “rancher-token-gpggv” but there is one with the name “rancher-token-5clp5”, I can’t find any reference to a rancher-token secret in the deployment config so I don’t know where it’s getting that name from

Any ideas?

Thanks

2

I got the same error after I upgraded to v2.6.3 coming from v2.6.1.
I used the Helm to upgrade. I also had to rotate webhook certificates before I could get the upgrade to work as instructed here: Rancher Docs: Rotation of Expired Webhook Certificates
Not sure if there is anything we can do right now except restore from backups.