Rancher Upgrade to 2.6.4 breaks cluster creation and upgrades

Hello, We are performing Kubernetes LCM work. we are upgrading rancher from v2.6.1 to v2.6.4. RKE is on v1.21.14. After upgrading rancher to v2.6.4, I am not able to upgrade my downstream rke cluster from v1.21.14 to v1.22.17. I get panic api error in rancher pod logs. Any help would be appreciated

2023/02/07 21:44:52 [ERROR] Panic serving api request:
Tue, Feb 7 2023 3:44:52 pm goroutine 66446 [running]:
Tue, Feb 7 2023 3:44:52 pm runtime/debug.Stack()
Tue, Feb 7 2023 3:44:52 pm /usr/lib64/go/1.17/src/runtime/debug/stack.go:24 +0x65
Tue, Feb 7 2023 3:44:52 pm github.com/rancher/norman/api.(*Server).ServeHTTP.func1()
Tue, Feb 7 2023 3:44:52 pm /go/src/github.com/rancher/rancher/go/pkg/mod/github.com/rancher/norman@v0.0.0-20220107203912-4feb41eafabd/api/server.go:175 +0x75
Tue, Feb 7 2023 3:44:52 pm panic({0x384e540, 0xc013670bd0})
Tue, Feb 7 2023 3:44:52 pm /usr/lib64/go/1.17/src/runtime/panic.go:1038 +0x215
Tue, Feb 7 2023 3:44:52 pm

@superseb I would appreciate any insights on this

@sam.patel Is this continuous or after some action was performed? Can you share more of the log?

Hey @superseb!! Sorry for the delayed response. I ended resolving it. The RKE Template did not include the Private Registry Password

1 Like

Good, this would still be valid for a bugfix but I need the full log and/or steps to reproduce. I can check some things based on what you shared, thanks.