Rancher Release v2.3.4

Release v2.3.4

Important

  • Please review the v2.3.0 release notes for important updates/ breaking changes if you are upgrading from a v2.2 release.

  • Due to support for cert-manager v0.12.0, a new Helm chart has been released for the 2.2.x series to install Rancher. The underlying Rancher server container has only been updated to support the updated cert-manager changes. If you are using older cert-manager versions, please see the documentation on how to upgrade cert-manager.

The following versions are now latest and stable:

Type Rancher Version Docker Tag Helm Repo Helm Chart Version
Latest v2.3.4 rancher/rancher:latest server-charts/latest v2.3.4
Stable v2.3.4 rancher/rancher:stable server-charts/stable v2.3.4

Please review our version documentation for more details on versioning and tagging conventions.

Features and Enhancements

  • Kubernetes 1.17 is now the default version [#23088] [#22655] - Kubernetes 1.17 is now the default version. Whenever upgrading to any Kubernetes version, please review the Kubernetes release notes for any breaking changes. With Kubernetes 1.17, support for etcd 3.4.x and calico 3.10.x is added.

  • Other Updates

    • Added EKS support for Canada Central [#24299] and Sao Paulo [#24400]
    • Updated Istio to 1.4.3

Experimental Features

We have introduced the ability to turn on and off experimental components inside Rancher. As of this release, you can manage feature flags through our UI. Alternatively, you can refer to our docs on how to turn on the features when starting Rancher.

Major Bugs Fixed Since v2.3.3

  • Fixed many issues around the vsphere node driver [#24304, #24314, #24315, #24416, #24453, #24480, #24620]
  • Fixed an issue where air-gapped cluster provisioning fails when the private registry set on the cluster requires authentication [#20029]
  • Fixed an issue when during rollback, the versions of the agent were not rolling back to the previous version [#21386]
  • Fixed an issue where rotating certs could cause Rancher to crash [#24097]
  • Fixed an issue where kontainer-driver-metadata changes are not updated for rkeaddons [#24675]
  • Fixed an issue where provisioning GKE clusters weren’t working behind a proxy [#22006]
  • Fixed an issue where certain setups requiring different MTU settings were unable to reliably set it [#24096]
  • Fixed an issue where fluentd json log key parsing was removed when upgrading fluentd versions [#23646]
  • Fixed an issue where activating cluster level logging sent lots of backslashes onto the fluentd target [#24545]
  • Fixed an issue where updating a role that is already used to inherit roles were not being reflected in the existing rolebindings [#23658]
  • Fixed an issue where labels couldn’t be added to imported clusters [#24123]
  • Fixed an issue where nodes will not delete if the AWS key has been deleted/disabled even after updating nodetemplate [#24234]
  • Fixed an issue where sometimes nested groups would get lost with Google OAuth [#24334]
  • Fixed an issue where Rancher HA upgrade failed when anti-affinity is set to required [#24518]
  • Fixed an issue where FreeIPA group memberships were not showing up in cluster membership auto complete [#18927]
    -Fixed an issue where node annotations removed through UI/API cannot be re-added [#24869]
  • Fixed an issue where charts with long names caused the catalog refresh to time out [#19724]
  • Fixed an issue where using numbers with integers in a yaml larger than 6 digits would be converted to scientific notation when deploying an app using Rancher CLI [#20704]
  • Fixed an issue where adding catalogs were not working and users would hit a certificate error [#23231, #24347]
  • Fixed an issue where a bad chart in a repo causes constant refreshes [#23368]
  • Fixed an issue where an email notifier test was from the Rancher server cluster instead of the user cluster [#22578]

Other notes

Air Gap Installations and Upgrades

In v2.3.0, an air gap install no longer requires mirroring the systems chart git repo. Please follow the directions on how to install Rancher to use the packaged systems chart.

Known Major Issues

  • RHEL 7.7 with selinux enabled and k8s 1.16 with RHEL Docker 1.13 is not working [#23662]
  • NGINX ingress controller 0.25.0 doesn’t work on CPUs without SSE4.2 instruction set support [#23307]
  • Windows Limitations - There are a couple of known limitations with Windows due to upstream issues:
    • Windows pods cannot access the Kubernetes API when using VXLAN (Overlay) backend for the flannel network provider. The workaround is to use the Host Gateway (L2bridge) backend for the flannel network provider. [#20968]
    • Logging only works on Host Gateway (L2bridge) backend for the flannel network provider [#20510]
  • Istio Limitation - Istio will not work with a restricted pod security policy [#22469]
  • HPA Limitation - HPA UI doesn’t work on GKE clusters as GKE doesn’t support the v2beta2.autoscaling API [#22292]
  • Hardening Guide Limitations - If you have used Rancher’s hardening guide, there are some known issues
    • kubectl in UI doesn’t work [#19439]
    • Pipelines don’t work [#22844]
  • Adding taints to existing node templates from an upgraded setup will not be applied unless a reconcile is triggered on the cluster. When scaling up/down worker nodes, no reconcile is triggered, but scaling up/down either control plane/etcd nodes or editing a cluster (like upgrading to the latest Kubernetes version) would update to support taints on the nodes. [#22672]
  • Cluster alerting and logging can get stuck in Updating state after upgrading Rancher. Workaround steps are provided in the issue [21480]
  • If you have Rancher cluster with OpenStack cloud provider having LoadBalancer set, and the cluster was provisioned on version 2.2.3 or less, the upgrade to the Rancher version v2.2.4 and up will fail. Steps to mitigate can be found in the comment to [20699]
  • In clusters that have a Kubernetes cloud provider configured and have agents registered with hostname or FQDN (so not valid IP addresses), kube-proxy will fail to start. This can be checked in the API output for the node (customConfig -> address or internalAddress) [RKE#1725]
  • Rancher log collection format changed when upgrading the Fluentd Kubernetes metadata plugin. A json log is no longer parsed and put into the log as top level keys. Issue to optionally bring back this behavior[23646]

Versions

Images

  • rancher/rancher:v2.3.4
  • rancher/rancher-agent:v2.3.4

Tools

Kubernetes

Upgrades and Rollbacks

Rancher supports both upgrade and rollback. Please note the version you would like to upgrade or rollback to change the Rancher version.

Please be aware that upon an upgrade to v2.3.0+, any edits to a Rancher launched Kubernetes cluster will cause all system components to restart due to added tolerations to Kubernetes system components. Plan accordingly.

Recent changes to cert-manager require an upgrade if you have an HA install of Rancher using self-signed certificates. If you are using cert-manager older than v0.9.1, please see the documentation on how to upgrade cert-manager.

Important: When rolling back, we are expecting you to rollback to the state at the time of your upgrade. Any changes post upgrade would not be reflected.

1 Like
  • Air Gap Users - Istio 1.4.3 is currently broken for an air gap users and will be addressed in the next release.