Rancher Release - v2.5.0

Release v2.5.0

Due to a random corruption of the rancher-agent image on Dockerhub, only amd64 is available for 2.5.0. Rancher 2.5.1 is a mirror release of 2.5.0 but with rebuilt images, please use that version instead. #29424

Important

The primary UI in Rancher since version 2.0 is now called Cluster Manager. Our new Cluster Explorer dashboard, experimentally released in Rancher 2.4, has graduated to GA status. There are new features only available in the new Cluster Explorer dashboard. Some of these new features are similar functionality to existing features in the Cluster Manager and we will try to differentiate them based on where they are located in the UI.

Install/Upgrade Notes

  • Rancher install or upgrade must occur with Helm 3.2.x+ due to the changes with the latest cert-manager release. #29213
  • Rancher HA cluster should be upgraded to Kubernetes 1.17+ before installing Rancher 2.5.
  • If using a proxy in front of an air-gapped Rancher, you must pass additional parameters to NO_PROXY. #2725
  • The local cluster can no longer be turned off, which means all admins will have access to the local cluster. If you would like to restrict permissions to the local cluster, there is a new restricted-admin role that must be used.
  • If you are using Rancher to manage other Rancher instances, do not upgrade at this time as there are known issues around conflicting controllers. #29364.
  • If you are running a forked build of the UI and set the ui-index setting to local, you currently are unable to force that to load . #29362

Docker Install

  • When starting the Rancher Docker container, the privileged flag must be used. See the docs for more info
  • If you’re using custom certs with the Docker install, clusters cannot be provisioned. #28605
  • When installing in an air gap environment, you must supply a custom registries.yaml file to the Docker run command as shown in the k3s docs. If the registry has certs, then you will need to also supply those. #28969
  • There are UI issues around startup time #28800, #28798

Duplicated Features in Cluster Manager and Cluster Explorer

  • Only 1 version of the feature may be installed at any given time due to potentially conflicting CRDs.
  • Each feature should only be managed by the UI that it was deployed from.
  • If you have installed the feature in Cluster Manager, you must uninstall in Cluster Manager before attempting to install the new version in Cluster Explorer dashboard.

Kubernetes 1.19

  • For K8s 1.19 and newer, we recommend disabling firewalld as it has been found to be incompatible with various CNI plugins. #28840
  • Certain alerts in Cluster Manager are not working with k8s 1.19 as certain metrics have changed in Kubernetes 1.19 #29292

Deprecated Features

Feature Justification
Cluster Manager - Rancher Monitoring Monitoring in Cluster Manager UI has been replaced with a new monitoring chart available in the Apps & Marketplace in Cluster Explorer.
Cluster Manager - Rancher Alerts and Notifiers Alerting and notifiers functionality is now directly integrated with a new monitoring chart available in the Apps & Marketplace in Cluster Explorer.
Cluster Manager - Rancher Logging Functionality replaced with a new logging solution using a new logging chart available in the Apps & Marketplace in Cluster Explorer.
Cluster Manager - MultiCluster Apps Deploying to multiple clusters is now recommended to be handled with Rancher Continuous Delivery powered by Fleet available in Cluster Explorer.
Cluster Manager - Kubernetes CIS 1.4 Scanning Kubernetes CIS 1.5+ benchmark scanning is now replaced with a new scan tool deployed with a cis benchmarks chart available in the Apps & Marketplace in Cluster Explorer.
Cluster Manager - Rancher Pipelines Git-based deployment pipelines is now recommend to be handled with Rancher Continuous Delivery powered by Fleet available in Cluster Explorer.
Cluster Manager - Istio v1.5 The Istio project has ended support for Istio 1.5 and has recommended all users upgrade. Istio 1.7 is now available as an Istio chart in the Apps & Marketplace in Cluster Explorer.

Versions

The following versions are now latest and stable:

Type Rancher Version Docker Tag Helm Repo Helm Chart Version
Latest v2.5.0 rancher/rancher:latest server-charts/latest v2.5.0
Stable v2.4.8 rancher/rancher:stable server-charts/stable v2.4.8

Please review our version documentation for more details on versioning and tagging conventions.

Features and Enhancements

  • Cluster Explorer: A UI to provide a deeper look into clusters under management. With this new UI, you can:

    • Manage all Kubernetes cluster resources including custom resources from the Kubernetes operator ecosystem.
    • Deploy Helm charts from our new Apps and Marketplace.
    • View and edit Helm3 CLI releases from the new Apps UI.
    • Visibility to resources based on RBAC permissions
    • View logs and interact with kubectl shell in a new IDE-like viewer.
    • New Observability and Operations Tooling :
      • Rancher Server Backups: Previously admins relied on etcd backups of the Rancher management cluster for disaster recovery and rollback scenarios. The new backup and recovery process no longer requires access to the etcd database. Now admins can perform ad-hoc or scheduled backups of the Rancher application directly from the Rancher dashboard. Restore data into any Kubernetes cluster.

      • Monitoring and Alerting powered by Prometheus: Users can now define how to monitor, alert, and visualize application health as part of the deployment configuration. The new monitoring system can be configured with GitOps based workflows using the Prometheus Operator custom resources. Alertmanager can be configured to use all built-in notification integrations. Alert templates can be customized to tailor notifications the on-call team receives. Users can deploy their own Grafana dashboards into the built-in Grafana instance.

      • Logging powered by Banzai Cloud: New cluster-level logging pipelines that incorporate lighter weight FluentBit and Fluentd to ship logs to a remote data store. The pipeline functionality is orchestrated Kubernetes objects enabling configuration through GitOps based workflows. Users will have the ability to customize both the FluentBit and Fluentd configurations.

      • Expanded CIS Scans powered by kube-bench: Rancher CIS scanning now runs on EKS and GKE platforms in addition to RKE clusters. The scanning tool also includes a new standard CIS-1.5 profile untailored for a specific K8s distribution. The benchmark versions have been updated to CIS 1.5, EKS-1.0, and GKE-1.0.

  • Rancher Continuous Delivery powered by Fleet: Rancher Continuous Delivery is a built-in deployment tool powered by Rancher’s Fleet project. Users can leverage this tool to deliver applications and configurations from a Git source repository across multiple clusters. Rancher Continuous Delivery is able to scale to a large number of clusters under management using a staged checkout and pull-based update model. The staged checkout allows the Continuous Delivery control plane to gradually roll deployments out to clusters instead of all at once. Using a pull-based update model, administrators don’t need to configure network access to every remote cluster. Administrators can organize clusters into groups for easier management within Rancher Continuous Delivery. Git source repositories are mapped to cluster group targets by admins, without needing the end-user to access the control plane configuration. Application owners and admins can deploy any Kubernetes resource defined by manifests, kustomize, or Helm.

  • Enhanced EKS Lifecycle Management: EKS provisioning has been enhanced to support managed node groups, private access, and control plane logging. Users can now register existing EKS clusters provisioned with other tools like eksctl into Rancher to allow the management of upgrades and configuration going forward. Support has been added for multiple managed node groups with heterogeneous configurations to enable GPU and non-GPU workloads in the same cluster. The EKS cluster configuration in Rancher now uses cloud credentials decoupling the AWS key management from the cluster configuration.

  • Istio 1.7: The Istio project has made a lot of changes to the installation process over the last several releases. The new Rancher Istio integration now deploys the latest Istio Operator allowing for users to deploy multiple ingress and egress gateways. Users can also manage the Istio custom resources in a customized UX in the new dashboard.

  • RKE Government: A new Kubernetes distribution that supports FIPS encryption, SELinux, and container-d. The RKE-Gov distribution has out-of-the-box CIS compliance. Management of etcd backups and restores are built into the distribution.

  • CentOS/RHEL 8 Support: Admins can use RKE Government clusters installed on RHEL and CentOS 8 systems. RKE support will come in a later update and use Docker CE.

Experimental Features

  • OPA Gatekeeper: Users can deploy and manage the updated GA version of OPA Gatekeeper through Rancher. Users must uninstall the first Rancher installed version OPA Gatekeeper before installing this new feature.

  • RancherD: A single binary installation of Rancher. Admins create 1 or 3 hosts, and start the RancherD binary to perform all the work of installing Rancher. Check out this blog article for more details.

Major Bugs Fixed Since v2.4.8

  • Rancher no longer panic’s during a drain action from the API #28905
  • Argo rollout pods can now be seen in the cluster explorer #27923
  • Helm’s max history can now be configured #28728
  • The Cluster Manager UI dropdown now works on mobile #23298
  • The new EKS provisioning supports API Server Endpoint Access Control #19051
  • Cluster Explorer supports CRUD action on CRDs #18013

Other notes

Known Major Issues

  • Cluster Manager’s Monitoring stack does not install on the local cluster if it is K3s #29328

Cluster Explorer Feature Caveats and Upgrades

  • General
    • Not all new features are currently installable on a hardened cluster.
    • New features are expected to be deployed using the Helm3 CLI and not with the Rancher CLI
    • The new Logging and Monitoring features do not yet work with windows clusters. #28721 #28327
  • Rancher Backup
    • When migrating to a cluster with the Rancher Backup feature, the server-url cannot be changed to a different location, it must continue to use the same URL.
    • Rancher Continuous Delivery (Fleet) is not handled during backup. Backup#46
  • Monitoring
    • The Monitoring stack currently is not installable on a K8s 1.16 cluster #29395
    • When the new Monitoring feature is deployed on K3s, it needs to have specific memory/cpu limits set #28787
    • Monitoring sometimes errors on installation because it can’t identify CRDs #29171
      • The new monitoring chart deploys a standard Prometheus installation which does not support project-level isolation.
  • Istio
    • Installation for Istio fails when the ingress gateway is disabled #29383
  • Longhorn
    • Longhorn has seen an issue where uninstallation may get stalled if there are daemonset changes occurring. Longhorn#1820
  • OPA Gatekeeper (Experimental)
    • The first edition of OPA must be uninstalled before the new OPA features are installed #29188

Air gap

  • Deploying an EKS cluster in an air gap environment is not supported #29070
  • Air gap has rarely seen an issue where the downstream cluster does not correctly get the registry setting, causing pods to fail. To workaround the issue, simply restart the agent #28923

Versions

Images

  • rancher/rancher:v2.5.0
  • rancher/rancher-agent:v2.5.0

Tools

Kubernetes

Upgrades and Rollbacks

Rancher supports both upgrade and rollback. Please note the version you would like to upgrade or rollback to change the Rancher version. There are different rollback instructions for Rancher versions 2.5.0 or newer and for versions 2.4.x or earlier.

Important: When rolling back, we are expecting you to rollback to the state at the time of your upgrade. Any changes post upgrade would not be reflected.