Rancher Release - v1.6.28 - Addresses Rancher CVE-2019-12274

Announcements
1

Release v1.6.28

Important

This release addresses a security vulnerability found in Rancher versions v1.6.27 and earlier (Cattle and Kubernetes users). It affects the built-in node drivers having a file path option that allows the machine to read arbitrary files from inside the Rancher server container. This can result in the machine creator gaining unauthorized access to the Rancher server. You can view the official CVE here CVE-2019-12274.

Versions

Supported Docker Versions

  • Docker 1.12.3-1.12.6
  • Docker 1.13.1
  • Docker 17.03-ce/ee
  • Docker 17.06-ce/ee
  • Docker 17.09-ce/ee
  • Docker 17.12-ce/ee
  • Docker 18.03-ce/ee
  • Docker 18.06-ce/ee
  • Docker 18.09-ce/ee

Note: Kubernetes 1.11/1.10/1.9 supports Docker 1.12.6, 1.13.1 and 17.03.2.

Kubernetes Versions

List of images required to launch Kubernetes template:

k8s 1.11.9

  • rancher/k8s:v1.11.9-rancher1-1
  • rancher/etcd:v2.3.7-17
  • rancher/kubectld:v0.8.8.1
  • rancher/etc-host-updater:v0.0.3
  • rancher/kubernetes-agent:v0.6.9
  • rancher/kubernetes-auth:v0.0.8
  • rancher/lb-service-rancher:v0.9.9
  • busybox

k8s 1.12.7

  • rancher/k8s:v1.12.7-rancher1-1-1
  • rancher/etcd:v2.3.7-17
  • rancher/kubectld:v0.8.10
  • rancher/etc-host-updater:v0.0.3
  • rancher/kubernetes-agent:v0.6.9
  • rancher/kubernetes-auth:v0.0.8
  • rancher/lb-service-rancher:v0.9.11
  • busybox

For the list of versions for the Kubernetes add-ons embedded in the Rancher Kubernetes images, please refer to the kubernetes-package repo for the specific images and versions.

Note: If you have configured the aws cloud provider, tagging the cluster resources with a ClusterID is now required as of Kubernetes 1.10+. You should add tags to your EC2 instances before launching Kubernetes.

Rancher Server Tags

Rancher server has 2 different tags. For each major release tag, we will provide documentation for the specific version.

  • rancher/server:latest tag will be our latest development builds. These builds will have been validated through our CI automation framework. These releases are not meant for deployment in production.
  • rancher/server:stable tag will be our latest stable release builds. This tag is the version that we recommend for production.

Please do not use releases with a rc{n} suffix. These rc builds are meant for the Rancher team to test builds.

Beta - v1.6.28 - rancher/server:latest

Stable - v1.6.28 - rancher/server:stable

Enhancements

Infrastructure Service Updates

When upgrading infrastructure services, please make sure to upgrade in the recommended order.

  • Kubernetes v1.11.9 - v1.11.9-rancher1-1
  • New images: rancher/k8s:v1.11.9-rancher1-1, rancher/etcd:v2.3.7-17, rancher/kubectld:v0.8.8.1, rancher/etc-host-updater:v0.0.3 , rancher/kubernetes-agent:v0.6.9, rancher/kubernetes-auth:v0.0.8, rancher/lb-service-rancher:v0.9.9
  • Kubernetes v1.12.7 - v1.12.7-rancher1-1-1
  • New images: rancher/k8s:v1.12.7-rancher1-1-1, rancher/etcd:v2.3.7-17, rancher/kubectld:v0.8.10, rancher/etc-host-updater:v0.0.3 , rancher/kubernetes-agent:v0.6.9, rancher/kubernetes-auth:v0.0.8, rancher/lb-service-rancher:v0.9.11
  • Scheduler - v0.8.6
  • New images: rancher/scheduler:v0.8.6
    - Addressed panic happening when metadata is not reachable [#10252]
  • Network Manager - v0.2.11
  • New images: rancher/network-manager:v0.7.22
    - Added configuration for ARP sync interval. [#15729]

Known Major Issues

  • Kubernetes 1.11.9 fails to start on setups having docker version 18.09.x [#17780].

Major Bug Fixes since v1.6.28

  • Fixed vulnerability affecting the built-in node drivers having a file path option that allowed the machine to read arbitrary files from inside the Rancher server container CVE-2019-12274

Rancher CLI Downloads

Important - More Security

  • The zulu-openjdk package has now been updated to support JDK v8u172.
  • The etcd nodes that are installed when creating a kubernetes cluster are now defaulted to having etcd cluster TLS enabled.
  • Jetty Server package got upgraded to to 9.2.26.v20180806 - a release having several security fixes
  • Default Kubernetes version got updated to v1.12.5-rancher1-1 - a release having a fix for CVE-2018-1002105 vulnerability

Important - Upgrade

  • Users on a version prior to Rancher v1.5.0: We will automatically upgrade the network-services infrastructure stack as without this upgrade, your release will not work.

  • Users on a version prior to Rancher v1.6.0: If you make any changes to the default Rancher library setting for your catalogs and then roll back, you will need to reset the branch used for the default Rancher library under Admin → Settings → Catalog. The current default branch is v1.6-release, but the old default branch is master.

  • Rollback Versions: We support rolling back to Rancher v1.6.18 from Rancher v1.6.22.

    • If you have upgraded to Kubernetes v1.10.5-rancher4 or v1.11.1-rancher1-3 or any version after these, there are some manual steps that need to occur to roll back to any version prior to these ones due to the updates for having etcd cluster with TLS enabled.

      1. Stop the kube-api service
      2. Exec into one of the etcd containers
      3. Run the following command. The output is in the following format: etcd_memeber_id: member_name peerURLs clientURLs.
        $ etcdctl member list
      4. You will need to run the following commands to update the cluster peer URLs from HTTPS to HTTP, and from names to IPs
      • replace - with . in the member_name to get the IP
      • run the member update command in the following format:
        etcdctl member update <etcd_member_id> <list of comma separated peerURL>
      1. After etcd has been updated, proceed with typical rollback steps.

    • Steps to Rollback:

      1. In the upgraded version the Admin → Advanced Settings → API values, update the upgrade.manager value to all.
      2. “Upgrade” Rancher server but pointing to the older version of Rancher (v1.6.18). This should include backing up your database and launching Rancher to point to your current database.
      3. Once Rancher starts up again, all infrastructure stacks will automatically rollback to the applicable version in v1.6.18.
      4. After your setup is back to its original state, update the upgrade.manager value back to the original value that you had (either mandatory or none).

Note on Rollback: If you are rolling back and have authentication enabled using Active Directory, any new users/groups added to site access on the Access Control page after the upgrade will not be retained upon rolling back. Any users added before the upgrade will continue to remain. [#9850]

Important - Please read if you currently have authentication enabled using Active Directory with TLS enabled prior to upgrading to v1.6.10.

Starting with v1.6.8, Rancher has updated the Active Directory auth plugin and moved it into the new authentication framework. We have also further secured the AD+TLS option by ensuring that the hostname/IP of the AD server matches with the hostname/IP of the TLS certificate. Please see [#9459] for details.

Due to this new check, you should be aware that if the hostname/IP does not match your TLS certificate, you will be locked out of your Rancher server if you do not correct this prior to upgrading. To ensure you have no issues with the upgrade, please execute the following to verify your configuration is correct.

  • Verify the hostname/IP you used for your AD configuration. To do this, log into Rancher using a web browser as an admin and click Admin → Access Control. Note the server field to determine your configured hostname/IP for your AD server.
  • To verify your the configure hostname/IP for your TLS cert, you can execute the following command to determine the CN attribute:
    openssl s_client -showcerts -connect domain.example.com:443
    You should see something like:
    subject=/OU=Domain Control Validated/CN=domain.example.com
    Verify that the CN attribute matches with your configured server field from the above step.

If the fields match, you are good to go. Nothing else is required.

If the fields do not match, please execute the following steps to correct it.

  • Open a web browser and go to Rancher’s settings URL. This can be done by logging into Rancher as an admin and click API → Keys. You should see an Endpoint (v2-beta) field. Take the value of that field and append /settings. The final URL should look something like my.rancher.url:8080/v2-beta/settings. Launch this URL in your browser and you should see Rancher’s API browser.
  • Search for api.auth.ldap.server and click that setting to edit it. On the top right, you should be able to click an edit button. Change the value of that to match the hostname/IP of the value found in your cert as identified by the CN attribute and click Show Request → Send Request to persist the value into Rancher’s DB. The response should show your new value.

Once this is completed and the hostname/IP matches your certs’ CN attribute, you should have no issues with AD login after upgrading to 1.6.8.