Certificate Issue: Same serial number as another certificate

1

Hello,

I’m using Rancher v2.1.6 for 2 different HA setups. Both are using the NGINX load balancer. Both have a DNS name in our system in the same domain. They are in different subnets, vlans, hosts and VM’s being setup. The first one I was able to access fine until today. All of a sudden I get this error:

You have received an invalid certificate. Please contact the server administrator or email correspondent and give them the following information: Your certificate contains the same serial number as another certificate issued by the certificate authority. Please get a new certificate containing a unique serial number. Error code: SEC_ERROR_REUSED_ISSUER_AND_SERIAL

The page you are trying to view cannot be shown because the authenticity of the received data could not be verified.
Please contact the website owners to inform them of this problem.

I’ve tried deleting files from Firefox for this, but it doesn’t make a difference. Tried this on another machine as well that had never accessed the Rancher sites I created and I still get the same issue. How do I fix this? I plan on building this several times, sometimes with rke, sometimes without, but I may wind up installing Rancher afterwards, so I don’t want these kinds of problems to persist if we keep building more and more Rancher clusters. I’m not sure if there’s some particular cert thing I’m supposed to do, but I basically went thru the HA steps on the Rancher page to make this work. Any help is appreciated.

2

I can get this to work in IE (Well not really it doesn’t load the site), Edge and Chrome, but not Firefox. In IE I just added the certs for the site and for cattle-ca into the Trusted Root and it’s working for Edge and Chrome, but still failing in Firefox.

3

I’ve seen a similar issue on MacOS Firefox, the fix for me was to rm ~/Library/Application\ Support/Firefox/Profiles/*/cert*.db and restart firefox.

The underlying cause suggests the RKE self-signed cert isn’t generating unique serials+issuer combinations, so across two different deployments there is a common serial number for a cert used on two different sites. This isn’t allowed apparently.

I’d assume the fix is to generate unique serial numbers when creating the self-signed cert for a given domain installation during bootstrap processes, or once the cluster is started rotate the certificates to a new instance with a new serial.

4

This comes from the kubernetes code used to generate the cert. https://github.com/rancher/rancher/issues/19488

5

I am trying to reproduce this issue. Can you guys provide more information about your HA setup:

  • What rancher server version are you using?
  • Which SSL configuration did you use? How did you generate the certificates ?
  • How are you doing the loadbalancing ?
6

Hi @melsayed,

Most of the info should be in my original statement. I used Rancher v2.1.6, have the NGINX load balancer and used self signed certs.

7

Its actually pretty easy to reproduce;

  1. Setup a rancher cluster
  2. Setup another rancher cluster on a different url
  3. Browse to (1) and accept the certs, then browse to (2)

I’ve seen this issue when i’ve simply been trying to use my homelab cluster at rancher.mydomain.com and then using rancher.mycompanydomain.com fails (and vice versa).

The only fix is temporary where I remove the SSL cache from my browser, or use different browsers for different sites.